Are run and evaluation the same thing?

Nearly. A run is the activity; an evaluation is the unit of allowance it spends. One run spends one evaluation.

Is a run the same as a version of the dataset?

A run is what you see. Dataset versions are internal and never appear on a public page.

REFERENCE

The words this platform uses, and exactly what each one means here.

Term	Means here
Index	The weighted percentage that ranks a model. Also called the VetEval Index.
Item	One examination question with its choices and its correct answer.
Item bank	The held-out set of items. Never published, never sent to a vendor for inspection.
Run	One evaluation of one model. It draws its own random sample of items in the governed category proportions, answers each of them three times, and is scored under the configuration recorded against it.
Run	One evaluation of one model. It spends one evaluation from your allowance.
Evaluation	The unit of allowance. One run costs one.
Epoch	One repetition of the paper inside a run.
Submission	A model as you registered it: name, endpoint, credential, declaration.
Provenance	Your declaration of where the model came from. See provenance.
Verification tier	How the run was carried out, from VetEval-verified down to self-reported.
Safety component	The positive scoring axis for harm avoidance, weighted at 10 percent.
Organization	Your company on VetEval. Owns models, runs, results and billing.
Public run	A run declared before it starts as one that will reach the leaderboard.
Private run	A run visible only to your organization. Never on the board.

Was this page helpful?