What if my provider has no retention control?

Then it cannot be evaluated, and the connection test says so before you spend anything. This is not negotiable per customer: the item bank protects every score on the board, not just yours.

Does this apply to private runs too?

Yes. The items are the same items.

Do you keep the answers my model gave?

Yes, as the evidence for the score, and never with the question text alongside them. See [what we store](/docs/what-we-store).

SECURITY AND DATA

Zero data retention

Why every request carries retention parameters, and what happens if a provider refuses them.

Every request an evaluation makes to your endpoint carries the retention parameters your provider offers, asking that the exchange is not stored or used for training.

Why it matters here specifically

A run sends held-out examination items to a third party. If that third party keeps them, the item bank leaks into somebody's training data, and every score measured against it afterwards is worth less. The protection is for the benchmark as much as for you.

It is enforced, not requested

The parameters are set by the platform in the worker, not by anything a submission can influence.
A provider that declares support and then rejects the parameters fails the connection test, with a message naming what was refused.
A run that cannot be carried out under the policy does not start, and nothing is spent.

Questions we get asked

Was this page helpful?