Order from Lab F630DB

Held out evals

Expert-written problems with independently verified answers that have never been online in any language, sold exclusively for evaluation. The seller and its authors must be bound never to publish, resell or show them to a model, with a refund for any item that surfaces.

How answering works

Any number of people fill one order. The lab receives one dataset, in its own columns.

Your agent reads the order
It looks at what you hold and says which columns you can fill, and what is missing.
Mora checks what you send
Every record: no copies, no personal data, not already public, not on Mora from someone else.
The lab pays per record it accepts
The lab named the price. You fill as much of the order as you can.

The columns the lab wants

You do not need every column. Mora says which ones each dataset fills.

Column Type What it must hold

item_id id One problem, stable across deliveries

domain label Field and subfield (maths, physics, chemistry, medicine, law, software)

problem_statement text The original problem as the expert wrote it, LaTeX allowed, figures as attached files

reference_answer text The final answer, in a form a machine can grade

answer_type label exact, numeric with tolerance, choice, or rubric

worked_solution text The author's full derivation

grader text Checker script or scoring rubric

author_id id Pseudonymous id of the expert, with a credential Mora has checked

verifier_ids id list At least two experts other than the author who solved it blind

verifier_agreement boolean Whether the blind verifiers reached the reference answer

written_at date When the item was authored

draft_history text Timestamped revisions showing a person wrote it

exposure_log text Every exam, class, tool, model or buyer the item was ever shown to

canary text A marker string unique to this buyer, to trace a leak

Column	Type	What it must hold
`item_id`	id	One problem, stable across deliveries
`domain`	label	Field and subfield (maths, physics, chemistry, medicine, law, software)
`problem_statement`	text	The original problem as the expert wrote it, LaTeX allowed, figures as attached files
`reference_answer`	text	The final answer, in a form a machine can grade
`answer_type`	label	exact, numeric with tolerance, choice, or rubric
`worked_solution`	text	The author's full derivation
`grader`	text	Checker script or scoring rubric
`author_id`	id	Pseudonymous id of the expert, with a credential Mora has checked
`verifier_ids`	id list	At least two experts other than the author who solved it blind
`verifier_agreement`	boolean	Whether the blind verifiers reached the reference answer
`written_at`	date	When the item was authored
`draft_history`	text	Timestamped revisions showing a person wrote it
`exposure_log`	text	Every exam, class, tool, model or buyer the item was ever shown to
`canary`	text	A marker string unique to this buyer, to trace a leak

How much, in the lab's words

3,000 items for the first order (500 in each of six fields), then about 300 fresh items a quarter. Counted as unique problems, not rows.

items
answers
verification
authors
provenance
difficulty
grader
custody