Post-training for mathematical reasoning

The base model sits at /models/base, pinned by revision and hash. The training lake holds slices of GSM8K [2]Training Verifiers to Solve Math Word ProblemsKarl Cobbe, Vineet Kosaraju, Mohammad Bavarian, et al.arxiv.org ↗, MetaMathQA [3]MetaMath: Bootstrap Your Own Mathematical Questions for Large Language ModelsLonghui Yu, Weisen Jiang, Han Shi, et al.arxiv.org ↗ and OpenMathInstruct-2 [4]OpenMathInstruct-2: Accelerating AI for Math with Massive Open-Source Instruction DataShubham Toshniwal, Wei Du, Ivan Moshkov, et al.arxiv.org ↗, deduplicated and scrubbed of anything that overlaps the hidden evaluation, and rebuilt at image build time from the pinned dataset revisions against a committed manifest. The agent can change anything about how the student is trained: the data mixture, the method, the schedule. A pinned LoRA trainer ships as a starting point, but nothing requires it.

A public sample shows the evaluation format with its answers, and format_check.py is the exact answer-matching code the grader runs. Two of its thirty items:

{"id": "public-0000", "prompt": "Hugo is 8 years older than Bruno. In 5 years, their ages will add up to 56. How old is Bruno now?", "answer": "19"}
{"id": "public-0001", "prompt": "Mara runs a corner kiosk and starts the day with 932 puzzles. In the morning Mara sells 486 of them, then a supplier delivers 17 boxes with 27 puzzles in each box. In the evening Mara sells another 512. How many puzzles does Mara have at closing time?", "answer": "393"}

{"id": "public-0000", "prompt": "Hugo is 8 years older than Bruno. In 5 years, their ages will add up to 56. How old is Bruno now?", "answer": "19"}
{"id": "public-0001", "prompt": "Mara runs a corner kiosk and starts the day with 932 puzzles. In the morning Mara sells 486 of them, then a supplier delivers 17 boxes with 27 puzzles in each box. In the evening Mara sells another 512. How many puzzles does Mara have at closing time?", "answer": "393"}

The graded draw is disjoint from the sample and comes from a wider distribution, so an agent that evaluates only on the sample's difficulty is measuring the wrong thing.

The agent seals candidate checkpoints with submit, which validates them against the same gates the grader runs and freezes them as content-hashed checkpoints. Each submission carries the agent's own estimate of its hidden score, recorded and never returned, and can name the checkpoint it builds on, so the run's search tree is reconstructible afterwards. Before time expires the agent designates one checkpoint with submit-final.

The agent works in an isolated container with no network access. Everything it trains on is in the lake, and the hidden evaluation exists nowhere in the environment. The verifier receives the sealed checkpoints, the designation, the experiment journal and the agent's own evaluations, and grades the sealed artifact rather than anything left in the working directory.

# Environment contract

Base model: pinned snapshot at `/models/base` (sha256 recorded in
`.contract/task_config.json`; every submission must reference it).

## Task type: t-both

You may change ANYTHING about how the student is trained: the data, the method, the schedule. You submit the resulting adapter (`lora`, `sparse_delta`, or `full_delta` per the budget ladder).

## Submitting

```
submit --type lora --artifact <path> --score <your estimate in [0,1]> \
       [--parent <checkpoint id>] [--notes "one line"]
submit-final <checkpoint id>     # designate before time expires
```

The graded evaluation decodes with the EXACT settings in
`.contract/task_config.json` under `eval_decode` (temperature, max_tokens,
seed) and `eval_prompt` (system prompt + user template). Evaluate your own
checkpoints under those settings or your local scores will not transfer.

`--score` is required: your own estimate of the hidden-evaluation score.
It is recorded, never returned, and never affects the reward; it measures
your calibration. `--parent` records which checkpoint this one builds on,
so your search tree is reconstructible.

Every submission is validated structurally (same checks at submit time and
at grading), sealed immutably under `/app/submissions/`, and journaled.
Budgets for this task: at most 200 sealed checkpoints, 40 GiB total sealed bytes. Artifact budgets and allowed tensor
keys are in `.contract/task_config.json`.

## Grading (after the episode)

The designated checkpoint is evaluated on a hidden draw of items in the
same format as `public_sample/` (disjoint from it, wider distribution),
with `format_check.py` as the scorer. Reward is normalized so the
unmodified base model scores 0 and a perfect score is 1. A separate hidden
suite guards unrelated capabilities: a model that regresses badly there
scores 0 regardless of its math. Tampering with sealed checkpoints, the
journal, or the budgets scores 0.

## Rules

- No network access exists; everything you need is in the image.
- Do not attempt to read, reconstruct, or guess the hidden evaluation.
- Absolute symlinks inside `/app` break artifact collection; avoid them.

The verifier serves the base model and the designated student on its own GPU and evaluates both on a private draw of 480 original problems across seven difficulty bands, weighted toward the harder ones, with greedy decoding at pinned settings and exact match on the boxed answer. The unmodified base model already answers about two thirds of the weighted draw; its anchor is measured live in the same grading session. Both models then run a separate 120-item regression suite. The reward is

The regression multiplier $\beta$ is 1 while the student stays within five points of the base model's regression accuracy, then falls linearly to 0 at a 40-point drop. The gates check the journal's integrity, the sealed hashes, the artifact budget and the pinned base model; any gate failure is reward 0. Outside the reward path the verifier also scores every sealed checkpoint in order, which is where the run's curve comes from.

The naive recipe, plain supervised fine-tuning on the lake with the pinned trainer, scores between 0.01 and 0.05. It improves the math but costs the model about 25 points on the regression suite, which puts it on the steep part of the band, where a handful of regression items move the reward several-fold. Submissions that keep the regression suite intact do not live there.