Skip to main content

Normalization scale

A task's normalized score is its final result minus the task's floor, divided by the average final result across the models on the roster, so 1 is an average score. A run's final result is the best private score it reached, read off the private split on its own terms rather than through whichever submission led the public board. Both halves of the scale move as rollouts land, so they are frozen (2026-09-02) and reported here rather than recomputed on the fly. A run's whole clock counts toward its final result, whatever length its campaign ran.

The floor is the metric's own lower bound rather than any chosen baseline, and it is zero throughout; aligned SNR, the one metric with no lower bound, clamps there. The divisor is the average of each roster model's mean final result above the floor, per-model means first, so a model that ran more rollouts does not move the scale. A task that runs on more than one board carries a divisor per board, since each board is its own measurement.

additive-smoothingjudge-gambench-e1-smoothjudge score0.000000.70776214
attribution-speedupar-lat-gpu-benchgraphs/s x memory factor, attribution GPU0.0000041.532195
attribution-speedupar-tracer-gpu-benchgraphs/s x memory factor, tracer GPU0.0000041.403215
aux-channel-protocoljoseph-owlbench-for-autoresearch-benchmarkattributed-transfer ladder score0.000000.79146215
cell-segar-cs-templatemAP over IoU 0.50-0.950.000000.36106205
cheri-triton-trainingar-cheri-for-autoresearch-benchmarktraining-step speedup0.000001.876275
dbscan-speedupar-dbscan-cpu-gpu-benchMpts/s x memory factor, CPU + GPU0.00000150.253215
ecgdigar-ecgdig-benchaligned SNR (dB)0.00000clamped6.492175
hutterjudge-hutter-env-rebuildcompression score, xz 0 to scaled record 10.00000clamped0.31872205
knowledge-editar-knowledge-edit-benchgated ratio0.000000.39739154
lintrackar-lintrack-benchtrack score0.000000.59714225
manifold-representationjudge-manifold-representationjudge score0.000000.78556183
ogbg-code2ogbg-code2-fulltoken-set F1 on the official test split0.000000.19371245
ogbl-biokgogbl-biokg-fullrescaled MRR on the official test split0.000000.85193164
opcdar-opcd-benchgated ratio0.000000.55565255
pca-speedupar-pca-benchspeedup over full scan x memory factor0.0000016.403155
post-trainingar-post-training-benchgated ratio0.000000.28016225
qwen-mtp-speedupar-mtp-benchtokens/s x memory factor0.0000026.029195
reasoning-adapterar-reasonadapt-benchexact-answer accuracy0.000000.64447225
regulatory-dna-designjoseph-regulatory-dna-design-for-autoresearch-benchmarkheld-out expression oracle score0.000000.85903285
shampoo-parityar-shampoo-benchscore on the shipped-to-reference line, unclipped0.000001.160155