ECG image digitization
Reconstruct all twelve millivolt traces from one image of a printed ECG, photographed, scanned or rendered, scored by aligned signal-to-noise ratio on a hidden split.
- Claude Opus 5
- GPT-5.6 Sol
- Gemini 3.7 Flash
- Kimi K3
- Grok 4.6
Enormous archives of electrocardiograms exist only as paper: printouts filed, scanned or photographed, with the underlying voltage traces lost, and none of it usable by software that needs signals rather than pictures. This task hands the agent images of printed 12-lead ECGs with their true signals, the official metric, an L40S and four hours, and grades it on how accurately it recovers the twelve traces from a single image of a page it has never seen, across clean renders, scans of damaged paper and handheld phone photos. Classical image processing plateaus quickly on the degraded and photographed pages; the task expects trained models.
The task
A record is one 12-lead ECG acquisition, ten seconds long, sampled at 250
to 1,025 Hz, printed on a page in the standard three-by-four layout: each of
the twelve leads is visible for one 2.5 second column, and a lead II rhythm
strip runs across the full ten seconds. Every training record
carries its signal table and nine images of the same printed page: a clean
digital render, colour and black-and-white scans, a phone photo of the page
and of the page shown on a laptop screen, photos of a stained and of a
heavily damaged printout, and colour and black-and-white scans of a
mold-degraded one. Every test record is one JPEG drawn from those same
conditions, unlabelled. The deliverable is a predictions file with the
reconstructed samples for every lead of every test record: fs × 10 rows
for lead II and floor(fs × 2.5) for each other lead, the samples of that
lead's visible window.
Environment
The agent works in an isolated container with no general internet access, on an
L40S with four hours on the clock. It has the training records, each carrying its
signal table alongside nine images of the same printed page, and an unlabelled
test folder whose records are one JPEG apiece. It also has the official metric,
shipped verbatim, so it can hold out its own training records and score itself
against exactly the code the grader runs; local validation and scored
submissions are both unlimited. What it cannot see is the split: the held-out
records are divided into a public part whose score submit returns and a
private part that is scored on every submission and never shown, and nothing in
the environment distinguishes them.
# Working in this environment
Everything lives under `/app`:
```
/app/ecgdig/ the dataset (read-only), data dictionary in ecgdig/README.md
/app/tools/ metric.py, validate.py, submit, clock
/app/src/ your workspace: code, checkpoints, predictions
/app/experiments/ your lab notebook lives here (history.jsonl)
/app/submissions/ your submission record, managed by `submit`, never edit by hand
```
The environment is offline. A GPU is available; torch, torchvision, scipy,
opencv, scikit-image, pillow, and pandas are preinstalled.
## The task
Each test record is ONE image of a printed 12-lead ECG. Recover the
underlying voltage time series in millivolts. The printed page uses the
standard 3x4 layout plus a lead II rhythm strip: lead II covers the full
10 seconds; every other lead is visible for one 2.5 second column (I, II,
III in the first quarter, aVR, aVL, aVF in the second, V1, V2, V3 in the
third, V4, V5, V6 in the fourth). Calibration is the standard 25 mm/s and
10 mm/mV against the background grid. `test.csv` gives the sampling
frequency `fs` and exact `number_of_rows` expected per record and lead.
## Predictions format
A CSV with header `id,value`, one row per predicted sample:
```
id,value
5001_0_I,0.0421
5001_1_I,0.0388
...
5001_0_II,-0.012
```
`id` is `{record}_{row}_{lead}`: the test record id, the zero-based sample
index within that lead's window, and the lead name. Lead II expects
`fs * 10` rows; the other eleven leads expect `floor(fs * 2.5)` rows, the
samples of that lead's visible 2.5 second column window. `value` is the
voltage in mV. Every expected id must appear exactly once; values must be
finite. `/app/ecgdig/sample_submission.csv` is a valid all-zeros
submission.
## Scoring locally (free, unlimited)
`/app/tools/metric.py` contains the exact official scoring code. The score
is an aligned signal-to-noise ratio in dB, higher is better: per record,
each lead's prediction is aligned to the truth (best cross-correlation
shift up to 0.2 s, then constant vertical offset removed), signal and
noise powers are summed over the 12 leads, and the final score is
`10*log10(mean over records of per-record SNR)`. A perfect reconstruction
scores 60 dB (capped); an all-zeros submission scores near 0 dB; adding
noise or misreading amplitudes drives the score negative.
```python
import sys; sys.path.insert(0, "/app/tools")
import pandas as pd
from metric import score, score_records
# build fold ground truth from train/ records you held out, in submission
# format (id,value plus an fs column), predict them, then:
s = score(solution_df[["id", "fs", "value"]], my_pred_df[["id", "value"]], "id")
```
Check a predictions file is structurally gradable:
```
python3 /app/task/validate.py src/predictions.csv
```
Hard-fails with a pointed message on missing ids, duplicates, unknown ids,
or non-finite values. A file that passes here cannot be rejected by the
leaderboard for structural reasons.
## Submitting
```
/app/tools/submit --file src/predictions.csv --code src/ --name unet-v2 --notes "..."
```
- Validates first (a rejection costs nothing), then registers the
submission and waits; on success it prints your **public** score.
- `--code` is required: that directory is zipped and sealed alongside your
predictions for reproducibility. It is archived, not executed.
- The private part of the test set is scored on every submission but never
shown; it decides your final result.
- You have unlimited scored submissions and unlimited local validation and cross-validation. Invalid and rejected attempts never count against any
limit.
- `submit --list` shows all your submissions and public scores.
## Experiment discipline
`/app/src` is initialized as a git repo. Conventions:
- one experimental idea per commit; the checked-out commit is always your
best-so-far; `git reset --hard HEAD~1` rejects a failed idea
- `.gitignore` (preseeded) keeps weights/logs/results.tsv untracked, so
resets never delete trained artifacts or your experiment record
- print `cv_snr: <float>` as the last line of every evaluation so results
are one grep away; append every attempt to `/app/src/results.tsv`
## Time and long-running commands
- `/app/tools/clock` prints elapsed and remaining time (a safe lower
bound). Check it whenever you are deciding what to attempt next.
- Training runs longer than a couple of minutes: launch them in the
background (`nohup python train.py > train.log &`) and poll the log, or
give the shell tool an explicit long timeout. Do not let a default
2-minute timeout chop a 20-minute training into a dozen polls.
## Lab notebook
Append one line per experiment to `/app/experiments/history.jsonl`:
```json
{"exp": "unet-c32-e20", "hypothesis": "trace segmentation transfers to photos", "cv": 8.3, "public": 7.9, "verdict": "keep"}
```
## Rules
- Predictions must come from methods applied to the provided images.
- Do not attempt to identify source recordings or reconstruct ground truth
from memory of public datasets.
- Don't create or modify anything in `/app/submissions/` by hand.
- `train/` signals are yours; test ground truth exists nowhere in this
environment./app, with the files it is given. The official metric ships verbatim, so it can score its own held-out records as often as it likes; what it cannot see is which of the test records are public and which are private. The images and signal tables are in the container but omitted here, and the task instruction is delivered as the agent's prompt rather than as a file.Evaluation
The leaderboard grades the submitted predictions file directly, on the public records for the score the agent sees and on the private records for the score that counts. Both are the same metric, and there is no normalization in the grader: the number the board returns is the raw aligned signal-to-noise ratio in decibels.
Per record, each lead's prediction is first aligned to the truth by cross-correlation within a shift of at most 0.2 seconds, with a vertical offset fitted at the same time. Signal and noise powers are then summed over the twelve leads, and the per-record ratio is that sum of signal power over that sum of noise power. The score is ten times the base-ten log of the mean of those per-record ratios:
Two bounds come out of the implementation rather than the formula. Each per-record ratio is capped at , so a perfect reconstruction scores 60 dB and no single record can carry the mean beyond it. And the returned value is floored at dB, so the metric is bounded below in code even though a log ratio is not bounded below in principle.
That lower bound is a long way below anything a submission should reach. An all-zeros predictions file scores near 0 dB, because the noise power then equals the signal power and the ratio is one. A score below zero therefore means a reconstruction that is actively worse than predicting nothing, which is why we clamp at 0 dB and treat that as this task's floor going forward: negative scores carry no information we want to propagate into the normalized scale.
When the run ends the verifier regrades every sealed submission from the raw files with its own copy of the ground truth, takes the submission with the best public score, and reports its private score.