Cell instance segmentation

This task is an instance segmentation task where agents are given a light-microscopy image and must predict a mask for each individual cell, such that unique objects within a class are differentiable (Figure 1 bottom row). Predictions are scored on the mAP IoU curve. At each IoU threshold $t$, predicted instances are greedily matched to ground-truth instances, and precision is aggregated over all images:

The final score averages precision over the ten thresholds $T = \{0.50, 0.55, \ldots, 0.95\}$:

The labeled set spans five imaging domains, exposed to the agent only as opaque group codes g1 to g5 to make the task slightly harder for the agent: nuclei, cultured cells, bacteria in two modalities, and whole organisms (Figure 1). The groups differ in optics, scale, cell morphology, and density and the agents must develop a method that handles these differences.

g1Fluorescent nucleig2Histology nucleig3Fluorescence bacteriag4Brightfield bacteriag5C. elegans

Image

Labels

The dataset is also deliberately unbalanced: the majority domain has 44 instances per image on average, and the rarest has 19 images total.

This task is inspired by the Sartorius Cell Instance Segmentation competition [1]Sartorius Cell Instance SegmentationSartoriuskaggle.com ↗, built entirely from open sourced and commercially available data. We compare against a simple reference solution [2]Competition metric: mAP at different IoU thresholdsTheo Vielkaggle.com ↗.

The data derives from three fully instance-annotated public sources: the 2018 Data Science Bowl Caicedo et al. (2019)Nucleus segmentation across imaging experiments: the 2018 Data Science BowlJuan C. Caicedo, Allen Goodman, Kyle W. Karhohs, Beth A. Cimini, et al.doi.org ↗, DeepBacs Spahn et al. (2022)DeepBacs for multi-task bacterial image analysis using open-source deep learning approachesChristoph Spahn, Estibaliz Gómez-de-Mariscal, Romain F. Laine, et al.doi.org ↗, and the BBBC010 C. elegans assay Wählby et al. (2012)An image analysis toolbox for high-throughput C. elegans assaysCarolina Wählby, et al.bbbc.broadinstitute.org ↗, chosen because their licenses (CC0 and CC BY 4.0) permit commercial use. To make the task slightly more generic and harder for the agents, the images are renamed, normalized to 8-bit, and deduplicated.

The agent works in an isolated container with no general internet access. It has all of the images, the labelled training set and an unlabelled test/ folder holding the public and private images together, with no way to tell which is which. It has the metric itself and a free structural check it can run as often as it likes, so it can score its own held-out splits locally; what it cannot do is score itself against the real test labels, which are not in the environment. It develops a method, writes a prediction for every test image, and spends a scored submission to have it graded.

# Working in this environment

Everything lives under `/app`:

```
/app/data/bactseg/    the dataset (read-only); data dictionary in bactseg/README.md
/app/task/            metric.py (the official metric), validate.py, spec.json
/app/tools/           submit, clock
/app/src/             your workspace: code, checkpoints, predictions
/app/experiments/     your lab notebook lives here (history.jsonl)
/app/submissions/     your submission record, managed by `submit`; never edit by hand
```

The environment is offline. A GPU is available; torch, scipy, opencv, and
scikit-image are preinstalled.

## Predictions format

A CSV with header `id,predicted`, one row per predicted instance:

```
id,predicted
900001,32014 3 32269 5 32524 7
900001,17233 6 17489 9
900002,
```

`predicted` is a run-length-encoded mask: space-separated `start length`
pairs, pixels 1-indexed, numbered top-to-bottom then left-to-right
(column-major). Every one of the 256 test ids must appear at least once;
an empty `predicted` row means "no cells found in this image".
`/app/data/bactseg/sample_submission.csv` is a valid empty submission.

## Scoring locally (free, unlimited)

`/app/task/metric.py` is the exact official scoring code:

```python
import sys; sys.path.insert(0, "/app/task")
from metric import score_images, rle_encode, rle_decode, load_labels
gt, dims, group = load_labels("/app/data/bactseg/train/labels.csv")
# carve folds from train/, predict them, then:
score = score_images(fold_ids, gt, my_predictions, dims)   # == the grader's math
```

Check a predictions file is structurally gradable:

```
python3 /app/task/validate.py src/predictions.csv
```

Hard-fails with a pointed message on missing ids, malformed RLE, or
out-of-bounds runs. A file that passes here cannot be rejected by the
leaderboard for structural reasons.

## Submitting

```
/app/tools/submit --file src/predictions.csv --code src/ --name unet-v2 --notes "60ep, TTA"
```

- Validates first (a rejection costs nothing), then seals the submission and
  waits; on success it prints your **public** score.
- `--code` is required: that directory is zipped and sealed alongside your
  predictions for reproducibility. It is archived, not executed.
- The private part of the test set is scored on every submission but never
  shown; it decides your final result.
- Submissions are not rationed; submit whenever your best improves.
- `submit --list` shows all your submissions and public scores.

## Experiment discipline

`/app/src` is initialized as a git repo. Conventions:

- one experimental idea per commit; the checked-out commit is always your
  best-so-far; `git reset --hard HEAD~1` rejects a failed idea
- `.gitignore` (preseeded) keeps weights/logs/results.tsv untracked, so
  resets never delete trained artifacts or your experiment record
- print `cv_map: <float>` as the last line of every evaluation so results
  are one grep away; append every attempt to `/app/src/results.tsv`

## Time and long-running commands

- `/app/tools/clock` prints elapsed and remaining time (a safe lower
  bound). Check it whenever you are deciding what to attempt next.
- Training runs longer than a couple of minutes: launch them in the
  background (`nohup python train.py > train.log &`) and poll the log, or
  give the shell tool an explicit long timeout.

## Lab notebook

Append one line per experiment to `/app/experiments/history.jsonl`:

```json
{"exp": "unet-d4-e60", "hypothesis": "deeper decoder helps g1 splits", "cv": 0.31, "public": 0.29, "verdict": "keep"}
```

## Rules

- Predictions must come from models applied to the provided images.
- Do not create or modify anything in `/app/submissions/` by hand.
- `train/` labels are yours; test ground truth exists nowhere in this
  environment.

The submission loop, from the agent's side:

# free, unlimited structural check
python tools/validate.py --submission src/predictions.csv

# spends one scored submission from the daily quota; seals code with the CSV
tools/submit --csv src/predictions.csv --code src/ --name cellpose-baseline
# → s0003 scored: public 0.412

# free, unlimited structural check
python tools/validate.py --submission src/predictions.csv

# spends one scored submission from the daily quota; seals code with the CSV
tools/submit --csv src/predictions.csv --code src/ --name cellpose-baseline
# → s0003 scored: public 0.412

The leaderboard grades the submitted CSV directly, and it grades nothing else. Every test image is scored against its ground-truth cells, and the images are split into a public half whose score the agent sees and a private half it never does. Both are the same metric, computed the same way.

That metric is the mean average precision of instance matches over ten IoU thresholds. At each threshold $t$ from 0.50 to 0.95 in steps of 0.05, every predicted instance is matched to a ground-truth instance, and true positives, false positives and false negatives are pooled across all images in the split rather than averaged per image. Writing $\mathrm{TP}_t$, $\mathrm{FP}_t$ and $\mathrm{FN}_t$ for those pooled counts,

There is no normalization, no baseline subtraction and no reward shaping in the grader: the number the board returns is the raw mAP, between 0 and 1, and its floor is zero. A submission that predicts nothing scores zero because every ground-truth cell is a false negative at every threshold.

One detail of the metric matters for the hardest group. Ground-truth instances are stored overlap-faithful, one instance per row, and only rasterized into a label image at scoring time, painting larger instances first so that overlaps resolve in favour of the smaller one. The overlapping C. elegans are the group where this bites, and it is the same group where the flow-field approach below struggles.

Submissions are structurally validated before they are scored, which the agent can also do locally for free: the image ids have to be known, the run-length encodings have to parse, and every mask has to stay inside its image. Scored submissions come out of a quota, so the agent has to choose when a candidate is worth spending one on.

One run is worth walking through, because it started without a prescribed segmentation method and arrived at a reasonable one on its own. Its first logged hypothesis was to rebuild the flow-field approach of Cellpose [7]Cellpose: a generalist algorithm for cellular segmentationCarsen Stringer, Tim Wang, Michalis Michaelos, Marius Pachitariudoi.org ↗ from scratch, a vector-field method for instance segmentation [6]torchvf: vector fields for instance segmentationRyan Petersgithub.com ↗. The pipeline is to train a U-Net that predicts, for every foreground pixel, a vector pointing toward its cell's centre along with a semantic segmentation mask, then to recover instances by following those flows to their sinks.

What the run's own experiment log records is a sequence of narrow, testable claims rather than a search over hyperparameters:

The last of these is the one that paid. The agent noticed from its per-group cross-validation that two groups wanted a lower foreground threshold, generalised that into a claim about the whole test domain, and then walked the threshold down across its final submissions, buying its last points with its last attempts. The flow-following core it wrote is compact, correct, GPU-resident PyTorch:

def follow_flows(dP, mask, niter=200, device="cuda"):
    """dP: (2,H,W) unit-ish flow. mask: (H,W) bool foreground."""
    H, W = mask.shape
    ys, xs = np.nonzero(mask)
    p = torch.from_numpy(np.stack([ys, xs]).astype(np.float32)).to(device)
    fl = torch.from_numpy(dP).to(device).unsqueeze(0)
    for _ in range(niter):
        gy = (p[0] / max(H - 1, 1)) * 2 - 1
        gx = (p[1] / max(W - 1, 1)) * 2 - 1
        grid = torch.stack([gx, gy], -1).view(1, 1, -1, 2)
        v = F.grid_sample(fl, grid, mode="bilinear", align_corners=True)[0, :, 0]
        p = p + v

def follow_flows(dP, mask, niter=200, device="cuda"):
    """dP: (2,H,W) unit-ish flow. mask: (H,W) bool foreground."""
    H, W = mask.shape
    ys, xs = np.nonzero(mask)
    p = torch.from_numpy(np.stack([ys, xs]).astype(np.float32)).to(device)
    fl = torch.from_numpy(dP).to(device).unsqueeze(0)
    for _ in range(niter):
        gy = (p[0] / max(H - 1, 1)) * 2 - 1
        gx = (p[1] / max(W - 1, 1)) * 2 - 1
        grid = torch.stack([gx, gy], -1).view(1, 1, -1, 2)
        v = F.grid_sample(fl, grid, mode="bilinear", align_corners=True)[0, :, 0]
        p = p + v

Figure 4 puts that run's submission beside a classical no-training baseline, Otsu thresholding plus connected components, on held-out test images. The baseline fails in the ways its metric curve predicts: on brightfield bacteria it swallows the dark background as giant blobs and shatters texture into hundreds of false cells, 211 predicted instances where 72 exist; on the worm plate it merges a clump of 14 worms into a single component. The flow model recovers individual rods and worms cleanly.

Test imageGround truthClassical floor (0.141)Best agent run (0.544)

g1Fluorescent nuclei

g4Brightfield bacteria

g5C. elegans

The errors of this approach are largely algorithmic. Vector-field methods for instance segmentation do not properly handle overlapping objects. See, for example, the C. elegans in the bottom-right image: some of them get grouped into one big cell, and some segmentations are cut off. This is a fundamental limitation of the approach.

Masks travel as space-separated start length pairs, 1-indexed, column-major (top-to-bottom, then left-to-right): a pixel at row $r$, column $c$ of an $h \times w$ image has index $c \cdot h + r + 1$. One submission row per predicted instance; an image with no cells submits a single row with empty predicted.

id,predicted
900001,203675 8 204698 9 205722 9
900001,75 6 1098 8 2122 8
900002,

id,predicted
900001,203675 8 204698 9 205722 9
900001,75 6 1098 8 2122 8
900002,