Sparse additive representation learning

The agent builds a representation learner from scratch. Each observation is one wide row: a few reusable latent objects from a pool the agent never sees are scaled, added together, and covered with observation noise. The job is to invert that. The learner must say which objects an unseen row contains, isolate what each object contributed, and keep each object's meaning stable from row to row.

The deliverable is two executables, fit and transform. fit reads one unlabeled training matrix and a short config. The config states the matrix shape, a cap on how many objects the learner may keep, and nothing else: no object count, no typical density, no labels. transform must then emit four arrays for every evaluation row: a reconstruction, a nonnegative presence strength per learned object, each object's additive ambient contribution, and one shared bias. The contract pins an exact identity

The number of learned objects $G$ is chosen at fit time and stays fixed. Presence rankings must agree with which contributions are active. Inference must be deterministic and row-independent: a row processed alone gets the same representation it gets inside any batch.

The environment is offline and nothing can be installed. NumPy, SciPy, and CPU PyTorch are available, but scikit-learn and any dictionary-learning or manifold-learning package are off-limits: the learner is handwritten. Runtime is part of the contract. On four CPU cores and 8 GiB of memory, fit gets 75 seconds and each transform call 15, with a modest extra allowance at the largest scale. Exceeding a limit on any dataset discards that dataset's result entirely.

Interpretability research models neural activations as sparse additive sums of reusable features, and sparse autoencoders are its standard unmixing tool. On real activations that work cannot be graded, because the true features are unknown. Source separation and spectral unmixing share the same shape and the same problem. Here the question comes with ground truth: every observation is generated from known objects,

Each object $m_i$ is a small curved shape (cycles, tori, and helices among them), bent by a hidden nonlinear lift and placed into the wide ambient space by a hidden frame. Each row activates a random small set $S$ of objects with signed, heavy-tailed amplitudes $a_i$. Some rows are empty, some carry a single object, and some are unusually dense.

The graded question is factorization: one learned object per true object, covering that object's full range of variation. Curvature invites shattering, since a circle tiles neatly into flat local detectors. Co-occurrence invites merging, since two objects that often fire together look like one. The task punishes both. Sparsity is what makes the problem solvable at all: rows where an object appears nearly alone are the purest evidence of its existence.

Reconstruction is deliberately not what the task pays for. Tiling one object's pattern across many unrelated features rebuilds the data almost perfectly and discovers nothing, so reconstruction is reported but earns nothing on its own. Credit goes to recovery: the true factors come back in every data regime, the representation stays compact, and each object's additive contribution is isolated exactly.

Grading runs nine hidden settings that vary the geometry, the mixing statistics, and the scale. Eight draw their objects from an analytic zoo; one derives its coordinates from real language-model activations and aligned drone imagery. One setting makes test rows denser than anything seen in training. Private evaluation also includes paired rows identical except for one resampled object: a coherent learner localizes that change in one contribution and leaves the others still.

Every metric is held out and continuous, scored as the fraction of a span crossed,

clipped to the unit interval. The floor is what the strongest measured non-solution reaches unaided; the target is what a reference solver demonstrates on calibration draws it never grades against. Credit begins where a plausible non-solution stops.

Factor recovery pays the most — one learned contribution recovering one complete true object — followed by structural coherence, presence discovery, compression, and a small efficiency term. The headline score also reads the worst regimes, not just the average: mastering eight settings and dying on the ninth costs real credit, and abandoning a regime cannot be hidden. Learned-to-true matching and presence cutoffs are fixed on splits the grader never scores, so no threshold is ever chosen against the graded rows.

Strong runs treated the missing ground truth as the first problem to solve: they wrote their own generators of the described process and graded their designs against that controlled truth. Weak runs shipped learners that reconstructed well and recovered nothing.

The best run built a sparse dictionary learner and grouped its atoms into objects by co-activation. It then fused fragments with an amplitude test: pieces of one object rise and fall together, while distinct objects that merely co-occur do not. Its sharpest move was diagnostic. It fed its simulator's true subspaces into its own detection stage, found detection nearly perfect, and concluded the entire remaining gap sat in unsupervised discovery. Its own tests also caught a contract bug: a row's answer changed with its batchmates through a shared early-exit flag. After measuring fit at 73.7 seconds against the 75 second budget, it rebuilt its time budgeting and finished with margin. Total working time: under eighty minutes.

A second strong run found a cleaner geometric route: after whitening, independent objects occupy orthogonal spans, so discovery becomes a rotation search plus a partition of coordinates. That carried eight of nine settings to near their targets. The ninth collapsed. In that setting distinct objects habitually co-occur. The run's grouping statistic reads co-firing as same-object evidence, so it fused objects it should have separated and factor recovery landed near the floor. The breadth rule did the rest, adjusting a raw 0.89 down to a final 0.73 over just under three hours of work.

The zero runs were not broken submissions. Both passed the program contract, and one held reconstruction above 0.89 on eight of nine settings. One spent under four minutes: it exercised its entry points on the placeholder noise that ships for format checks, ran a single quick synthetic check on a small gated autoencoder, and declared the system complete. The other spent ninety minutes reasoning itself out of subspace models and into a stiff curve-tracing model. Its closing note was honest: initialized at the truth the model held, but discovery from data did not. Every paying metric on both runs sat at or below what unaided PCA reaches, and exactly zero is what that earns.