Regulatory DNA design

We task the agent with designing four short DNA sequences that drive transcription as strongly as possible. Its workspace is a library for sequence-to-function models of gene regulation, and one function inside it is graded. It receives a scoring model, a query budget, a sequence length and a number of designs, and returns that many sequences.

The scoring model is frozen. The agent never trains it and never sees its weights. It reaches the model only through a callable that maps sequences to predicted activity in two promoter contexts, developmental and housekeeping. Every sequence scored counts against the budget. The graded call asks for four sequences of 249 bases on the developmental output, under 40,000 queries. A search that overruns its budget earns nothing for that submission. Everything runs on four CPU cores with no GPU.

The search itself is what the agent controls, and any method is allowed that fits in NumPy, PyTorch and the standard library. The same function is called with other tracks and budgets as well, so it cannot be shaped around one regime. It has to be deterministic given its arguments, and it has to raise on arguments that make no sense.

The agent commits its workspace and submits as often as it likes. Each submission is graded elsewhere, and a public score comes back a few minutes later. The number the run is scored on is computed the same way on a different held-out set, and it is never shown.

Skip this section if you already know why designed enhancers are hard to trust.

An enhancer is a stretch of noncoding DNA that raises transcription of a nearby gene. Its strength comes from the transcription factor motifs it carries and from how those motifs are arranged. Networks trained on reporter assays predict that strength well enough to run the problem backwards: sequences designed against a network can outperform natural ones in the laboratory (de Almeida et al., 2022, Taskiran et al., 2024). Programmable regulatory elements have real uses in gene therapy.

The catch is that a network is not the biology. Push a search hard enough against one frozen model and the winner exploits that model instead of the grammar underneath it. Published design work therefore validates candidates outside the model that produced them (Gosai et al., 2024). The quantity worth measuring is how much of a predicted gain survives an independent model. Sample efficiency matters for the same reason: measurements are expensive, so a search that needs unbounded queries is not a usable method.

Three held-out models score the four submitted sequences. None is reachable during the search, and none is present on the agent's machine. Each design takes the lowest of the three predictions, so a sequence has to convince the strictest model rather than the most generous one. Averaging those minima over the four designs gives the activity number, a z-score against natural regulatory sequences.

The activity number is normalized against a reference solution:

The naive anchor is 0.5, the level reached with no real signal. The saturation point is 26.31, placed so that our reference implementation lands at 0.70. That reference screens natural regulatory sequences through the model, keeps the best of them, and evolves them one or two substitutions at a time.

Activity carries almost all of the weight. The remainder comes from eight checks that the search rejects impossible arguments: a zero budget, an unscoreable length, a track that does not exist. A submission that satisfies the output contract but does no real search scores zero. So does one that returns the wrong count, the wrong length, characters outside A, C, G and T, or overspends its queries. Four further conditions each cut the whole score to a quarter. Among them are scoring artifacts left in the submitted workspace, and two replays of one call returning different designs.