Nablaframe: Beating the Null Isn’t Enough

Abstract #

Nablaframe is a small GPT-2-style transformer whose training run replays bit for bit, and each of its explanatory views carries a badge of what a gate measured for it. Its tools are held, gate by gate, to tasks with planted variables: induction, an in-context two-hop lookup and modular addition. Each trained model is measured beside a random-weights twin. Everything here is toy scale, with one to four layers, residual widths of 32 to 128, and no claim about large models.

Beyond the gates, we re-measured three findings on five or six training seeds per task, against bars written before the new seeds were measured. These results are tagged feasibility: they were measured outside the gates.

1. A causal meter with a null control still reads structure into random weights. Swap a set of dictionary latents after one block, and the matching set one block later at the same position follows on every twin: 35% to 56% of the time, 20 to 35 points above a null of random latents. That is below the 80% the gates ask of an edge, but a test that asks only "above the null?" accepts it. The residual stream passes the edit along. With the block between them frozen, the swap still follows 30% to 51% of the time, and that is the test that exposes it. Try it in Figure 4 →
2. Whether weak neuron activations can be removed depends on the model. Zeroing each neuron's activations below its 75th percentile costs induction at most 0.013 nats. It costs modular addition 2.1 to 2.5 nats: less than a random removal of the same activation mass, yet it still changes 66% to 75% of the predictions. On four of five two-hop seeds it costs more than the random removal. Each order holds against all 20 random draws on every seed. Figures 5 and 6 →
3. At one site, how well units are named doesn't predict how well they steer. At induction's second MLP, single neurons name 78% to 86% of the copied token's values, yet swapping the 16 best-named neurons moves no prediction on any seed. Sixteen transcoder latents name fewer values and move 98% to 100%. The site is one the model barely needs, so this compares two bases as steering handles, not as accounts of what the model uses. Figure 7 →

Four other bars failed, and the failures are reported. Two-hop's replacement of a working earlier route held as written on one seed of five, and its neuron account on none. A sparse edit steering past its whole component's own output cleared its bar on three seeds where four were needed. Tuned neurons out-steering a sparse autoencoder on modular addition held on no seed.

Contributions #

1. An instrument and its judged record. Models are trained the ordinary way on tasks whose variables are planted in the data. Each judged name and edge is reported beside a null and a random-weights twin, training replays bit for bit, and a gate rules on each phase before the next begins. Bars are fixed before a gate runs and rulings go in an append-only decision log. The parts exist elsewhere: ground truth by construction (Tracr, InterpBench), benchmarks of methods (MIB, SAEBench), hypothesis tests by resampling (causal scrubbing) and random-model baselines (Adebayo et al.; Heap et al.). What we add is the combination. We found no system with all of them, which is a search result. The ground truth is the task's variables, not a known circuit. Where a model holds them is found by patching.
2. A twin baseline for a causal meter (feasibility, five two-hop seeds; seed 0 recorded in G6.3). At one position, a swap meter reads a residual carry as an edge. A null of random latents doesn't catch it; a twin and a frozen-block pass do. Two remedies are measured.
3. A random removal matched on mass, beside the weak-activation ablation (feasibility, three tasks, five or six seeds each). The two removals rank differently by task, and on two-hop by seed. So a small loss from removing weak activations is a fact about one model.
4. Naming and steering measured on the same units at the same budget (feasibility, five induction seeds). Under the per-neuron threshold the neuron-reading literature uses, neurons name more values than transcoder latents on every seed, and 8 or 16 of them steer nothing. Three qualifications belong to the result. All 256 neurons together steer as well as the MLP's own output, the MLP is one the model barely needs, and under the latents' own rule (active above zero) the neurons' naming collapses on seed 0, the only seed where that was measured.
5. The bars and their outcomes, failures included (Figure 1).

The instrument #

Nablaframe is built in phases, and each phase ends at a human gate that rules on scripted measurements before the next phase starts. Phases 0 to 7 are complete (D61, 2026-11-07).

How it's built #

• The model: a GPT-2-style transformer in JAX on CPU, written with Equinox and Haliax and trained with Optax. Its layer-norm epsilon and attention scaling follow Levanter's GPT-2. Levanter and Haliax come from the Marin dev builds, pinned by the lockfile.
• Traces: an append-only store of JSONL events and safetensors shards. Tap names follow TransformerLens's hook names, so a site here has the name it would have there.
• Cross-checks: a PyTorch lane with TransformerLens and the J-lens reference, and a metaljax lane on Apple Silicon as an accelerator probe. JAX on CPU is the reference; the other two only check it.
• The viewer: one static HTML page in plain JavaScript, with no build step and no network. It re-hashes every tensor it shows against the trace.
• Environments: uv, with one lockfile for all three lanes.

The zoo #

Three tasks with planted variables, and a reference run on real text. All models have four heads per layer and dense GPT-2-style blocks.

Two-hop carries most of this report. Its intermediate b is never in the query, so the model must compute it. Figure 2 shows a window and the sites the results read.

Controls beside the numbers #

• Twin: a model with the same configuration, initialized and never trained. A tool that finds a planted variable on a twin is reading the input or the architecture, not the training.
• Null: the same intervention on randomly chosen units. For a set of latents, four random active latents are swapped for four random others at the set's amounts.
• Replay: retrained from step 0, each zoo run's final state hash is identical (G4.1, G7.1), and each dictionary kind trains byte-identically (G5.1). So a counterfactual over training can be replayed exactly on the same machine. None has been run yet.

What the tags mean #

• Judged: a gate script held the number to a bar set beforehand, and the number is in reports/. The bars come, with margin, from earlier feasibility measurements on the same specimen. So judged is a check on reproducibility and on the twin, not an out-of-sample test.
• Recorded: in a gate report, with no bar.
• Feasibility: measured outside the gates, by scripts no gate has checked. All three headline results are feasibility.

The judged record #

Every row is seed 0's specimen. The gates judge tools against one specimen's planted variables. On two-hop they don't say how the task is solved in general: Result 1 and Bars that failed show other seeds holding b elsewhere.

1A null is not a twin #

On all five two-hop twins, a swap meter clears its null where little is computed: the carry of b from block 1 to block 2 follows 35.2% to 56.1% of the time, against a null of 15.2% to 20.9%. A null asks whether this set is special among sets; a twin asks whether this model is special among models. For this causal meter, the two questions come apart.

What was measured. Two-hop's b has a set of latents in the SAE after block 1 and another in the SAE after block 2, both at ⇒. Swapping the first set to another value's makes the second set follow. On the twins the excess over the null is 20 to 35 points. That is below the 80% the gates ask of an edge, so the gates' own bar would not pass it. A test that asks only whether the meter beats its null does pass it. With block 2's attention and MLP frozen, the twin's carry still follows 29.7% to 51.4% of the time, so block 2 computes little of it: the residual stream passes the swapped set on.

The trained models, and the other twin checks #

• The trained carry exists on seeds 0, 1 and 3, is weak on seed 4 and absent on seed 2, where the dictionary names no b at that site (0% of values). So the claim rests on the twins; the trained side shows the label working where there is something to label.
• The twin clears what it should. The hop edge, which seed 0 computes (91.6%, and 3.1% through the residual alone, G6.3), stays within 2.7 points of its null on every twin. Names on the twins follow at most 2.8%.
• Some names pass on twins too. A transcoder on the twin's first MLP names 33% to 55% of token(p) values, more than the trained model's transcoder on four of five seeds. The token is in the input, and a dictionary finds it in a network that has learned nothing. Heap et al. reported the label side of this; the carry is the causal side.
• The trained hop on seeds 1 to 4 follows 0.4% to 32.8%. On those seeds, patching at the site of b moves few answers (see Bars that failed). The judged hop edge is a fact about seed 0's specimen.

Two remedies, each measured once. Label an edge at one position by its rate with the blocks between frozen, as Phase 6 does ("carried" against "computed"). Or judge only edges that cross positions, or that stop following when frozen. On the twins, the one such edge measured, the hop, stays within 3 points of its null. Either check costs one forward pass with the blocks frozen, or one randomly initialized copy of the model, and needs no ground truth.

What it doesn't show. That carries are as large in deeper models, or under another normalization or initialization. The twins are five draws of one initialization scheme at one architecture, and how much a random residual stream carries depends on how close its blocks are to the identity.

2Weak activations need a random control matched on mass #

On modular addition, removing each neuron's weakest activations costs 2.07 to 2.51 nats on every seed: less than a random removal of the same mass (4.07 to 4.66), and still enough to change 66% to 75% of predictions. Without the random removal beside it, the first number reads as weak activations mattering. With it, they matter less than an average activation of the same mass, and the model is broken either way.

The design copies AMX07's replication of "Neural Networks Are Already Clean", run on a 4-layer TinyStories model. It zeroes each neuron's activations below its 75th percentile of magnitude, beside a removal of the strongest activations matched on activation mass. Its random removal is matched on count: it drops 75% of activations, strong ones included, and costs +2.389 against +0.037 for the weak removal. The replication's own review asks for a random removal matched on mass. That removal, with 20 draws per seed, is what we added. Figure 5 shows the difference between the controls on one neuron.

The result on real models, every included seed, all MLPs together:

Random ranges are the first draw on each seed, as first measured. Figure 6 shows all twenty draws.

• Induction fits the replication's reading: its weak activations carry almost nothing, on every seed.
• Modular addition shows why the random control matters: weak below random on all six seeds, and the model still broken.
• Two-hop goes the other way on seeds 0 to 3: the weak removal costs more than random. On seeds 1 and 3 it costs more than removing the strongest activations. Seed 4 has the opposite order. This is the least stable part of the result: seed 3 ends at 92.6% accuracy with a base loss of 0.197 nats, and two-hop's loss is read on 512 rows.

Twenty random draws per seed. The random draws vary little: their range on any seed is at most 0.32 nats (two-hop seed 3), against weak-minus-random gaps of 0.06 to 2.23 nats. On new seeds alone the counts are 4 of 4, 5 of 5 and 3 of 4.

Mass matching. The random removal's mass is within 0.02 of the weak removal's on every seed but two-hop seed 2 (0.04, with random removing more, which works against the finding). The strongest removal's mass is off by up to 0.08.

What this bears on. A small loss from a weak removal needs a random removal of the same mass beside it before it says anything about weak activations. This isn't a test of the essay, whose evidence comes from image classifiers and large language models. On a medium ConvNeXt, removing the weakest 75% of activations moves accuracy from 72% to 71%, and the essay concludes that almost all weak activations can be ablated "for almost no cost". On our modular-addition and two-hop models that doesn't hold; they are one- and four-layer models on algorithmic tasks, outside what the essay studied. Our result says nothing about large models. Goodfire's technical companion, Neurons as an Interpretable Basis (Tigges et al., 2026-11-10), zeroes the weakest 92.2% of routed neuron activations in Qwen3.6-35B-A3B, and its loss rises from 1.886 to 1.940. At a matched count of 87.5%, its random removal costs +2.328 where keeping the strongest costs +0.066. Both random controls with numbers, AMX07's and Goodfire's, are matched on count. A mass-matched random control was tried before on a small sorting classifier, without reported numbers (Gadeev, 2026-10-22). Added here: the mass-matched control with numbers, across seeds, on tasks with planted variables, with the order changing by task.

What it doesn't show. What in each model makes the difference. Two-hop's seeds, whose patchable state sits at different depths, differ among themselves.

3At one site, names don't predict steering #

At induction's block 1, neurons name 78% to 86% of the copied token's values, yet swapping the 8 or 16 best-named neurons moves 0.0% of predictions on every seed. Sixteen transcoder latents, which name 58% to 77% of values, move 98.4% to 100%.

• The component can do it. All 256 neurons swapped together move 99.8% to 100%, and so does the donor patch, the MLP's own output from a row where the token is the other value. The code is in the neurons, spread over more of them than a set of 64 (4.9% to 38.1%).
• Steering isn't use, at our own site. A mean ablation of this MLP changes 0.0% to 8.8% of predictions. Attention already carries the copied token (G4.4 attributes +9.95 to the MLP and +3.35 to attention on seed 0), so the MLP's write is sufficient to move the answer and nearly unnecessary for it. In Che and Li's terms the site would score high on installation and low on removal. The comparison is between two bases as steering handles; neither number says the model relies on those units.
• The naming rule matters. Neurons win on naming only under a per-neuron threshold. Counted as active above zero, as latents are, seed 0's mean best neuron F1 falls from 0.65 to 0.20.
• The edit rule matters. Units are set to their mean for the other value. The 16-neuron edit leaves the MLP's output unusually small (1st to 6th percentile by norm), so a different edit rule could change the neuron numbers.
• A second instance, on one seed. Two-hop seed 1 holds b at ⇒ after block 2 in named latents (81% of values named as planted), and swapping them moves the answer 9.9% of the time, against 92.6% on seed 0. On seed 1 the answer is already readable at that depth (a probe for c reads 0.86), so one reading is that the name is right and the variable is no longer on the path. This was noticed after the fact.

Over-steering: seen on four seeds, below its bar #

Switch Figure 7 to block 0. The other direction, a sparse edit that steers more than its whole component can, failed its pre-registered bar (3 of 5 seeds; 4 needed). It is reported as an observation.

Induction's block 0, token(p) at every position, 512 instances per seed. Nulls follow 0.0%.

• Editing more latents removes the excess. 64 latents steer 2.5% to 13.7%.
• The dictionary's directions alone don't explain it. Writing in the transcoder's reconstruction of the donor row steers within 1.2 points of the donor patch, and every latent swapped to its mean steers within 0.7 points of every neuron swapped to its mean. What over-steers is editing a few latents and leaving the rest. We don't know why.
• Natural values unit by unit still over-steered. The edit sets each latent to a value it takes on natural inputs, and the edited output's norm is typical. Still, on four of five seeds it moves predictions more than the donor patch, the MLP's own output from a row holding the other value. Che and Li note this possibility for their own natural-value tests; their contract adds removal and rescue to natural range.
• Prior work. Comparing a subspace patch against a patch of the full MLP is Makelov et al.'s. On indirect object identification in GPT-2 small they found the same order: a one-dimensional direction in MLP 8 moved their metric by 46.7%, and patching the full MLP activation moved it by −8% (their Section 4). Ours is the comparison run on dictionary latents with a donor patch, on models with planted variables.

4Bars that failed #

Two candidates failed outright, and one finding turned out to describe one run. Each is reported because the failure changes what the single-seed result meant.

One run is one specimen: two-hop's formation #

On seed 0, two-hop answers 74.3% of held-out windows at step 5,000 while a probe for b at ⇒ reads 0.366 and the query patch moves 1.2% of answers. By step 6,000 the probe reads 0.981, G6.5's patch at position 25 moves 78.5% and the name's envelope begins (G6.5: the envelope is judged, the rows are recorded). The earlier route is seen and not explained (H6). Figure 8 uses G4.3's query patch, which moves positions 24 and 25 together; that is a different patch, and it reads 71.9% at step 6,000 (feasibility; G4.3's report gives only the final 85.2%).

As written before the seeds were measured, the test looked for the same order of events after block 2. It holds on seed 0 only. Scrub through training to see where each seed's patchable state appears.

• Seeds 1, 2 and 4 end with a patch that swaps the answer after block 1 (90.6%, 96.1%, 99.2%) and not after block 2 (9.8%, 0.0%, 0.0%).
• Seed 3 ends at 92.6% accuracy with no swap after any block.
• Seeds 2, 3 and 4 end with a linear probe for b at ⇒ reading 0.50, 0.31 and 0.31, where seeds 0 and 1 read 0.99 and 0.98.

The data show a patchable state at different depths on different seeds. They don't show different algorithms. On seeds 2 and 4 b may sit at the query's a (position 24) rather than at ⇒, or be held non-linearly; neither was measured. A probe for b at position 24, and G4.3's patch split into its two positions, would test the first of these.

A hypothesis for the next round, with its bar fixed now. Read after the fact at whichever block ends with the largest swap, four of five seeds answer 71% to 79% of held-out windows before any swap appears, then swap at least 75%. That reading was chosen after seeing the data, so it is a hypothesis, not a result. The bar for testing it: on at least 8 of 10 new seeds, held-out accuracy of at least 60% at a checkpoint where the query patch swaps at most 10% after every block, followed later by a swap of at least 75% after some block.

The neuron account failed too. On seed 0, 63% of the neurons that take up the new route already had a name, but 83% of all neurons at that block are named, so re-use is below the base rate (notebook finding 36). What holds on every induction seed is narrower: a named neuron rarely takes an unrelated name. Across induction's formation at most one does, where a shuffled null gives 5 to 42. Over the same window 14 to 97 named neurons lose their name.

Sparse dictionaries and distributed codes #

The candidate that sparse dictionaries miss modular addition's distributed code fails on every seed. With about 30 latents, G7.2's SAE steers the sum 60.5% to 77.5%, against 13.7% to 45.1% for the frequency-tuned neurons. G7.2's 0.9% (recorded) is a four-latent result. The two edits sit at different sites: the SAE rewrites the residual stream after block 0, which also carries attention's write, and on 4 of 6 seeds it goes past all 128 neurons together (35.9% to 87.9%). So the comparison doesn't rank the bases either way.

Discussion #

The three results suggest three cheap checks on what an interpretability result means. Each costs one extra model or one extra pass. None needs more ground truth than the result it checks, so each could in principle run on a larger model. We haven't tried.

For a safety reader the pitch is evidence hygiene, not a claim about large models. A twin run for a causal meter, a random removal matched on mass, and a ceiling for a steering result are each easy to leave out. On these toy models, leaving each one out would have misled on at least some seeds.

What a single run supports. Of the single-seed findings we started from, two-hop's learned-twice story and its neuron re-use account didn't survive new seeds as written. The judged two-hop results stand, but they describe seed 0's specimen. On the other seeds the patchable state sits at a different depth, or a probe at ⇒ doesn't read b. The gates were built to judge tools against one specimen's planted variables, and that is all they judge.

What stays open. Two-hop's earlier route (H6), the cause of the over-steering at induction's block 0, and why four of nine induction seeds are slow.

Using Nablaframe #

There are two ways to use this work. The three checks need nothing from Nablaframe, and Nablaframe itself is a place to test a method where the answer is known.

The three checks, on your own model #

1. A twin run. Initialize a second copy of the model with the same configuration and a seed no trained run uses, and never train it. Train any dictionaries on it the way they were trained on the real model. Run the same meter and the same null on both, and report the twin's excess over its null beside the trained model's. For an edge between two sites at one position, also rerun it with the blocks between them patched back to the unintervened pass.
2. A random removal matched on mass. Measure the share of activation mass, the sum of magnitudes, that the removal takes. Then zero each activation at random with that probability, so the random removal takes the same mass in expectation. Draw it at least 20 times, and report where the removal sits against all the draws.
3. A donor-patch ceiling. Replace the component's output with its own output from an input where the variable holds the other value. That is the most an edit inside the component should move, so a sparse edit that moves more is doing something the component doesn't. Mean-ablate the component as well, to see whether the model needs the site at all.

Nablaframe itself #

• Test an interpretability method against planted variables. Each zoo task declares the variables a model must compute, in src/nablaframe/zoo/declarations.py. The gate scripts in gates/phase4/ to gates/phase7/ show how a tool is held to them, beside a null and the twin.
• Study how a mechanism forms. Training replays bit for bit on one machine, so any checkpoint can be rebuilt and compared exactly.
• Add a task to the zoo by writing its declaration.
• Teach with it. The viewer shows a trained model's lenses, attention and training timeline in a browser, from a recorded trace.

Quickstart #

You need Python 3.12 or later, uv and a CPU. The tests run on macOS or Linux. The gate scripts, the last line among them, run on macOS only.

git clone https://github.com/apolloh0/nablaframe
cd nablaframe
UV_PROJECT_ENVIRONMENT=.venv-jax uv sync --extra jax
UV_PROJECT_ENVIRONMENT=.venv-jax uv run --extra jax pytest -q
UV_PROJECT_ENVIRONMENT=.venv-jax uv run --extra jax python gates/phase4/g41_zoo_training.py --only induction

The tests, 387 of them, took about 13 minutes on a 2-core Linux machine. The last line trains the induction model for 4,000 steps, retrains it from step 0 and checks that the two final state hashes are identical; both trainings together took just under 2 minutes on an M-series Mac.

What it isn't. Nablaframe isn't a library for interpreting arbitrary pretrained models, and it works at toy scale on a CPU.

Related work #

Random-weights baselines. Adebayo et al. (2018) introduced the randomized-model sanity check for saliency maps. Heap et al. (v2, 2026) found that auto-interpretability scores and SAE reconstruction metrics barely separate trained from random transformers, ran no interventions, and recommended randomized baselines as routine. Later work cites them as a reason to anchor metrics to causal effect, with matched random feature sets as the control (probe prompting); that randomizes the features, not the model. SAEBench's Appendix E comes nearest: it trains SAEs on a randomly initialized Pythia-1B and runs two latent-ablation metrics on them (SCR and TPP), warning that such cross-model comparisons can mislead. We found no random-weights baseline for a swap meter between two sets of dictionary latents. That is a search result, not a claim to be first. Makelov et al. argue informally that an MLP between two components that communicate through the residual stream is likely to hold a subspace that looks like a mediator. The carry is the residual stream's own version, measured on random weights.

Steering and mechanism. Makelov et al. (ICLR 2024) showed a subspace patch can reach its effect through a dormant pathway, and compared a one-dimensional MLP direction against the full MLP. Che and Li (2026) separate installation from removal and propose tests at natural values, with removal and rescue. Arad, Mueller and Belinkov (2025) split SAE features into input and output features, of which only the second steer well: the nearest relative of our names without steering, at LLM scale. AxBench (Wu et al., 2025) found SAEs uncompetitive for steering against simple baselines. MIB (Mueller et al., 2025) found SAE features no better than neurons for locating causal variables. Arora et al. (2026) and "Many Transcoder Features Are Neurons in Disguise" (2026) treat neurons as a circuit basis. Yavuz (2026, a single-author preprint) reports a 12-neuron group in Qwen3-0.6B that passes a random-neuron control, while resampling its activations from other questions changes almost nothing. He finds no evidence that the group carries question-specific content.

Weak activations. Cammarata's essay (Goodfire, 2026-10-15) holds that almost all weak activations can be ablated at almost no cost, and shows a ConvNeXt going from 72% to 71%. AMX07's replication repeats the test on a 4-layer TinyStories model, with a mass-matched strongest removal and a count-matched random one. Evidence Press audits Goodfire's published neuron gallery and summarizes three partial replications; it rates the weak-activation question unresolved. Goodfire's technical companion, Neurons as an Interpretable Basis (Tigges et al., 2026-11-10), reports the 92.2% removal on Qwen3.6-35B-A3B and leaves what weak activations do to future work. Gadeev describes a mass-matched random control on a sorting classifier (Habr, 2026-10-22).

Formation and two-hop. Guo et al. (2025) describe a 3-layer transformer on in-context two-hop that guesses among end tokens, then switches to a sequential query; its early phase is guessing, not a working route. Wang et al. (NeurIPS 2024) and Ye et al. (2025) study two-hop composition over parametric facts. Nanda et al. (2023) showed modular addition's circuit forming before test accuracy moves, which G7.4 repeats. Xu et al. (2024) track SAE features across checkpoints, Mechanistic Data Attribution (2026) ties induction-head formation to training samples, and MAGIC (Ilyas and Engstrom, 2025) gets near-exact attribution from deterministic training.

Benchmarks with ground truth. Tracr (Lindner et al., NeurIPS 2023) compiles human-readable programs into decoder-only transformers, so the structure is known by construction. InterpBench (Gupta et al., NeurIPS 2024) trains realistic models to known circuits. MIB scores localization methods on LLMs. SAEBench (Karvonen et al., ICML 2025) scores SAEs on eight evaluations. Causal scrubbing (Chan et al., Redwood Research, 2022) tests a stated hypothesis by resampling ablations. InterpBench knows the circuit. Nablaframe knows only the task's variables.

Limits #

• Toy scale. Three tasks, models of 1, 2 and 4 layers with residual widths 32, 64 and 128. Nothing here is a claim about large models.
• Feasibility, not judged. All three headlines were measured outside the gates, by scripts no gate has checked. Seed 0's carry is in G6.3's report as a recorded edge, not a judged one.
• Judged means reproduced, on one specimen. A judged bar came from an earlier measurement of the same specimen, so it checks reproducibility and the twin, not generalization.
• Seed 0 is the discovery seed. Each finding was first seen on seed 0 and the bars were written afterwards; Figure 1's caption counts new seeds separately.
• Seeds were selected on success. Four of nine induction seeds and one modular-addition seed are left out for missing their accuracy bar. The results describe runs that learned their task in the allotted steps.
• Modular addition's seeds share a train split. Only the initialization and batch order differ.
• One architecture, one initialization scheme. Dense GPT-2-style blocks; the twins are five or six draws from that one scheme.
• One dictionary seed per model seed. Dictionary variance isn't separated from model variance.
• Intervals are within a seed and treat 512 instances as independent, which they aren't fully. Across seeds every value is listed and nothing is pooled.
• The weak-activation result has 20 random draws per seed, but one calibration set and one evaluation set. The original bar, one sign on four seeds, would have passed with either sign; the 20-draw rule adds separation from every draw, not a direction.
• Rules for naming and steering differ by basis. Neurons get a fitted threshold each, latents count above zero; units are steered to a mean.
• Planted variables, not planted circuits. The task fixes what must be computed, not where. Where a run holds it varies by seed.
• Nothing is judged on real text. The TinyStories reference run has no planted structure, so every result on it is recorded only (D44).
• Sources. Every source once marked for checking was opened on 2026-11-14. Gadeev's post and Adebayo et al. were not re-read, and the remaining sources are as the related-work search left them.

What resources would buy #

In the order the results suggest:

1. Compute for the architecture dial and wider models. The twin baseline and the weak-activation control are cheap tests whose answers may change with depth, normalization and initialization. The charter's dial (LayerNorm-free or element-wise norm, weight-sparse, bilinear MLP) is the first place to look.
2. More seeds, with dictionaries per seed and per checkpoint. Two-hop's seeds disagree about where b lives. Ten or more seeds with a probe at every position would turn that into a distribution, and test the formation hypothesis against the bar fixed above.
3. API credits for the weights-only reader. The harness is built and self-tested with its scoring bars fixed and hashed (SHA-256 e8906dc2…); it hasn't run.
4. Phase 8's ledger. Leave-a-batch-out replay over the 1,000 steps where seed 0's second route forms. Attribution by deterministic replay is known (MAGIC; Mechanistic Data Attribution); here it would be applied to a mechanism with a judged envelope.

To take up any of these, or to report a result that doesn't reproduce, open an issue on the repository.

References #

1. Adebayo et al. Sanity Checks for Saliency Maps. NeurIPS 2018.
2. AMX07. Are Neural Networks Already Clean? A small replication. README at commit 715c62b.
3. Arad, Mueller, Belinkov. SAEs Are Good for Steering, If You Select the Right Features. EMNLP 2025.
4. Arora, Wu, Steinhardt, Schwettmann. Language Model Circuits Are Sparse in the Neuron Basis. ICML 2026.
5. Cammarata. Neural Networks Are Already Clean. Goodfire, 2026-10-15.
6. Chan et al. Causal Scrubbing. Redwood Research, 2022.
7. Che, Li. Does the Model Use the Feature? Separating Steering from Mechanism in LLMs. 2026.
8. Comparing Sparsified MLPs and Transcoders: Many Transcoder Features Are Neurons in Disguise. NeurIPS 2026.
9. Evidence Press. Are Neural Networks Already Clean?
10. Gadeev. Post on Habr, 2026-10-22 (a mass-matched random control on a sorting classifier).
11. Guo et al. How Do LLMs Perform Two-Hop Reasoning in Context?
12. Gupta, Arcuschin, Kwa, Garriga-Alonso. InterpBench. NeurIPS 2024.
13. Heap, Lawson, Farnik, Aitchison. Automated Interpretability Metrics Do Not Distinguish Trained and Random Transformers (v2).
14. Ilyas, Engstrom. MAGIC: Near-Optimal Data Attribution for Deep Learning.
15. Karvonen et al. SAEBench. ICML 2025.
16. Lindner et al. Tracr: Compiled Transformers as a Laboratory for Interpretability. NeurIPS 2023.
17. Makelov, Lange, Geiger, Nanda. Is This the Subspace You Are Looking For? ICLR 2024.
18. Mechanistic Data Attribution.
19. Mueller et al. MIB: A Mechanistic Interpretability Benchmark. ICML 2025.
20. Nanda et al. Progress Measures for Grokking via Mechanistic Interpretability. 2023.
21. Automatically Interpreting Attribution Graphs via Probe Prompting.
22. Tigges, Kowal, McGrath, Lewis, Cammarata. Neurons as an Interpretable Basis. Goodfire, 2026-11-10.
23. Wang, Yue, Su, Sun. Grokked Transformers Are Implicit Reasoners. NeurIPS 2024.
24. Wu et al. AxBench. ICML 2025.
25. Xu, Wang, Wang. Tracking the Feature Dynamics in LLM Training.
26. Yavuz. Are Causal Neuron Circuits Unique and Transferable? 2026.
27. Ye et al. How Do Transformers Learn Implicit Reasoning?