The starting idea: what if a translator rewrote the model in a new language, for a new harness, keeping only what is needed? It mixes three things of very different difficulty: translating without losing anything, cutting what is not used, and rewriting the network as real code. Here is what each can deliver, why the second runs into a phenomenon called superposition, how much a hybrid translator would save, and what the experiment run right here showed.
Promisingupdated
I
The idea, simply
Think of a thousand-page cookbook written in Latin. You only want to make desserts. A good translator could do three increasingly daring things. One: translate the whole book into English, cutting nothing. You read faster, nothing is lost. Two: translate only the dessert chapter. The book gets thin, but if a dessert says "shortcrust pastry, see page 212", you are stuck. Three: replace the recipes that follow a rule with a formula ("1 egg per 100 g of flour") and keep the text only for the exceptions.
It is the same for an AI model. The "language" is the form the model is written in; the harness is the program that runs it (like llama.cpp on an ordinary computer); the translator is the tool that goes from one to the other, optimising along the way.
II
Three levels, from the certain to the dream
The three levels are nothing alike in difficulty. The first already exists everywhere, the second half works, the third only exists on toys.
Established
Level 1 · Translate without losing anything: compilation
This is what a compiler does: the same calculation, run more efficiently. Fusing operations, rounding the weights to fewer bits, fitting the calculation to the chip. Gains of 1.5 to 3 times are common, without changing the answers (or barely, with low precision). Some chips even speed up zero weights if they follow a regular pattern: two zeros out of every four, for up to twice the math throughput.
SparseGPT sets half of the weights of the largest open models to zero in a single pass, with an almost invisible loss of quality. Wanda does as well with an even simpler rule: a weight matters if it is large and receives strong signals. These methods target a general model. For a narrow task, one hopes to cut much more. That is where the difficulty begins.
Level 3 · Rewrite the network as real code: decompilation
The reverse direction already works: Tracr, at DeepMind, compiles a small program into transformer weights. And we know how to train small transformers so that they can be converted back into readable Python programs. But reading an 8-billion-parameter model and getting code out of it — nobody can do that. The reason has a name.
We picture a network as a chest of drawers: one drawer for chemistry, one for grammar, one for addition. You would just keep the right drawers. That is not how it works.
A network has fewer neurons than concepts to represent. It superposes them: each neuron takes part in dozens of concepts at once, each concept is spread over dozens of neurons, like voices blended in a choir. Elhage and colleagues, at Anthropic, showed it on toy models: as soon as concepts are rare, the network stores far more of them than it has dimensions.
The direct consequence for our translator: a weight that is "useless" for the target task may carry part of another skill the task silently depends on. And you never know in advance what question the user will ask tomorrow. Cutting by task is therefore a bet: you win big if questions stay in the expected lane, you lose if they leave it.
Rather than choose between keeping everything and cutting everything, the translator would make three piles. What follows a rule (arithmetic, dates, formats, conversions) becomes code: exact, almost free, and valid even beyond the examples seen. What is fuzzy (language, nuance, common sense) stays as a network, in a specialist pruned for the task. What leaves the lane is sent back to the full model: the emergency exit.
This is where two of the notebook's ideas meet: H3, "rules + exceptions = compression" where things can be checked, and the cascade, where a small model hands over when it is unsure. The translator is not a new lead: it is the tool that assembles them.
Take three numbers. p, the share of questions we know how to send to code: say 20%. s, the cost of the pruned specialist, as a fraction of the full model. f, the call-back rate: the share of questions where the specialist is unsure and calls back the full model. When it calls back, you pay for both.
The average cost of a question is then (1 − p) × (s + f). With a specialist at 50% and 10% call-backs: 0.8 × (0.5 + 0.1) = 0.48, the cost is divided by 2.1. Add level 1, a compilation that gains 1.5 times: 0.48 ÷ 1.5 = 0.32, that is a third of the original cost.
Where is the limit? The gain disappears when (1 − p) × (s + f) reaches 1. With s = 0.5, it takes f = 1 ÷ 0.8 − 0.5 = 75% call-backs to lose everything: the system is robust. But to divide by 2, you need f ≤ 0.5 ÷ 0.8 − 0.5 = 12.5%. With a specialist at 30%, the margin rises to 32.5%; with a specialist at 70%, dividing by 2 becomes impossible. Remember the lesson of the figure: what matters most is not the size of the specialist, it is that it can recognise what it does not know.
Fig. P — What a hybrid translator costs
Specialist at 30% of the cost: ÷ 2 as long as call-backs stay under 32.5%Specialist at 50% of the cost: ÷ 2 as long as call-backs stay under 12.5%Specialist at 70% of the cost: never divided by 2
Illustrative diagram: it shows the idea, not measurements.
A calculation, not a measurement. 20% of questions go to code, which costs almost nothing. The other 80% go to a pruned specialist; when it is unsure, it calls back the full model and you pay for both. Cost = 0.8 × (specialist cost + call-back rate). The specialist does not need to be perfect: above all, it must know when it does not know.VI
H10, born already broken into links
As promised on the page "Putting the hypotheses to the test", every new hypothesis arrives broken into links. Here is H10: a translator can rewrite a model for a new harness and tailor it to a task, dividing the cost by at least 2 with no visible loss on that task.
Link a, 95%: lossless translation and compilation work; that is the case today. Link b, 65%: given that, cutting by task, with an emergency exit, divides the cost by at least 2 with no visible loss. This link depends on superposition and on the call-back rate. Calculation: 0.95 × 0.65 = 62%.
The ambitious form adds a link c, 20%: a large share of a big model can be rewritten as real code. It would only be worth 0.95 × 0.65 × 0.2 ≈ 12%. We keep it as a question, not a hypothesis.
We have no graphics card, only two processor cores. So we ran the experiment in miniature, but for real: a small 0.8-million-parameter transformer, trained here on two mixed tasks. A, a rule: two-digit additions, 20% of them held out for testing. B, a lexicon with exceptions: the plurals of 600 invented words, 80% regular (+ s), 20% irregular and to be learned by heart, plus 150 new words never seen.
Then the three levels. Level 1: a separately written harness, in NumPy, reloads the weights, in 32 then 8 bits. Level 2: Wanda pruning calibrated on task A only, from 25 to 90% of the weights, then removal of whole neurons. Level 3: we search, from the model's behaviour, for the simple program that reproduces each task; what the program does not explain becomes a table of exceptions.
Our predictions were committed to the repository before the first run, so that we can be proved wrong. The main ones: the NumPy harness gives at least 99.9% identical answers; at 50% pruning, addition keeps at least 95% of its accuracy; beyond 70%, the lexicon suffers more than addition; the program "a + b" reproduces at least 99% of the model's answers; the rule "+ s" reproduces 80% ± 5; and on three-digit additions, never seen, the model almost always fails where the code gets everything right.
The model (805,929 parameters) learned in 20 minutes on two cores. At the start: 100% on additions never seen, 100% on the 600 learned words, but only 83.3% on the 150 new words, when adding an "s" was all it took: the network picked up the rule, but not perfectly. And 0% on three-digit additions: it did not understand addition in general, only the addition it had seen.
Established
Level 1 · Translate: holds, but not faster
The NumPy harness, written separately, gives 100% identical answers. In 8 bits, 99.96% of answers stay identical and accuracy does not move by a tenth of a point, for a size divided by 3.7 (3.2 MB → 0.87 MB). But our harness, simple and unoptimised, is 7.7 times slower than PyTorch: translating does not make things fast by magic; the gain comes from the compiler's work.
Promising
Level 2 · Prune: one failed prediction, one clear lesson
With 50% of the weights removed based on addition, addition keeps 93.5%: we had predicted at least 95%, so this one failed, narrowly. At 60%, addition drops to 71.9% and the lexicon to 45.2%; at 70%, 18.2% against 3.0%: what looked useless to addition was carrying the lexicon, as predicted. Calibrating on the lexicon gives the reverse: at 70%, it keeps 39.5% while addition falls to 5.5%. Removing half of the MLP neurons chosen from addition leaves it 95.9% and costs the lexicon 22 points.
Promising
Level 3 · Decompile: the surprise
The program "a + b" reproduces 99.99% of the model's 10,000 answers (a single difference). The rule "+ s" reproduces 80.17%, and the exception table holds 119 words: 119 of the 120 irregulars, the last one landing on an "s" by chance. The hybrid model scores 100% everywhere, including where the network failed: 100% on new words (against 83.3%) and 100% on three-digit additions (against 0%). For these two tasks, two lines of code and a 1,071-byte table replace a 3.2 MB model.
IX
What the experiment changes
Seven predictions out of eight hold; the one that fails was about pruning, by 1.5 points. The doubt has moved. Pruning alone disappoints: even in a tiny model with well-separated tasks, removing half the weights already costs a few points, and the specialist's advantage only appears at rates where the target task collapses too. That is superposition, measured.
Conversely, rule extraction surprises: the code does not just imitate the network, it beats it where the network generalised poorly. A network learns a rule roughly; once written down, the rule holds everywhere. That is the strongest argument for the hybrid translator.
Should H10 move? By our own rules, a number only changes if a link changes. And a 0.8-million-parameter toy model says almost nothing about an 8-billion-parameter one: we leave H10 at 62%, noting which way the clues point. Within link b, the "cut" route weighs less and the "turn into code" route weighs more. The next step would be the same experiment on a real small language model, with mixed questions.
The toy was not enough. We redid the experiment on SmolLM2-135M, a real 135-million-parameter language model, on a personal computer that could download it. Three domains: Python code, prose (WikiText-2) and 200 two-digit additions asked after three examples. Seven predictions, committed to the repository before the run. The calculation took six minutes.
At the start, the model predicts code well (perplexity 5.1: on average it hesitates between five words), prose less well (20.5), and solves only 14% of the additions. Code, for its part, solves 100%.
Translate to 8 bits: almost lossless, yet one prediction failed
Perplexity moves by 0.4% on code and 0.5% on prose; the size goes from 538 to 220 MB (÷ 2.45, the vocabulary staying in 32 bits). But only 85.5% of the addition answers stay identical: we had predicted at least 95%. On questions where the model hesitates, a tiny rounding is enough to tip the answer.
Promising
Prune: the specialist exists, but it is costly
At 50% based on code, code perplexity rises by 56% (predicted: at least 20%) and prose perplexity is multiplied by 2.6; addition falls to 0%. Calibrating on a domain protects that domain well: code fares better pruned based on code (7.9) than on prose (10.3), and prose better based on prose (44) than on code (52). But our prediction "calibrated on prose, code suffers most" fails at 50 and 60%: prose is such a broad domain that a prose specialist remains a generalist.
Promising
Knowing when you do not know: the signal exists, the margin does not
To bring the specialist pruned at 50% back within 5% of the full model's loss on code, the full model must be called back for 76.6% of the passages when relying on the specialist's doubt, against 83.3% at random and 73.4% with a perfect compass. Doubt is therefore a good guide, almost as good as the perfect compass. But you still have to call back three times out of four: we had predicted 60% at most.
XI
What round two changes
Four predictions out of seven hold. The three that fail tell the same story: a small model has no slack. Rounding makes it hesitate, pruning damages it everywhere, and even a specialist that knows when it doubts has to hand over almost all the time.
Let us redo Fig. P with the real numbers. A specialist at 50% of the cost that calls back the full model 77% of the time costs 0.8 × (0.5 + 0.77) ≈ 1.01: more expensive than doing nothing. At this size, the "cut" route pays nothing. The "turn into code" route holds: 14% → 100% on addition.
This time a link moves, as our rules require: link b goes from 65% to 55%, and H10 from 62% to 0.95 × 0.55 ≈ 52%. We do not lower it further, because large models prune much better than small ones: that is what SparseGPT and Wanda show. The decisive question therefore becomes: from what size does a pruned specialist pay back more than it costs? That will be round three.
Round two left a question open: is the pruned specialist doomed, or just too small? We took three models from the same family, SmolLM2 with 135 million, 360 million and 1.7 billion parameters, and the same protocol for all three: code, prose, additions, pruning at 25 and 50%, measuring the doubt. Seven predictions, committed before the run. The whole thing took a quarter of an hour on a Mac's graphics chip. The 135-million model, measured again, gives almost the same figure as last round (77.1% callbacks against 76.6%): the measurement is reproducible.
At the start, size changes everything for addition: 14%, then 87.5%, then 95% success. Code still gets 100%. In 8 bits, answers stay identical 85.5%, 98% and 99% of the time: the bigger the model, the less rounding makes it hesitate.
Calibrated on code, pruning multiplies code perplexity by 1.53, then 1.29, then 1.22: the drop with size is clear, as predicted. The 1.7-billion model even keeps 85.5% success on additions with half its weights, where the small one fell to 0%.
Promising
Doubt: the callback rate falls, but not fast enough
To stay within 5% of the full model on code, the specialist pruned at 50% must call back the full model for 77.1%, then 68.8%, then 62.5% of the excerpts. At random it would take 82.6, 75.1 and 72.5%: doubt remains a good guide at every size. We had predicted at most 60% for 1.7 billion: missed by 2.5 points. The ideal cost goes from 1.02 to 0.95 then 0.90: below 1, so worth it, but barely.
Promising
Pruning at 25%: nearly free, at every size
The real surprise comes from the gentlest rate. At 25%, the specialist stays within 5% of the full model on code with no callbacks at all, at all three sizes. Ideal cost: 0.8 × (0.75 + 0) = 0.6, a cost divided by 1.67. Code loses only 2 to 3% in perplexity, prose 6 to 8%: the task's lane is protected, the rest a little less.
XIII
What round three changes
Six predictions out of seven hold; the one that falls concerned the callback rate of the largest model, by 2.5 points. One nuance: our prediction "doubt beats chance at every rate" does not arise at 25%, since no callback is needed there.
Let's extend the trend, carefully: there are only three points. The callback rate at 50% falls by about 13 points each time the model grows tenfold. For an 8-billion model, that would give about 53% callbacks and an ideal cost of 0.83. To get down to 12.5%, the threshold that would halve the cost, it would take a model on the order of 10 trillion parameters. Conclusion: at 50%, pruning alone will never halve the cost within our 5% tolerance.
The good road takes another shape: prune gently (25%, no callbacks, cost 0.6), compile (× 1.5: 0.6 ÷ 1.5 = 0.4) and turn into code whatever follows a rule. On paper, that divides the cost by 2.5. One big caveat: the only pruning pattern today's chips really accelerate, two zeros out of four, corresponds precisely to 50%, the expensive rate. Pruning at 25% only pays off if the chip knows how to use it.
Link b therefore rises from 55% to 60%, and H10 from 52% to 0.95 × 0.60 = 57%. The next question is no longer "from what size?" but "on which chip does 25% pruning actually become faster?".
Round four: cutting whole neurons, stopwatch in hand
Round three's cost of 0.6 was "ideal": it assumed the chip takes advantage of weights set to zero. But zeros scattered in a matrix of the same size are computed like any other number. This time we removed whole neurons from each model's MLP blocks, 12.5%, 25% or 50% of them: the matrices really shrink, on any chip. And we timed the result on a Mac's graphics chip instead of calculating it. Nine predictions, committed before the run.
The control confirms the starting point: with 25% scattered zeros, the model takes exactly the same time as the full model, within 0.4%, at all three sizes. Zeros gain nothing on this chip.
The time saved tracks the weights removed almost exactly
Removing whole neurons really does speed things up. For the 1.7-billion model, reading time falls to 0.91, 0.84 then 0.69 times that of the full model when 12.5, 25 then 50% of the MLP neurons are removed. The model then keeps 91, 82 and 65% of its weights: time tracks size almost exactly. The small models gain a little less, because their MLP blocks weigh less in the total. Word-by-word generation gains as much as reading.
Established
But a clean cut does more damage
At 25%, code perplexity is multiplied by 1.16, 1.12 then 1.10 (against 1.02 to 1.03 with scattered zeros), and prose by more than 2. The full model must then be called back for 35, 33 then 29% of the code excerpts (at random: 46, 44 and 42%). Once those callbacks are counted, the measured cost is 1.25, 1.20 then 1.13: dearer than cutting nothing. At 50% it is worse still: about 80% callbacks and a cost near 1.5.
Promising
At 12.5%: the first real time saving, with no callbacks
Cutting gently, the specialist stays within 5% of the full model on code with no callbacks at all, at all three sizes. The gain is modest but measured: 5%, 6% then 9% less time (cost 0.95, 0.94 then 0.91). The 1.7-billion model even gets 98% of the additions right, against 95% for the full model. Prose, for its part, already loses a lot (perplexity × 1.4 to 1.5): the cut did follow the code's lane. Calibrating on prose would have been a mistake: at 25%, code perplexity would be multiplied by more than 2 instead of 1.1.
XV
What round four changes
Seven predictions out of nine hold. The first to fall is the most interesting: we had bet that no cut would save real time once callbacks were counted. Wrong: at 12.5%, the specialist is faster and needs no callbacks. The other is decided by a thousandth: at 25%, the largest model's code perplexity is multiplied by 1.099, and we had predicted more than 1.10. A third holds narrowly and not for the reason we gave: we thought generation would gain less than reading for the small model; both gain the same.
The lesson fits in one simple rule: on this chip, the time saved roughly equals the share of weights removed. Halving the cost would therefore require removing half the model while almost never calling back the full one. Without retraining, we are far from that: at 50%, eight excerpts out of ten must be called back.
The next lead is well known: after a cut, a short retraining repairs a good part of the damage. LLM-Pruner does it in a few hours with a light method; Sheared-LLaMA builds good small models this way for a fraction of the cost of full training. If such retraining brought callbacks at a 25% cut down towards zero, the measured gain would go from 9% to about 16%.
Our rule, written in advance, said: a measured cost below 0.9 would raise link b; S9 holding would lower it. The best cost is 0.91 and S9 falls: link b stays at 60%, H10 at 57%. But the cost is no longer only calculated: for the first time, it is timed.
In round four, removing 25% of the MLP neurons saved 10 to 16% of real time, but the specialist had to call back the full model for a third of the excerpts: the gain vanished. This time, after the cut, the specialist takes a short catch-up session. The full model plays the teacher; the cut model, the student, learns to give the same probabilities as the teacher on code. This is distillation. The training code comes from different files than the evaluation, so the student cannot learn the answers by heart.
Two ways of repairing were tried. The first retrains the weights of the cut MLP blocks directly. The second, LoRA, freezes those weights and trains only small matrices added alongside, like sticky notes in a textbook's margins, then merges them in: that is what lets the 1.7-billion model and its teacher fit in a Mac's 24 GB. Eleven predictions, committed before the run. The whole thing took about an hour and a half.
At a 25% cut, before repair, the full model must be called back for 35%, 33% and 29% of the code excerpts (135M, 360M, 1.7B): round four is reproduced to the figure. After repair, only 6% are needed for 135M and 15% for 360M, repaired directly, and 23% for 1.7B, repaired with LoRA. The measured cost, callbacks included, falls to 0.93, 1.03 and 1.06. The small model becomes worth it: 7% of time saved. The other two come close to 1 without going under. At a 50% cut, repair is not enough: two thirds of callbacks remain.
Promising
Retraining the weights repairs better than LoRA
On the same 135-million model, with the same number of steps, direct repair leaves 6% callbacks and LoRA 15%. LoRA's "sticky notes" amount to only 2.5 million adjustable numbers, against tens of millions for direct repair: they correct less. The 1.7-billion model only got LoRA, for half as many steps, for lack of memory and time: its result (23%) is therefore a floor, not a ceiling. Its repair took 19 minutes.
Promising
Surprise: prose improves too
We had predicted that retraining on code would damage prose further. It is the opposite, at all three sizes: prose perplexity drops a little after repair (for example × 2.59 → × 2.38 for 135M). The explanation lies in distillation: the student does not learn only the right answer on code, it learns the teacher's whole probability distribution, that is, part of its general way of predicting what comes next in a text.
XVII
What round five changes
Six predictions out of eleven hold: our least successful round. We had bet that repair would bring callbacks below 10% and save at least 10% of real time; it stops halfway. The direction is right, but the effect is weaker than we imagined.
The best result comes from the smallest model: cut by 25% then repaired, it costs 0.93, better than the gentle 12.5% cut without repair (0.95). For the first time, a clean cut followed by repair saves real time, callbacks included. The gain is modest, and the catch-up session has to be paid for once: a few minutes for a small model, about twenty for the 1.7-billion one.
The rule written in advance said: a cost below 0.9 would raise link b; more than 20% callbacks after repair at every size would lower it. Neither happens: link b stays at 60%, H10 at 57%. The next question is clear: with longer repair, or direct retraining of the 1.7-billion model, do callbacks fall below 10%? It will take a bigger machine than a Mac.
When round five's repair stopped, callbacks were still falling. Was time all that was missing? We kept the same 25% cut and repaired for longer, with checks along the way: after 300, 600 and 1,200 steps for the 360-million model (direct retraining), after 150, 300 and 600 steps for the 1.7-billion one (LoRA). The learning rate stays constant after a short warm-up, so that the checks of one run can be compared. Seven predictions, committed before the run. The whole thing took two and a half hours on a Mac.
360 million: no callbacks at all after 1,200 steps
Callbacks go from 33% before repair to 10% after 300 steps, 4% after 600 steps, then 0% after 1,200 steps: on every one of the 48 code excerpts, the cut specialist stays within 5% of the full model. Its code perplexity is now multiplied by only 1.07 (against 1.12 before repair), prose keeps improving (× 2.44 → × 2.14) and additions rise a little (55 → 60%). Meanwhile, the student saw the training code barely more than once: it had no time to learn it by heart.
Promising
1.7 billion with LoRA: the drop slows down
Callbacks go from 29% to 23%, 21% then 19% after 150, 300 and 600 steps: each doubling of the time now gains only about 2 points. LoRA corrects less than direct retraining, as in round five; its small matrices seem to level off. The measured cost falls to 1.05 without going below 1. The 1.7-billion model would need direct retraining, too heavy for a Mac's memory.
Speculative
A less reliable stopwatch this time
Without callbacks, the specialist's cost comes down to its speed. This time we measure 0.97 for the 360-million model. But during this long run, the full model itself sped up by 10% between start and end (0.68 → 0.61 seconds per batch), whereas that gap stayed under 1% in rounds 4 and 5. Those two rounds gave the same cut model a speed of 0.87 to 0.88. Our rule, written in advance, takes the value measured in the run: we keep it. The true cost is probably closer to 0.88: the next round times the same model again.
XIX
What round six changes
Five predictions out of seven hold. The two that fall concern the 360-million model's cost (0.97 measured, against below 0.95 predicted) and the 1.7-billion model (19% callbacks, against at most 15%).
The main lesson: for the 360-million model, only time was missing. A 25% cut followed by an hour of repair gives a specialist that never again needs to call the full model on code. The callback question is settled for this model; what remains is the speed of the cut, which rounds 4 and 5 measured at 12 to 13% of time saved.
The rule written in advance: a measured cost below 0.9 would raise link b; more than 20% callbacks at both sizes would lower it. Neither: link b stays at 60%, H10 at 57%. The remaining doubt is no longer about callbacks but about the stopwatch.
In round five, repair through distillation also improved prose, even though it dealt only with code. Our explanation: the student learns not only the right answer, it learns all of the teacher's hesitations. To put it to the test, an idea that came out of a discussion: take two identical students, the same model cut by 25% twice, and give them to the same teacher, on the same texts, in the same order, for the same time.
Only one thing changes: the way of teaching. The "nuances" student learns the teacher's whole probability distribution: "return at 70%, yield at 20%…". The "answers" student learns only the teacher's favourite word, as if it were given only the answer key. Two sizes (135 and 360 million), six predictions committed before the run, a little under an hour on a Mac.
The "nuances" student redoes round five to the figure
It takes exactly round five's setting, and it gives exactly the same callbacks: 6.2% for 135M and 14.6% for 360M. The measurement reproduces from one day to the next. Its prose improves as it did then (× 2.59 → × 2.38 for 135M; × 2.44 → × 2.29 for 360M).
Established
The "answers" student ends up worse than before the lesson
It is not just that it progresses less: it regresses. Its callbacks go from 35% to 73% (135M) and from 33% to 77% (360M), its code perplexity is multiplied by 1.5 to 1.6, against 1.12 to 1.16 before the lesson, and its prose gets worse too. Yet it did learn what it was asked: it finds the teacher's favourite word more and more often. But the gap with the teacher's probabilities widened during the lesson instead of shrinking.
Promising
Why: it learns to be sure of itself
When the teacher hesitates between two words, the "answers" student learns to put all its weight on the first. It becomes overconfident. On real text, every time the second word was the right one, it is heavily penalised. The "nuances" student, on the contrary, learns to hesitate where the teacher hesitates. This is the "dark knowledge" described by Hinton and his colleagues in 2015: the wrong answers the teacher finds plausible teach almost as much as the right one.
XXI
What round seven changes
Six predictions out of six hold: a first in this notebook. Our explanation from round five passes the test, and more clearly than expected. Copying the teacher's nuances is what repairs; copying only its answers does damage.
Good news for the stopwatch: this time it was stable (the full model takes the same time at the start and the end, within 1%). The 360-million model cut by 25% runs at 0.87 of the full model's time, as in rounds 4 and 5. The specialist repaired for 1,200 steps in round six, which no longer needed any callback, would therefore cost about 0.87.
That figure would fall below the 0.9 threshold that would raise link b. But it puts two rounds together, and our rule is about a measurement made within a single run. Link b therefore stays at 60%, H10 at 57%, pending a confirmation run: the 1,200-step repair and the stable stopwatch together, with the prediction written in advance.
Taken together, rounds 6 and 7 suggested a specialist that never calls back the full model and runs at 0.87 of its time. But our rule requires everything to be measured within a single run. So we redid the 1,200-step repair of the 360-million model cut by 25%, then timed it alternating: full model, specialist, full, specialist… seven measurements in all. If the machine speeds up or slows down along the way, both models are affected alike. Five predictions, committed before the run.
Established
Zero callbacks, again
After 600 steps, the full model must be called back for 4.2% of the excerpts; after 1,200 steps, for none. Code perplexity is now multiplied by only 1.065, prose keeps improving and additions climb back to 60%. Round six's figures reproduce almost exactly.
Established
A stopwatch finally under control
The four measurements of the full model differ by at most 1.7%, against 10% in round six. The specialist takes 0.88 of the full model's time, and that ratio barely moves from one round to the next (0.882, 0.884, 0.882). Removing a quarter of the MLP neurons saves 12% of real time.
Established
A measured cost below 0.9
With no callbacks at all, the measured cost comes down to that ratio: 0.88. For the first time, within a single run and with a stable stopwatch, a specialist that was cut and then repaired is faster than the full model, without straying more than 5% from it on any code excerpt.
XXIII
What round eight changes
Five predictions out of five hold. The rule written in advance said: if the stopwatch is stable and the cost below 0.9, link b rises from 60 to 65%. That is the case: H10 goes from 57 to 0.95 × 0.65 ≈ 62%.
What this means, without overstating it: on an ordinary computer's chip, one hour of repair is enough to turn a model cut by 25% into a code specialist 12% faster, with no notable loss. That is far from the halving of cost that H10 announces; the rest of the way will have to come from elsewhere (a deeper cut better repaired, compilation, turning into code whatever follows a rule). But the "cut, then repair" road works, and it is now measured.
Round nine: a teacher who exaggerates its hesitations
Round seven showed that learning the teacher's hesitations repairs, and that learning only its answers does damage. So does a teacher who exaggerates its hesitations teach even better? That is the idea of "temperature", proposed by Hinton and his colleagues in 2015: teacher and student flatten their probabilities before the lesson. At T = 2, "return 90%, yield 8%" looks more like "return 70%, yield 20%"; at T = 4, even more so. Three identical students per size (T = 1, 2 and 4), same teacher, same texts, same duration. Five predictions, committed before the run.
The T = 1 student takes exactly the lesson of round seven's "nuances" student. It gives exactly the same callbacks: 6.2% for 135M, 14.6% for 360M. This is the third time these figures come out identical, in rounds 5, 7 and 9.
Established
The more the teacher exaggerates, the less the student learns
Callbacks rise with temperature: 6 → 10 → 15% for 135M, 15 → 17 → 21% for 360M, at T = 1, 2 and 4. Code and prose perplexity worsen slightly in the same order. We had bet that T = 2 would do at least as well as T = 1: wrong, at both sizes.
Promising
Why: neither too sure nor too hesitant
The student that learns an exaggerated version of the hesitations ends up hesitating too much, just as round seven's "answers" student ended up too sure of itself. Both drift away from the teacher, in opposite directions. Temperature helps when a small student learns from a large, very confident teacher whose nuances are almost invisible. Here, the student is a damaged copy of the teacher, and its goal is to resemble it exactly: the best thing is to copy its probabilities as they are.
XXV
What round nine changes
Three predictions out of five hold: the reproduction at T = 1, the decline at T = 4 and doubt as a better guide than chance. The two that fall are the ones that bet on a gain at T = 2.
Rounds 7 and 9 together sketch a simple rule for repairing a specialist: it should learn the teacher's probabilities as they are. Hardened too much (answers only) or softened too much (a high temperature), they teach less well. This round was about how to repair, not about cost: H10 stays at 62%.
Round eight's specialist had been repaired and evaluated on code from Python's standard library: different files, but the same style and the same authors. Is it good at Python code, or only at standard-library code? We redid round eight exactly, adding a second evaluation on foreign code: 48 excerpts taken from the sources of PyTorch and Transformers, never seen during the cut or the repair. Five predictions, committed before the run.
Established
On home ground, round eight reproduces
On the standard library, round eight's figures come back identical: 0% callbacks after 1,200 steps, a speed ratio of 0.88, a stable stopwatch (1% drift). Measured cost: 0.88.
Established
On foreign code, it calls back nearly one excerpt in two
On PyTorch and Transformers code, the cut specialist calls back the full model for 71% of the excerpts before repair, and still 48% after 1,200 steps. Repair helps a little (perplexity × 1.29 → × 1.16), but far less than at home. Measured cost on this code: 1.36, dearer than cutting nothing. What we called a "Python code specialist" is really a standard-library specialist: its domain is that of its repair.
Established
And its doubt no longer guides it
At home, following the specialist's doubt always needed fewer callbacks than chance. On foreign code, that is no longer true: 48% callbacks following the doubt, 48% at random. The specialist cannot tell which excerpts it handles badly once they leave its domain. This matters for the translator: its alarm signal is only worth something where it was tuned.
XXVII
What round ten changes
One prediction out of five holds: the reproduction on the standard library. The other four fall, and by a wide margin. Our least successful round, and probably the most useful.
The rule written in advance said: more than 20% callbacks on foreign code bring link b back from 65 to 60%. It takes 48%: H10 falls back from 62 to 57%. The gain measured in round eight still holds, but only on the exact ground of the repair, and a real translator would need to know when it leaves that ground: its doubt does not.
To get closer to halving the cost, we have to cut more. We removed 50% of the 360-million model's MLP neurons, then repaired eight times longer than in round five, with checks after 600, 1,200 and 2,400 steps. Same interleaved timing, same double evaluation (standard library and foreign code). Five predictions, committed before the run.
Cut in half, it takes 0.75 of the full model's time, with a stable stopwatch (0.7% drift): a quarter of the time saved, as in round four.
Established
But repair levels off
On the standard library, callbacks go from 81% to 54%, 50% then 46% after 600, 1,200 and 2,400 steps. Each doubling of the time now gains only 4 points. Code perplexity stays multiplied by 1.17, prose by nearly 5. Measured cost: 1.21. On foreign code it is worse: 73% callbacks, cost 1.48.
Speculative
Why: too much knowledge was removed
At 25%, repair recovered almost everything the cut had lost. At 50%, it recovers only part of it, even with two and a half passes over the training code. One possible explanation: half the neurons are no longer enough to hold what the model knew, and distillation cannot create room. Work such as Sheared-LLaMA repairs with billions of tokens, not a few million.
XXIX
What round eleven changes
Two predictions out of five hold: callbacks fall at every check, and the specialist does run at 0.75. The other three fall: after 2,400 steps, 46% callbacks are still needed and the cost stays at 1.21.
The rule written in advance does not move H10: it stays at 57%. Rounds 10 and 11 say the same thing in two ways: on a machine like a Mac, cutting then repairing works for a moderate cut on a precise ground. Cutting harder, or aiming wider, calls for a repair of a quite different scale.
Round twelve: studying with two teachers, and a customs officer
Round ten had shown two weaknesses: the specialist repaired on the standard library held up poorly on foreign code, and its doubt did not spot the excerpts it handled badly. We tackled both at once. First, the cut and the repair used a mix: half standard library, half Transformers sources. Then we tested a detector that does not look at the specialist's hesitation but at "what the excerpt looks like": a fingerprint of its internal states on known code, and the distance of each new excerpt from that fingerprint. Six predictions, committed before the run.
Established
A varied repair holds on code never seen
After 1,200 steps, the specialist calls back the full model for 4% of the standard-library excerpts, 0% of the set-aside Transformers files, and 21% of the PyTorch excerpts, which no step had seen. In round ten, repaired on the standard library alone, it called back 48% on foreign code that mixed PyTorch and Transformers. On the set-aside Transformers files, it even does slightly better than the full model (perplexity × 0.997).
Established
The customs officer recognises foreign code
The detector tells PyTorch excerpts apart from known code in 93% of randomly drawn pairs (area under the ROC curve of 0.93). Doubt, for its part, fails again on PyTorch: following it needs more callbacks than chance (21% against 17%). Looking at what the specialist reads works better than asking it whether it hesitates.
Promising
On a mixed stream, the cost falls below 1
On a stream of 96 excerpts, half standard library, half PyTorch, as in real use, calling back first the excerpts the detector deems foreign needs 9.4% callbacks, against 11.5% following the doubt, 13.8% at random and 7.3% for an all-knowing oracle. With a speed ratio of 0.88 and a stable stopwatch (1.4% drift), the measured cost falls to 0.97.
XXXI
What round twelve changes
Six predictions out of six hold: the second time in this notebook. Round ten's two weaknesses each have their remedy: a more varied repair, and a detector that looks at the excerpt rather than the hesitation.
Our rule written in advance asked for more to raise H10: a cost below 0.95 on the mixed stream. At 0.97, a little is still missing: link b stays at 60%, H10 at 57%. Yet the real translator is taking shape: a specialist repaired on varied ground, a customs officer that sends the foreign to the full model, and a modest but measured gain, even when the stream mixes known and unknown.
Round thirteen: twelve teachers, and a hasty customs officer
In round twelve, two families of code had been enough to bring callbacks on PyTorch down from 48 to 21%. We pushed the idea further: repair on twelve families of Python code in equal shares (standard library, Transformers, NumPy, pandas, SymPy, NetworkX, pip, setuptools, Rich, Pydantic, Pygments, Jinja2), with PyTorch still the final exam. And we tried a hasty customs officer, who judges the excerpt after 8 layers out of 32 instead of waiting for the last: if it looks foreign, the specialist stops there and the full model takes over. Six predictions, committed before the run.
Established
More teachers do not help on PyTorch
After 1,200 steps, the specialist calls back the full model for 0% of the set-aside files of the eleven other families: it knows what it studied. But on PyTorch it calls back 23%, as many as with two families (21%), and on the standard library it rises to 8% (4% in round twelve). Part of the explanation lies in our setup: in equal shares, the smallest family limited all the others, and the student went over a corpus smaller than in round twelve three times.
Established
The final customs officer loses its nose
With twelve families in its fingerprint, known code becomes very varied, and PyTorch stands out from it less clearly: the area under the ROC curve falls from 0.93 to 0.73. On the mixed stream it still needs fewer callbacks than doubt (11.5% against 15.6%), but the cost stays at 0.99.
Promising
The hasty officer sees nothing, but saves time
After 8 layers, the detector no longer tells PyTorch from known code (area under the ROC curve of 0.45, like a coin toss): following its sorting needs 24% callbacks. Yet its cost is the best of the round, 0.95: reading 8 layers costs only 0.2 of the full model, and every excerpt sent back early spares the specialist three quarters of its reading. The gain comes from stopping early, not from a good nose.
XXXIII
What round thirteen changes
Two predictions out of six hold: the good mark on the families studied, and a hasty officer cheaper than the final one. The other four fall: PyTorch stays at 23%, the standard library rises again, the hasty officer recognises nothing, and the final officer's cost stays at 0.99.
The rule written in advance required callbacks on PyTorch to fall below 12%: H10 stays at 57%. The lesson is twofold. Adding families does not replace a longer study of each. And a customs officer needs a sharp enough fingerprint: the wider the known ground, the more the foreign blends into it. The best setup remains, for now, that of round twelve.
Round twelve's final customs officer had a good nose, but it judged only after the specialist had read everything. Round thirteen's, placed after 8 layers, saved time while seeing nothing. We took round twelve's setup exactly and placed six customs officers along the way: after 8, 12, 16, 20 and 24 layers out of 32, and at the end. To avoid picking the best one after the fact, the H10 rule applied in advance to a single one of them, the one at 16 layers. Five predictions, committed before the run.
Established
Round twelve reproduces to the figure
Same setup, same results: after 1,200 steps, 4.2% callbacks on the standard library, 0% on the set-aside Transformers files, 20.8% on PyTorch, and the final officer's area under the ROC curve comes back at 0.93. The stopwatch is stable (0.5% drift).
Promising
The nose only appears at the very end
The intermediate officers see almost nothing: area under the ROC curve of 0.63 at 8 layers, 0.62 at 12, 0.52 at 16, 0.49 at 20, 0.69 at 24, then 0.93 at the end. The nose does not grow gradually with depth: it appears suddenly at the last layer. One possible explanation, which this round does not test: the last layer serves to predict the next word, and that is where each library's own vocabulary leaves a clear trace; the middle layers describe the structure of code, common to all.
Established
Stopping early wins without seeing
The officer at 16 layers, the one in our rule, needs 21% callbacks and costs 0.98, more than the final officer (0.97). The cheapest is still the one at 8 layers, 0.94, although it sees almost nothing: as in round thirteen, it wins because it stops the specialist early on part of the excerpts, not because it sorts well. Picking that officer after the fact would be cheating; our rule does not allow it.
XXXV
What round fourteen changes
One prediction out of five holds: the reproduction of round twelve. The other four fall: at 16 layers the officer has no nose, no more than at 8, and its cost exceeds the final officer's.
The rule written in advance does not move H10: it stays at 57%. We now know that, in this model, recognising foreign code requires going all the way to the last layer. To win on both counts, a subtler officer than a plain distance on hidden states would be needed, for instance a small classifier trained to spot the foreign from the first layers.
Scale. A 0.8-million-parameter model on two clean tasks is not an 8-billion-parameter model facing real questions. Superposition is much denser there, and tasks blend far more.
Decompiling from behaviour is not reading the network. Our level 3 guesses a program from the answers; it does not read it in the weights. That works for a rule we can already name; finding an unknown rule is another problem, the one AlphaEvolve tackles.
The specialist's doubt. The whole calculation rests on a specialist that knows when it does not know. Models are often overconfident; a router that is silently wrong costs more than it saves.
The real gains of pruning. Setting weights to zero speeds nothing up if the chip still computes the zeros. You have to cut whole blocks, or follow the patterns the chip can exploit, and those cuts do more damage.