Deep dive · Hypotheses
H2 under test: our toy pilots
Ten toy pilots on H2, learn the top and look up the bottom: 34 of 51 pre-registered predictions held, and in pilots 4 to 7 most of the false ones are failures to learn, not refutations. A tiny model, an invented rule, an ideal memory in most cases: not evidence about real language models.
The question
H2 says that parameters are scarce and should be kept for reasoning, while facts sit in an external memory. Before touching a real model we built a toy world: 1024 entities, each with a random digit (its fact), and a rule that adds the digits of two entities modulo 10 (three entities in the fifth pilot). Pairs used for testing never appear in training, so the model must apply the rule, not recite it.
The model is a transformer with 2 layers and width 64, trained for 2000 updates. Every figure is a mean over 5 seeds, chance is 0.1, and the size judged is 1024 entities (256 is reported, not judged). Predictions were written before each run; from the second pilot on they were written after seeing the earlier ones, so they are not blind. The public data are also on [the lab page](https://huggingface.co/spaces/Kyyzen01/OrigoTech-AI-Research-Lab). Pilots 6 to 10 reuse the same model and world; the training length varies (2000 to 4000 updates, plus fine-tuning in pilot 10), as stated in each protocol.
Pilot 1: facts in the weights or in a memory
With 64 entities, all three conditions reach at least 0.9985 on held-out pairs. With 1024, a model that keeps the facts in its weights recalls them perfectly (fact accuracy 1.0000) yet scores 0.0973 on held-out pairs, and 0.1235 with about 2.6 times the parameters (width 128): close to chance. The same model scores 1.0000 when an ideal memory puts the two facts in its context. All 5 predictions held. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 1: facts in the weights or in a memory (results, JSON; protocol alongside)
Pilot 2: explicit steps or longer training
If the model first writes out the two facts and then adds them (explicit steps, supervised in training), it reaches 1.0000 at 1024 entities without any memory. Doubling the training to 4000 updates instead leaves the plain model at 0.1285 (0.1018 at 2000 updates). At 256 entities the longer run reaches 0.9393, from 0.4710 at 2000 updates, with seeds from 0.6965 to 1.0000. So in this toy a memory is not needed to compose, but the steps are supervised, which says nothing about real models. All 5 predictions held. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 2: explicit steps or longer training (results, JSON; protocol alongside)
Pilot 3: updating facts
With the facts in the context, edited facts score 1.0000 and entities never seen in training 1.0000, against 0.1066 and 0.1008 for the model without memory. Fine-tuning the weights on the edits only, for 200 updates, reaches 0.2376 on edited facts while the accuracy on untouched facts falls from 1.0000 to 0.1337. Adding replay of the old facts keeps untouched facts at 0.9975, but edited facts reach only 0.3399. 5 of 6 predictions held; the false one expected the edit to be absorbed within 200 updates. One fine-tuning recipe, no dedicated knowledge editor. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 3: updating facts (results, JSON; protocol alongside)
Pilot 4: imperfect retrieval
Each retrieved item is replaced, with probability 1 − r, by the item of another entity. At r = 0.9, 0.7 and 0.5 the accuracy is 0.8366, 0.5604 and 0.3306, close to r² + (1 − r²) × 0.1 (0.829, 0.541, 0.325): both items must be right, otherwise the answer is chance. The no-memory steps model scores 0.9997, so the memory model falls more than 0.10 below it already at r = 0.9. The distractor test is not concluded: a model trained with 4 random distractors did not learn the rule (0.1163), so what happens with 8 cannot be read. 3 of 5 predictions held; the two false ones are failures to learn. Random substitution, not a real search engine; search cost not measured. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 4: imperfect retrieval (results, JSON; protocol alongside)
Pilot 5: three facts
With three facts to add, one pass scores 0.0989 and plain explicit steps 0.1215, although the three facts are recalled at 0.9999. Adding one intermediate partial sum lifts the score to 0.9998. The external-memory model averages 0.2659 (seeds from 0.0985 to 0.9285): it learns late, and in this run only one seed in five learned the rule. 3 of 5 predictions held; the two false ones are failures to learn within 2000 updates, which is not evidence against composition. One toy rule, one model size. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 5: three-fact composition (results, JSON; protocol alongside)
Pilot 6: distractors and a curriculum
Here the memory also holds random distractor items. With 4 distractors, a model trained for 4000 updates reaches 0.2456 on held-out pairs (seeds from 0.0955 to 0.7690), and one trained with a curriculum (0, then 2, then 4 distractors) 0.3856 (0.1275 to 0.7440). With 8 distractors the curriculum model stays at 0.1052, chance level, against 0.9999 for the same format without distractors. The control on the retrieved facts fails in several cells (for example 0.1440 and 0.1666 at 8 distractors), so the other results are not readable: the question stays open. 1 of 5 predictions held; the four false ones are failures to learn. The clean reference is trained for 2000 updates against 4000 (not paired). Random distractors only. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 6: distractors and a curriculum (results, JSON; protocol alongside)
Pilot 7: three facts, longer training
With three facts and 4000 updates, the external-memory model averages 0.3286 (seeds from 0.0955 to 0.8415; none reaches 0.90), against 0.1046 at 2000 updates. Starting with two facts for a 1500-update warm-up lifts the mean to 0.6981 (0.1230 to 0.9995; +0.3695 over the memory alone), but the spread between seeds is wide. Plain explicit steps without a partial sum stay at 0.1645 (0.1030 to 0.2425). The three facts are recalled at 1.0000 in all three conditions. 2 of 5 predictions held (the curriculum is at least as good as the memory alone; the facts are recalled). The false ones are failures to learn, with one lucky-seed cell. One warm-up length, one toy rule. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 7: three facts, longer training (results, JSON; protocol alongside)
Pilot 8: frequent in the weights, rare in memory
With entities drawn from a Zipf law (a few frequent ones, many rare ones), the model without memory scores 0.9520 on pairs of frequent entities and 0.9260 on pairs of rare ones, so the rare facts are learned in the weights too. The hybrid (frequent facts in the weights, rare ones in memory) scores 0.9904 and 0.9926, and the full-memory model 1.0000 and 0.9999. Weighted by frequency, the hybrid is 0.0024 below full memory (0.9976 against 1.0000). 3 of 5 predictions held; the two false ones are that the rare facts would not be learned in the weights, and that the hybrid would be at least as good as full memory. One Zipf split; the frequent facts may have been memorised rather than composed. Ideal retrieval. Details: results, protocol.
Pilot 9: how many rule examples
Trained on only 2% or 5% of the pairs, the external-memory model reaches 1.0000 on held-out pairs (0.9977 at 20%, 1.0000 with all). Without memory (explicit steps) it reaches 0.9998 at 2% but 0.9132 at 5% (seeds from 0.5960 to 1.0000), against 0.9980 with all pairs. 4 of 5 predictions held; the false one (more pairs never hurt) fails because, without memory, 2% scores 0.0018 above 100%. The rule uses only 100 digit couples, so a few examples are enough here: this is specific to the toy and says nothing about real models. Ideal retrieval. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 9: how many rule examples (results, JSON; protocol alongside)
Pilot 10: longer or stronger updates
After 1000 updates, fine-tuning on the edited facts with replay of the old ones reaches 0.5310 on edited pairs, and 0.8212 with a learning rate three times larger (seeds from 0.6720 to 0.9665), while the untouched facts stay at 0.9876. Without replay and with a small learning rate, the untouched facts fall to 0.2092 and edited pairs reach only 0.2138. A heavier share of edited rows with replay reaches 0.4054, below the plain replay run (gap −0.1256). After fine-tuning, entities never seen in training stay at chance (0.099 to 0.103), whereas the external memory scores 0.9998 on edited pairs and 1.0000 on new entities with no update at all. 3 of 5 predictions held; the false ones are that forgetting would be limited without replay (0.2092, below the 0.50 expected) and that a heavier share of edits would help. Four fine-tuning recipes, one value each, no search; no dedicated knowledge editor. Details: results, protocol.
Source: OrigoTech, H2 toy pilot 10: longer or stronger updates (results, JSON; protocol alongside)
Reading: what the toy world suggests
In this toy world, composing many stored facts in one pass is the hard part, and two things solve it: an ideal external memory, or writing the facts out step by step. What a memory adds clearly is updating: immediate with a memory, slow in the weights and, without replay of old facts, very forgetful (with replay and a larger learning rate, edits reach 0.8212 after 1000 updates). Its advantage shrinks as retrieval gets worse. With three facts, writing out the facts is not enough and an intermediate sum is needed; a warm-up on two facts helps on average (0.6981 against 0.3286) but the seeds differ widely. A hybrid with frequent facts in the weights was not distinguishable from keeping everything in memory, and a few rule examples were enough in this toy. The distractor question stays open. 34 of the 51 pre-registered predictions held (5 of 5, 5 of 5, 5 of 6, 3 of 5, 3 of 5, 1 of 5, 2 of 5, 3 of 5, 4 of 5 and 3 of 5); in pilots 4 to 7 most of the false ones are failures to learn, not refutations, whereas those of pilots 8 to 10 are measured results.
What these numbers do not prove
A toy world: one rule (addition modulo 10), random independent facts, one small model, 5 seeds. Retrieval is ideal, except in pilots 4 and 6, where it is corrupted by random substitution or distractors, and the cost of searching is not measured. The explicit steps are supervised reasoning, not shown for real language models. The size 256 is reported but not judged; no significance test is run and no percentage of data saved is estimated. The predictions after the first pilot were written knowing the earlier results. Pilot 1 was run at 2000 updates after a first attempt at 3000 was stopped for time, and a duration check on a CPU, whose accuracies the authors saw, preceded the judgement. In pilot 6 the clean reference is trained for 2000 updates against 4000 for the others (not paired); pilots 7 and 8 use one duration and one head/tail split; pilot 10 has four fine-tuning recipes with one value each. The 13 predictions that held in pilots 6 to 10 (of 25) were also written knowing the earlier pilots. These pilots do not validate or refute H2 for real models: its 64% confidence is unchanged.
Other deep dives
- Learning in order
- Throwing away wisely
- A memory inside the model
- The cost becomes logarithmic
- Thinking in ideas, not words
- Switch on only part of the model
- Ask the small model first
- Distil into programs, not models
- Compute with fewer bits
- Change the machine
- Checking is cheaper than writing
- How the seven leads combine
- A small model on a par with a giant?
- A small model that improves on its own, on an ordinary computer?
- Putting the hypotheses to the test
- Translate a model and keep only what is needed?
- H1 under test: our experiments
- How our H1 experiments work
Sources
- OrigoTech, H2 toy pilot 1: facts in the weights or in a memory (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 2: explicit steps or longer training (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 3: updating facts (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 4: imperfect retrieval (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 5: three-fact composition (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 6: distractors and a curriculum (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 7: three facts, longer training (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 8: frequent facts in the weights, rare ones in memory (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 9: how many rule examples (results, JSON; protocol alongside)
- OrigoTech, H2 toy pilot 10: longer or stronger updates (results, JSON; protocol alongside)