Deep dive · Folio IV
Distilling usually means teaching a small model to imitate a big one. There is another way: ask the big model to write a program that solves the problem, then run the program. A program costs almost nothing to run, and it can be read and checked.
Many tasks repeat with different values: convert a date, schedule a meeting, sort a list by a rule. Answering each time with a large model means paying again and again for the same reasoning.
The alternative: use intelligence once to produce a tool, a short program, then reuse it. It is the difference between asking a mathematician for every sum and asking them once to teach you how to add.
In LATM (Large Language Models as Tool Makers, Google DeepMind and Princeton, 2023), a powerful model acts as toolmaker: from a few examples, it writes a Python function, tests it and wraps it with instructions. A lighter model acts as tool user: it recognises the type of problem and calls the function.
Tools are kept in a cache of functions: instead of storing answers in words, the system stores the way to produce them, for a whole family of questions.
Source: Cai et al., Large Language Models as Tool Makers (ICLR 2024)
Fig. H — Pay once, then almost nothing
Large model answers every timeTool written once, then reusedcost of writing the tool
Illustrative diagram: it shows the idea, not measurements.
Promising
On difficult reasoning tasks (Big-Bench), tools written by GPT-4 and used by GPT-3.5 Turbo give the same results as GPT-4 doing everything, when GPT-4 then cost more than 15 times as much per call. The one-off cost of making the tool is spread over every use.
Source: Cai et al., Large Language Models as Tool Makers (ICLR 2024)
Established
Google DeepMind couples Gemini models with automatic evaluators to evolve whole programs. A scheduling rule it found for Google's data centres has run for over a year and recovers on average 0.7 % of Google's computing resources worldwide. It also sped up a key Gemini operation by 23 %, cutting total training time by 1 %.
Established
Multiplying two 4×4 matrices of complex numbers took 49 multiplications since Strassen's method of 1969. AlphaEvolve found a way with 48. A small step, but on an operation repeated billions of times in AI. Here the program is the distilled knowledge: exact, readable, verifiable.
Not everything fits in a program. Translating a poem or catching irony does not reduce to a rule, and that is exactly where large models shine. A tool must also be checked: a wrong tool fails silently at every use. That is why AlphaEvolve only works where an automatic evaluator can score each attempt.
This lead speaks to H3, “rules plus exceptions = compression”. A program is the most compact form of a rule: a few lines replace thousands of examples. What the program cannot handle becomes the exception, left to the model.
Spot the questions that come back often on OrigoTech (currency conversions, percentages, dates), ask a large model to write each function once, then count how many questions the function answers correctly with no model at all.