Amdahl's law
Speeding up one part of a job only helps for that part. If a step takes half the time, even making it instant saves only half.
Glossary · Tomorrow's AI
Glossary
The notebook's technical words, explained simply. In the pages, the first appearance of each term is underlined with dots and links here.
Benchmark
A set of questions whose answers are known, used to compare models under the same rules. MMLU, GSM8K and AIME are examples.
Cascade (and router)
You ask a small, cheap model first, and only climb to the large one if the answer is not convincing. The router is the program that decides where each question goes. In a mixture of experts, a router likewise chooses which experts to switch on.
Distillation
Passing the knowledge of a large "teacher" model to a smaller "student", by training the student on the teacher's answers. In this notebook we also speak of distilling knowledge into programs.
External memory
A store of texts or notes that the model looks up instead of holding everything in its parameters, the way you open a dictionary rather than learn it by heart.
Inference
Using the model once it is trained: answering a question. That cost is paid for every answer, hence the interest in reducing it.
Loss
The model's error score during training: the lower it is, the better it guesses what comes next. Two methods are often compared "at equal loss".
Memory wall
In a computer, moving a number between memory and the calculating unit costs far more energy and time than the calculation itself. It is one of the main brakes on large models.
Mixture of experts (MoE)
A model split into many sub-networks, the "experts". For each token, a small router switches on only a few of them: a large store of knowledge for a small amount of computation.
Model collapse
What happens when models learn mostly from text written by other models: rare cases vanish first, then quality degrades from one generation to the next.
Parameter (weight)
A number tuned during training. A model holds billions of them, and together they make up what it knows. Picture the billions of small knobs on a huge mixing desk.
Quantisation (fewer bits)
Writing each of the model's numbers with fewer bits. In 16 bits a number can take 65,536 values; in 4 bits, only 16. It is like rounding prices to the nearest euro: lighter, a little less precise.
Reinforcement learning
Learning by trial and error: the model tries, a score rewards it or not, and it adjusts its parameters to get better scores. Many reasoning models are trained this way.
Scaling law
A measured relation between a model's size, the amount of data and the quality obtained. It lets you predict the result of a big training run from small ones.
Speculative decoding
A small model proposes several words ahead, and the large one checks them all in a single pass. The final text is the same as the large model's, but it comes out faster.
Token
The piece of text a model reads: often a short word or part of a word. "Unbelievably" can be three tokens. Data volumes and prices are counted in tokens.
Training
The phase in which the model learns: it reads text, guesses what comes next, and its parameters are corrected after each mistake. It is long and expensive, but done only once.
Verifier (judge)
Whatever says whether an answer is right: tests for code, a calculation for mathematics, or another model. Checking is often much easier than finding.