The whole thing, running

You now own every part. This chapter adds the last three small pieces, bolts everything together, and switches it on. You will watch a real model learn, look inside it while it thinks, and let it write.

A job to be wrong at

Every machine in this article learned the same way. It made a guess, measured how wrong it was, and turned its knobs downhill. So before the big machine can learn anything, it needs a job where its guesses can be checked.

For the bike rides, someone had to go out with a stopwatch. For language there is a job that needs no stopwatch at all. Take any text. Cover up a word. Ask the machine what it is. Then uncover it and check.

The version every large language model uses is the simplest one: guess the next word. Do it yourself first. The stories below come from a tiny world, where a few people own a few things.

Those three rounds are the whole job, and each one taught something different.

Round 1 was about pattern. A name is followed by “has”. To get it right, the machine has to learn how sentences are built.

Round 2 could not be known. So the right output is not one word. It is a bet: a share of 100% for every word that could come next. That is a job for softmax, from chapter 7.

Round 3 needed you to look back. You found the right earlier words and ignored the others. That is attention, from chapter 8.

Guess the next word. Every word of every text ever written is a free example, with the answer attached.

One thing has to change from chapter 2. The guess is now a set of shares, so “how far off” needs a new meaning. Look at the share the machine gave to the word that really came next. If that share was 100%, the machine was not wrong at all. If it was tiny, the machine was badly surprised. Wrongness is that surprise.

Which word came first?

Chapter 8 ended with a gap. Attention mixes words, and a mix does not care about order. Add up the same things in a different order and you get the same total.

For language that is a disaster. Here are two sentences made of the same three words.

The fix uses a trick you already know. In chapter 6, every word got a place: a list of numbers, learned by the machine. Now every position gets one too. There is a list for “1st”, a list for “2nd”, and so on.

These lists belong to the positions, not to any word. The list for “1st” is the same whatever word happens to be first. And each one is exactly as long as a word’s list, so that the two can be added.

Before a word goes into the machine, the place of its position is added to the place of the word, number by number. That is the adding of places from chapter 6.

How many position lists are needed? One for each position a word can be in. If the longest text the machine will ever read has 20 words, it needs 20 of them. It is a count of positions, and has nothing to do with how many different words the machine knows.

What goes into the machine is no longer “dog”. It is “dog, in 1st position”. From there on, questions and labels can ask about position as easily as about meaning: “who is two words back from me?”

Bolt it together

Two more small decisions, and the machine is complete.

Room to think. Attention moves facts from word to word. But it is only a mix, and chapter 5 showed that a mix cannot bend. So after each round of attention, every word goes through a small network of neurons, by itself, to work on what it just gathered.

More than one round. One round of attention lets each word hear from the others once. Do it again, and a word can ask about what the others learned in the first round. You will see in a moment why that matters.

Here is the result. Nothing in it is new.

Switch it on

I trained this exact machine on stories from the tiny world, with the loop from chapter 3. Make a bet, measure the wrongness, pass the blame backwards through every part, turn all 9,717 knobs a little downhill. Twenty thousand times.

That takes a couple of minutes on a laptop, so I did it ahead of time and saved the knobs at eleven moments along the way. What you see below is not a recording, though. At each moment, your browser takes the saved knobs and runs the machine for real.

The order in which it learned things is worth a second look.

The pattern came first. Within a hundred steps, it wrote stories that were shaped right. Learning “a name comes after a full stop” removes a lot of wrongness for very little work, so the slope in that direction was steep.

Then it got stuck. For well over a thousand steps the wrongness hardly moved. The machine was betting on a thing that had been mentioned, often without knowing whose it was. The green line shows it: on test stories, it put less than half of its bet on the right thing. The valley was nearly flat there.

Then it found the look-up. The green line jumps, and the wrongness drops to its floor. The test stories behind the green line all contain a pair the machine never saw while learning, so the jump is not memory. It is the look-up itself.

Notice where the wrongness started: about 3, the number for a machine that spreads its bet over all 21 words. And notice that it never reaches zero. It cannot. Nobody can know what Anna will have before the story says so. A perfect machine in this world would still be surprised by every new fact.

Two honest notes. The step I used is a slightly cleverer version of the one in chapter 3: it gives every knob its own step size and adjusts it as it goes. You met the reason in chapters 3 and 4, where different knobs needed different step sizes. And learning like this depends on luck in the random starting knobs. I tried several starts, and not all of them found the look-up.

Look inside while it thinks

The machine gets the answer right. But how? In chapter 8 I set the questions and labels by hand. Here nobody did. All 9,717 knobs were found by the loop. So let us open it up and see what the loop built.

Here is what the machine worked out for itself, in plain words.

In round 1, every thing asks: whose am I? “Dog” looks back at “ben”, three words before it, and mixes in his message. After round 1, the list for “dog” is no longer just “dog”. It is “dog, belonging to Ben”.

Also in round 1, the last word asks: who is this about? It looks two words back and picks up the name.

In round 2, the last word asks: which thing belongs to that person? This question could not have been asked in round 1. Before round 1, “dog” did not know whose it was. That is what a second round is for.

Nobody wrote these questions. The machine was told only to make its bets less wrong, and this is what turning the knobs downhill produced.

Did it learn the idea, or memorise the stories? While it was learning, I kept four pairs back, such as “ben has a kite”. It never saw them once. Afterwards I tested it on stories with those pairs and two people. It got 255 out of 256 right. It learned the look-up, which works for any name and any thing. With three people it is less sure, as you may have found.

Let it write

So far the machine has only made bets. To make it write, add one small loop around it. Get the bet. Pick a word by chance, in proportion to the shares. Add that word to the story. Feed the whole story back in. Repeat.

Each word is chosen without any plan for the words after it. And still the story holds together, because every new bet is made by looking back over everything written so far, including the machine’s own earlier words.

The boldness control is something you already did in chapter 7. Turning it down is “double every score”: the gaps grow, and the favourite takes nearly everything. Turning it up shrinks the gaps, and the machine starts to gamble. In real products this control is called temperature.

A machine that only bets on the next word, run in a loop, writes. When you chat with a large model, this loop is what is running. Your message is the start of the story. Its reply is the words it adds, one at a time.

From this to the big ones

Our machine has 9,717 knobs and knows 21 words. What separates it from the ones in the news?

Size. They have a hundred billion knobs or more, as chapter 1 promised. Each word’s list has thousands of numbers, not 24. There are dozens of rounds, not two.

Several questions at once. In each round, each word asks not one question but many, side by side, each with its own knobs. One might ask “who is this about?” while another asks “what tense are we in?”. Each of these is called a head.

Pieces of words. They do not have a place for every word. They cut text into common pieces, called tokens, so that “unhelpful” can be built from “un”, “help” and “ful”.

A few steadying parts. With so many rounds, the numbers can grow or shrink out of hand, so there are small extra steps that keep them in range. They change nothing about the idea.

The text. Not stories about Anna’s kite, but a large part of everything people have written. To guess the next word of a proof, it helps to follow the maths. To guess the next word of a story, it helps to keep track of who knows what. The job never changes. What it takes to do the job well is what grows.

And one honest limit. We could open our machine and read its questions, because it is tiny. Doing the same for a machine with billions of knobs is an open research problem. People know exactly how these machines are built, and still do not fully know what they have learned inside.

The whole road

Look back at where you started, and how each chapter handed something to the next.

1. One knob. A model is a rule with a number you can turn.
2. How wrong? One number that says how badly the rule fits.
3. Which way to turn. The slope of the wrongness tells each knob which way is downhill.
4. Many knobs. A weighted sum: several things mixed in learned amounts.
5. Bends. Neurons that cut off at zero, so the rule can take any shape, and blame passed backwards so that deep knobs can learn.
6. Words as places. A word becomes a list of numbers that the machine learns.
7. Agreement. A dot product to score a match, and softmax to turn scores into shares.
8. Attention. A mix where the words work out the amounts themselves.
9. The whole thing. Places, attention and neurons, stacked, and trained to guess the next word.

At no point did the idea change. In chapter 1 you turned one knob by hand until a ghost rider kept up with a real one. The machine you just watched does the same thing, 9,717 knobs at a time. The largest ones do it with a hundred billion or more.

A rule with knobs, and a way to tell which way to turn them. That is all it ever was.