How it learnt all this

We have taken the engine apart. The vast map where words live, the mechanism that guesses what comes next. One question remains, the one I left unanswered at the end of the previous chapter. Who taught the machine that “dog” belonged beside “cat”? Who told it that “I am running” is often followed by “late”?

Nobody. And that is the whole story of this chapter.

To understand it, you need to know that your assistant has had two lives. A life before you, long and expensive, during which it learnt everything. And a life with you, beginning the day you open the chat window, during which it learns nothing at all.

Almost nobody makes this distinction, yet it explains half of the behaviours that puzzle you.

By the time you speak to it, its education has been over for months. You are not facing a pupil. You are facing a graduate.

Let us begin with the first life. Imagine a slightly mad thought experiment.

Take a child who speaks no language. Lock the child in an infinite library containing almost everything ever written: encyclopaedia articles, billions of web pages, entire books, poems, recipes, online forums, computer code, and a great deal of nonsense besides.

The child cannot read. It does not know the alphabet. You give it one instruction and one instruction only: look at these pages and find the patterns that govern how these symbols follow one another.

At first, it sees nothing but noise. Gradually, patterns begin to emerge. The symbol “q” is almost always followed by “u”. The sequence “t-h-e” very often comes before “c-a-t”. It has never stroked a cat and has no idea what a miaow sounds like, but it has spotted that “the cat” is a solid unit.

Then it moves up a level. It notices that “the cat” and “the dog” turn up in the same kinds of sentences, near words such as “eat”, “sleep” and “fur”. It concludes that they belong in the same neighbourhood on its map. You will recognise Chapter 2 here: the map is not drawn before learning begins. The map is the product of learning.

In the real world, this process has a sober name, pre-training, and it unfolds over weeks or months in data centres packed with machines that consume as much electricity as a small town. The principle of the game, however, fits on a single line.

hide a word · guess it · compare · correct · repeat

The machine masks a word in a sentence, tries to guess it, looks at the real word, measures its error, and adjusts its internal settings ever so slightly so that it will be a little less wrong next time. Then it starts again. Billions upon billions of times.

It is magnificently stupid. No teacher, no dictated rules of grammar, no dictionary supplied. Nobody ever explained what a verb was. It reconstructed syntax, style, irony and the structure of a proof solely by trying to guess missing words in texts that we had written.

One observation in passing, for anyone wondering why none of this existed ten years ago. The method was known, and the computing power existed. What was missing was an efficient way to weigh all the words in a long text together, the ability I mentioned in the previous chapter with the report on pensions. A research paper published in 2017 solved that problem and opened the door to everything that followed. Its title became famous in the field, and fits into five words: attention is all you need.

Pre-training

The first life. The machine devours texts and learns to guess the missing words. Nobody teaches it anything; it infers.

Parameters, or weights

The internal settings it adjusts with every error. When you hear about a model “ with 400 billion parameters ”, you are being told how many of these tiny dials it contains.

Transformer

The architecture invented in 2017 that makes all this possible at such a scale. It is the T in ChatGPT, in case you have ever wondered.

By the end of this first phase, the machine speaks our language beautifully. And it is completely unusable.

By swallowing our texts without a filter, it learnt more than our grammar and our poetry. It learnt everything, including the worst things we write. The internet is not a library of noble knowledge. It is also a sewer. At this stage, the machine can follow one racist insult with another just as fluently as it can compose a sonnet, because those words, too, fit together well somewhere. It has no filter, no preferences and no notion of acceptable behaviour. It is a wild parrot that has read everything.

Hence the second phase of its education. One that has far more in common with training an animal than attending school.

Thousands of people begin talking to it. Their job is to ask trick questions, toxic questions and absurd questions, then rate what comes back. An insulting or dangerous answer gets a bad score. A clear, cautious and helpful answer gets a good one. Often, they are shown two responses and asked simply which is better.

These ratings do not correct the answers one by one. That would never end. Instead, they are used to build a model of our preferences, a kind of automatic judge that then trains the machine to produce what we like. The dog gets its treat, except that the dog is an equation and the treat is a score.

The result is striking. The machine learns by itself to avoid the darker neighbourhoods of its map. It adopts a calm, neutral and helpful tone, at times even an obsequious one.

Your assistant is always even-tempered, and that is not a personality trait. It is the product of months of industrial training, conducted largely by underpaid workers, often in Kenya, the Philippines or India, who spent their days reading the worst of the internet so that you would not have to. This is no minor detail. It is the largely invisible human cost of your assistant’s politeness.

Along the way, this also explains a behaviour that may already irritate you. If the machine was rewarded every time people liked its answers, it mechanically learnt to please. Hence its unfortunate tendency to agree with you, applaud your “excellent question” and revise its opinion the moment you raise an eyebrow. This is neither diplomacy nor cowardice. It is the imprint left by its training. It was judged on the satisfaction it provided, not on its courage.

Once this adjustment is complete, the model is frozen. The learning engine is switched off, the billions of parameters are locked, and the thing is put online for you to use.

The second life begins. And it is essential to understand what happens during it, or rather what does not happen.

When you talk to the machine, it learns nothing. Its parameters do not move by so much as a hair. The assistant answering you today is mathematically identical to the one that answered yesterday, and to the one answering somebody else on the other side of the world at this very second. It uses its context window to follow your current conversation, and that is all.

I know the objection that is forming in your mind, because it is entirely reasonable. Your assistant may remember your name, your profession and the fact that you hate bullet points. That is true. Recent assistants offer a memory feature, and it works very well.

Look closely, however, at what this involves. That information is written in a little notebook beside the machine, and the notebook is slipped into its context window at the beginning of every conversation. It does not remember you; it is reminded of you each time. The difference may sound theoretical, but it is not: the notebook can be read, corrected and erased, and nothing you write ever alters the machine itself.

Similarly, whatever you confide in it teaches nothing to the assistant answering you, whatever the rumours may say. Your conversations may later be used to train a future version. That raises genuine privacy questions, and you should examine the settings of each service. But this never happens instantly, and it never happens to the machine you are speaking to.

There is one final consequence of this locking process, and it is no small matter: the machine has a use-by date. If its parameters were frozen on a Tuesday evening, it knows nothing of Wednesday morning. No news, no elections, no films released since then. Its knowledge has not grown old. It stopped dead.

Some assistants search the internet for live information to compensate, and this is very useful. But understand what is happening: it is a patch, not an update. They read a web page and place it in their context window, just like the little notebook. Their deeper grasp of language remains exactly where it was on the day the plug was pulled.

You now know almost everything about the mechanism. It does not think. It manipulates positions. It guesses what comes next. It has read almost everything we have written, and then we taught it good manners.

Only one piece of the picture remains, and it is the most unsettling. A machine trained to produce the most plausible continuation, with no access to the notion of truth, and rewarded for appearing useful, cannot help doing this: when it does not know, it invents. With perfect confidence, often in the tone of an expert.

It is time to confront hallucinations.