How AI Actually Works

The one trick at the centre: predicting the next word

Here's the secret that sounds too simple to be true. A chatbot's entire job is answering one question, over and over: "given everything written so far, what word most likely comes next?"

You already have this machinery in your own head. Finish this sentence:

"I take my tea with milk and ______."

You said "sugar" — almost everyone does. You didn't look it up anywhere; a lifetime of hearing sentences taught you that after "milk and", the word "sugar" is overwhelmingly likely, "honey" is possible, and "bicycles" is absurd. An LLM (the language model inside a chatbot) does exactly this — except its feel for "what comes next" was built from a truly enormous amount of text, and it plays the game not just for one blank, but for every single word of its reply, one after another.

"But wait," you might say, "it doesn't just finish sentences — it answers my questions, writes my emails, explains tax rules." True. And here's the elegant part: answering is just very good sentence-finishing. If the text so far is "Q: What's the capital of Australia? A:", then the most likely next words are "The capital of Australia is Canberra." Predict-the-next-word, done well enough, at large enough scale, starts to look exactly like understanding. Whether it "really" understands is a lovely philosophical debate — for getting your work done, it doesn't matter.

What "training" actually means: a billion tiny dials

So where does this feel for language come from? Not from programmers writing rules — nobody typed "after milk comes sugar". It comes from training.

Picture a machine covered in billions of tiny dials, every one set to a random position. In that state, the machine's guesses are garbage. Training works like this: show it a real sentence from a book with the last word hidden, let it guess, then nudge the dials — a hair's breadth each — in whatever direction would have made the right answer more likely. Now do that not once but trillions of times, across a colossal library of text: books, encyclopedias, news, websites, manuals, forum arguments. Slowly, the dials settle into positions that capture the patterns of human language — spelling, grammar, facts, styles, even the rhythm of a joke.

Those dials have a technical name — parameters — and modern models have hundreds of billions of them. When people say a model "learned" something, this is all they mean: the dials got nudged into a shape that makes good predictions. No rules were written. No facts were filed into a cabinet. It's patterns, all the way down — which is exactly why the model is so fluent, and also why it can misremember, as we'll see in a moment.

Tokens: why AI sees "pieces", not letters

One small technicality earns its place here because it explains some famously silly failures. The model doesn't actually read whole words — or individual letters. Text gets chopped into tokens: chunks that are sometimes a whole word ("tea"), sometimes a piece of one ("unbelievable" might become "un-believ-able"), sometimes just punctuation. Common words are one token; rare or long words get split.

Why should you care? Because it explains a classic embarrassment: ask a chatbot how many letter r's are in "strawberry" and it may confidently miscount. To you, the word is ten letters you can point at. To the model, it's a couple of sealed chunks — it never sees the individual letters, so it's guessing at what's inside them, the way you might guess how many bricks are in a wall you only saw from far away. Brilliant at essays, shaky at counting letters: tokens are the reason. (It's also why pricing and limits for AI tools are often quoted "per token" — roughly, a token is three-quarters of an English word.)

One word at a time — and a pinch of dice

Watch a chatbot reply and you'll see the text appear word by word, like a fast typist. That's not a loading animation — it's literally how the thing works. The model predicts one token, adds it to the conversation, then predicts the next token given everything including the one it just wrote, and around it goes until it predicts a special "I'm done" token. It never plans the whole essay in advance and then types it out; it builds the reply brick by brick, each brick chosen in light of all the previous ones.

And here's a detail that surprises everyone: at each step, the model doesn't always pick the single most likely word. It rolls a weighted dice among the top candidates. That deliberate pinch of randomness has a name — temperature — and it's why asking the same question twice gives you two different answers. A little randomness keeps the writing varied and human, instead of every reply about tea sounding like every other reply about tea.

This is genuinely useful to know: if an answer disappoints you, regenerating or re-asking is a legitimate move — you're rolling the dice again. And if two runs of the same factual question give contradicting answers, that's a red flag the model is guessing, not knowing.

Not a database — and why that explains everything

Now we can settle the biggest misconception. A chatbot is not looking your question up in a giant database of stored answers. There is no filing cabinet inside. There are only the dials — a compressed, statistical impression of everything it read, the way you carry an impression of every book you've read without being able to recite any of them.

Hold that picture and both faces of AI make sense at once:

• Why it's so fluent and flexible: because it's not retrieving canned text, it can write things that have never existed — a poem about your cat in the style of a legal notice, a diet plan for your exact budget. Recombining patterns is its native talent.
• Why it confidently gets facts wrong: for the same reason. When you ask about something rare — a small-town office's opening hours, an obscure court case — the honest answer is "I don't remember clearly." But the model's job is to produce likely-sounding text, so out comes a plausible, fluent, fabricated answer, delivered in the same confident tone as the true ones. This failure is called a hallucination, it's the single most important flaw to understand, and it gets a full topic in Chapter 3.

The knowledge cutoff — and the web-search patch

Training a model takes months and staggering amounts of computing power, so it's done in big batches, not continuously. That means every model's built-in knowledge stops at its training cutoff — the date its library was assembled. Ask about anything after that date and, from the dials alone, it simply has nothing. Yesterday's match result, this month's news, current prices: not in there.

The fix is bolted on rather than built in: most chatbots can now search the web, read a few live results, and weave them into the answer — usually with source links you can click. When freshness matters, make it explicit: say "search the web for this" or tap the search toggle if your app has one. A good habit from day one: for anything recent, look for sources in the answer. No sources and no search — assume the model is running on old memory.

What happens when you press Send

Let's put the whole machine together in one journey. You type "Write a polite reminder to my landlord about the leaking roof" and press Send:

1. Your message travels to the company's servers. The model doesn't live on your phone — it's far too big. Your words go over the internet to a data centre.
2. The text is chopped into tokens. Along with the conversation so far, so the model has the full context.
3. The model predicts one token. Billions of dials do their arithmetic and out comes a ranked list of candidates; the dice pick one — say, "Dear".
4. Loop. "Dear" is appended and the game runs again for the next token, and the next, dozens of times per second.
5. The answer streams back to your screen as it's being written — which is why you see it appear word by word — until the model produces its "done" token and stops.

Want to see the machinery admit all this itself? Ask it directly:

Explain to me, in simple non-technical language:
1. How you generate an answer (next-word prediction, tokens) — in 5 lines.
2. What your training cutoff date is, and what that means for questions about recent events.
3. One kind of question I should NOT fully trust you on, and why.

Then, to feel the dice roll: ask the exact same creative question twice — say, "give me a one-line slogan for a tea stall" — and watch the two answers come out different. That's temperature at work. (Once you're set up with a tool from Meet the Tools, your first real conversation is two pages away.)

Key takeaways

• The core trick is next-word prediction: the model repeatedly answers "what word likely comes next?" — done at massive scale, that starts to look like understanding.
• Training means nudging billions of dials over trillions of examples until the guesses get good. No rules written, no facts filed.
• Text is chopped into tokens — which is why a model that writes brilliant essays can miscount the letters in "strawberry".
• It writes one token at a time, with a deliberate pinch of randomness — so re-asking the same question is a legitimate move.
• It's a pattern engine, not a database: that one fact explains both its fluency and its confident mistakes (hallucinations — Chapter 3).
• Built-in knowledge stops at the training cutoff; for anything recent, make sure web search is on and sources are shown.