Let us understand what an LLM actually is.
No maths, no jargon. Four things you can click, and by the end you will understand why ChatGPT invents hotel rates, forgets your company overnight, and answers differently every time.
Everyone uses these tools. Almost nobody has been told what is happening inside them — so every strange behaviour feels random.
It is not random. There is one simple mechanism underneath, and once you have seen it, every frustrating thing AI does stops being a mystery and becomes predictable. That is worth fifteen minutes, because predictable means manageable.
LLM stands for large language model. Three words, and each is carrying weight. Language — it works on text, nothing else. Model — it is a mathematical guessing machine, built by reading an enormous amount of writing and adjusting itself until its guesses got good. Large — the machine has hundreds of billions of internal dials. That size is the only reason the guessing feels like thinking.
And what exactly is it guessing? That is where we start.
It never sees your sentence.
Before anything else happens, your text gets chopped into pieces called tokens. A token is roughly three-quarters of a word — common words stay whole, unusual ones get cut into fragments, and the space in front of a word usually travels with it.
Type your own sentence below and watch it happen. Try a supplier name or a small town nobody has heard of, and see how it shatters compared to ordinary words.
The · marks a space. Notice that spaces belong to the word that follows them, common words survive in one piece, and unusual words — place names, supplier names, long words — get chopped up. This is an approximation of the real thing, close enough to see the shape: the model never sees your sentence. It sees this.
This matters for two practical reasons. Tokens are the unit everything is measured and billed in — when someone says a model has “a 200,000 token context window,” they mean it can hold about 150,000 words of chat and attachments at once. And it explains a whole family of odd failures: the model struggles to count letters in a word or do arithmetic on long numbers, because it never sees the letters or the digits. It sees these chunks.
All it does is guess the next token.
This is the whole thing. Given everything it has read so far, the model produces a ranked list of what might come next, with a score against each option. Then it picks one. That is the entire skill it was trained on — no understanding, no intent, no plan for the paragraph.
Below, you are the model. Here is the start of an email; choose what comes next. The percentages are what a real model’s ranked list looks like.
Dear Maria, thanks for your note. The welcome dinner will be
Two things worth noticing. It never picks a word that is wrong for the sentence — after three decades of reading, the grammar is effortless. And it has no idea whether the dinner is actually on Thursday. Both options were fluent. Only one was true, and nothing in the mechanism can tell the difference.
Fluency and truth are separate things. The machine is built for the first one. Everything you have heard about hallucination comes down to this single sentence.
One token is useless. So it runs the loop.
Guessing one word does not write an email. What happens is that the chosen token gets glued onto the end of the text, and the model reads the whole thing again — prompt plus everything it has written so far — to guess the next one.
This is why text appears word by word on your screen: you are watching the loop run in real time, not a finished answer being typed out for effect. It is also why the model cannot revise. It never goes back. If it commits to a bad opening sentence, everything after it is built on that sentence — which is exactly why “try again” often works better than “fix the third paragraph.”
Your chat is not a chat.
Here is the thing that reframes everything. There is no assistant sitting on the other side following your conversation. The model has no memory between messages — none at all. It finishes a response and retains nothing.
So how does it answer follow-up questions? Because your chat app quietly re-sends the entire conversation with every single message. The left panel is what you see. The right panel is what actually gets sent. Step through it.
A conversation. It looks like the assistant is sitting there, remembering.
Draft a welcome email for the Lisbon group. 120 guests, arriving Thursday.
Here is a draft: “Dear guests, we are delighted to welcome you to Lisbon this Thursday…”
One request. No memory attached. The entire history is re-typed and re-read from scratch.
You are a helpful assistant. Today is 14 March. Be concise and professional.
Draft a welcome email for the Lisbon group. 120 guests, arriving Thursday.
Message one is simple enough. Now send the next one and watch what happens to this panel.
A cleaner way to picture it: every message you send goes to a brand-new stranger, who is handed the full transcript of a conversation they were not part of, reads it in one pass, writes the next line, and is then dismissed forever. The next message goes to a different stranger with the same transcript plus one more line.
The continuity you feel is an illusion produced by the app, not a property of the model. Which is oddly good news: the transcript is the only thing that matters, and you fully control the transcript.
Six things that stop being mysterious.
“It invented a hotel rate and stated it with total confidence.”
It was never looking up a rate. It was producing text that fits the pattern of a rate answer, and “€240 per night” fits that pattern beautifully. A wrong number and a right number look equally plausible from inside the prediction. The model has no separate place where facts live and no signal that tells it “I do not actually have this.”
“It was brilliant yesterday and forgot everything today.”
Nothing carried over, because nothing ever does. A new chat is a new transcript. Yesterday you spent ten minutes explaining your company, and that explanation lived entirely inside yesterday’s text. This is also why saved instructions — Projects in ChatGPT and Claude, Gems in Gemini — are so useful: they quietly paste that explanation into the top of every new conversation for you.
“I asked the same thing twice and got two different answers.”
At every single step it is choosing from ranked options, and it rolls the dice rather than always taking the top one. Two runs diverge the moment one pick differs, and then every later pick is reading a different sentence. It is not being inconsistent — it is doing exactly what it always does.
“Attaching the actual RFP works so much better than describing it.”
Every prediction is conditioned on the text in front of it. Your three-line summary gives it three lines to work from; the real document gives it the client’s own words, numbers, and requirements to build on. You are not “giving it information” in the way you would tell a colleague. You are changing the raw material of every prediction it makes.
“Telling it “you are an experienced DMC operations manager” genuinely changes the output.”
It feels like a silly trick, and it is not. Text that follows that sentence in its training looks different from text that follows nothing — more operational vocabulary, different concerns, different structure. You are steering which region of writing it predicts from. Same reason “do not invent rates” helps: it makes hedged, careful text more likely than confident invention.
“Long conversations get vague, or it loses the thing I said at the start.”
Every message re-sends the whole transcript, so the pile grows with each turn. There is a hard limit on how much it can read at once — the context window — and as you approach it, early material gets squeezed or dropped. When a chat gets long and mushy, start a fresh one and paste in only what matters. That is not a workaround; it is working with the mechanism.
You are not chatting. You are assembling a transcript.
That is the whole shift. Every good habit follows from it without needing to be memorised: attach the real documents, because the transcript is the raw material. Say what you do not want invented, because you are steering a prediction. Start a fresh chat when one gets long and muddy. Keep your company information somewhere you can paste it in, since nothing carries over on its own. And check every number before it reaches a client, because fluent and true are different properties and the machine only guarantees one.
None of that is a trick. It is just what the mechanism rewards — and now you can see the mechanism.