If you take just one idea from this series into your daily use of AI, I’d want it to be this one. Language models can be confidently, fluently, completely wrong. They can invent facts, make up quotes, cite books that don’t exist, and state all of it with the same calm assurance they’d use for something perfectly true. This behaviour has a name: hallucination. Understanding why it happens is the difference between using these tools wisely and trusting them too quickly.
The reassuring news is that hallucination isn’t a mysterious glitch. It follows directly from everything we’ve built up in this series. Once you see the cause, the behaviour makes sense, and you’ll know how to protect yourself.
The root cause: plausible, not true
Let’s return to the single most important fact about how these models work. A language model was trained to do one thing: predict the next token that would plausibly come next. Not the true next token. The plausible one. The one that fits the pattern of how text usually flows.
Sit with that distinction, because it’s the whole explanation. The model is, at its core, a system for producing text that sounds right. Truth was never directly part of the game it was trained on. It played guess-the-next-word, optimising for plausibility, for fluency, for fitting the pattern. Being correct is something that often comes along for the ride (because true things appear a lot in its training text), but it was never the actual target.
So what happens when you ask a model something it doesn’t really know? It doesn’t have a little voice that says “I’m not sure about this.” It just does what it always does: generates the most plausible-sounding continuation. And a plausible-sounding continuation to “The 1923 Nobel Prize in Physics was awarded to…” is a confident-sounding name, whether or not it’s the right one. The model fills the gap with something that fits, delivered with fluency, because fluency is what it was trained for. Figure 1 captures this.
Figure 1: The model aims for what sounds plausible, not what is true. When it doesn’t know something, it doesn’t stop. It generates a confident, plausible-sounding answer, which may be fiction.
An analogy that makes it click
Here’s the picture I find most helpful. Imagine an incredibly well-read improviser, someone who has absorbed a vast library and can talk fluently about almost anything, but who has one quirk: they will never say “I don’t know.” Their whole instinct is to keep the conversation flowing smoothly, to always produce a confident, well-formed response that fits.
Ask this person something they know, and you get a good answer. Ask them something obscure that they don’t know, and they won’t stop or hesitate. They’ll smoothly improvise something that sounds exactly like a real answer, complete with confident details, because producing fluent, fitting responses is simply what they do. They’re not lying, in the sense of knowingly deceiving you. They have no reliable internal signal separating “things I know” from “things I’m making up.” That’s a language model.
Why you can’t just “fix” it
A natural reaction is: can’t we just patch this out? The honest answer is that we can reduce it a great deal, but we can’t fully eliminate it, because it’s not a bug bolted onto the system. It’s a direct consequence of how the system fundamentally works. The same machinery that lets a model write fluently about things it does know is what lets it write fluently about things it doesn’t. Fluency and hallucination come from the same source.
The alignment step from Article 15 helps. Models can be nudged toward admitting uncertainty and toward accuracy, and newer models hallucinate less than older ones. But underneath the polish, it’s still a plausibility engine, and it can still be confidently wrong. This is why a little healthy skepticism is not paranoia; it’s just understanding the tool.
How to protect yourself
Here’s the useful part. Because we understand why hallucination happens, we know how to guard against it. Figure 2 lays out the main defences.
Figure 2: Four practical defences: give the model real source material to work from, ask it to flag uncertainty, verify anything that matters, and lower the temperature for factual tasks.
- Give it the facts to work from (grounding). The single most powerful defence. Instead of relying on the model’s fuzzy memory, hand it the actual source text and ask it to answer from that. This is the whole idea behind retrieval (RAG): fetch real, relevant material and put it in front of the model so it’s reading rather than guessing. A model answering from a document you gave it is far more trustworthy than one answering from memory.
- Invite uncertainty. Models will more readily admit the limits of their knowledge if you explicitly allow it. Asking something to tell you when it isn’t sure, rather than always producing an answer, helps.
- Verify what matters. Treat the model’s output as a confident draft, not gospel. For anything important, such as a fact, a figure, a quote, a citation, a line of medical or legal or financial reasoning, check it against a reliable source. Especially be suspicious of specific-sounding details like exact statistics, names, dates, and references, which are prime hallucination territory.
- Lower the temperature for facts. Remember the creativity dial from Article 16? For factual work, a lower temperature keeps the model on the safe, likely path and reduces the chance of it wandering into invention.
The mindset that keeps you safe
The healthiest way to hold all this is to keep two truths in mind at the same time, without letting go of either. Language models are very capable and often useful. And they are, at bottom, pattern-completers that optimise for plausible text, which means they can be confidently wrong, and it’s on you to verify what matters.
People who believe only the first truth get burned when a model invents a fact they trusted. People who believe only the second dismiss a powerful tool. Holding both is what lets you get value from these systems while staying safe: lean on them for what they’re good at, ground them in real information when you can, and verify anything that counts. Hallucination isn’t a reason to distrust AI entirely. It’s a reason to use it with your eyes open.
We’ve now covered how these models work, all the way to their most important limitation. In the final article, let’s zoom out and take an honest, balanced look at the whole picture: what these systems are genuinely great at, where they fall short, and where things might be heading.
Next in the series: Capabilities, Limits & Where This Is Heading: a wrap-up of what LLMs can and can’t do.