Here’s a small experiment you can try. Ask a language model a slightly tricky word problem and demand just the final number, and it may well get it wrong. Ask it the exact same problem but add four little words, “let’s think step by step,” and it often gets it right. Same model, same question, different accuracy. The only thing that changed was that you let it work through the problem.
That’s chain-of-thought prompting, and it’s one of the most reliable ways to get better answers on anything involving reasoning, logic, or multiple steps. It’s also a useful window into how these models work. Let me show you.
Why “blurting” leads to mistakes
Remember from the AI Primer that a model generates its answer one token at a time, each token building on the ones before it. Now think about what happens when you force it to jump straight to a final answer on a multi-step problem. It has to produce the whole result in one leap, with no room to work through the intermediate steps. That’s a bit like being asked to shout out the answer to a maths problem instantly, without scratch paper. Even a capable person makes careless mistakes under that pressure.
The model has the same trouble. When it commits to an answer immediately, it hasn’t given itself the intermediate reasoning it needs to get there reliably. The steps that should lead to the answer never got written down, so they never got “thought.”
The fix: let it show its working
The remedy is simple: ask the model to work through the problem step by step before giving the final answer. When it writes out its reasoning, each step becomes part of the context for the next one. The model is, in effect, building itself a set of stepping stones to the answer, like the scratch paper it was missing. Figure 1 shows the contrast.
Figure 1: Asked for the answer directly, the model leaps and often stumbles. Asked to reason step by step, it lays down intermediate steps, each one supporting the next, and reaches the correct answer more reliably.
Here’s a concrete example. Consider this problem:
“A shop has 120 apples. It sells 30% of them on Monday, then 25% of what remains on Tuesday. How many apples are left?”
Ask for just the number, and a model might blurt out something wrong, tripping over the “25% of what remains” twist. Now ask it this way:
“A shop has 120 apples. It sells 30% of them on Monday, then 25% of what remains on Tuesday. How many are left? Think step by step, then give the final number.”
And it reasons through it:
“Monday: 30% of 120 is 36 sold, leaving 84. Tuesday: 25% of 84 is 21 sold, leaving 63. Final answer: 63.”
By working through Monday before tackling Tuesday, it handles the “what remains” part correctly because the number 84 was written down and available when it got to Tuesday’s step. The reasoning wasn’t just for show; it produced a better answer.
The magic words (and when to use them)
The classic trigger phrases are simple: “think step by step,” “show your reasoning,” or “work through this before answering.” Any of them invites the model to reason rather than rush.
Chain-of-thought helps most when the task involves:
- Arithmetic or maths of more than one step.
- Logic puzzles or problems with conditions to track.
- Multi-step decisions where one thing depends on another.
- Anything where you’d naturally reach for scratch paper yourself.
For simple lookups or straightforward requests (“what’s the capital of Japan?”), it’s unnecessary. There’s nothing to reason through, and asking for steps just adds clutter. As always, match the technique to the task.
A practical wrinkle for real use
There’s one catch worth planning for. Chain-of-thought makes the model verbose. It writes out all its reasoning, which is exactly what you want when you’re checking its logic, but often not what you want when an app just needs the final answer.
The fix is to ask for both, clearly separated. Tell the model to reason first and then put the final answer on its own line, in a form you can easily pick out:
Solve this problem. Think step by step, then write the final answer
on a new line in exactly this format: ANSWER: <number>
Problem: A shop has 120 apples. It sells 30% on Monday, then 25% of
the remainder on Tuesday. How many are left?Now the model reasons through it and ends with a clean ANSWER: 63 line that a program can grab, while a human can still scroll up to check the working. You get the accuracy benefit of reasoning without drowning your application in text.
Why this works, and a small caution
It’s worth understanding why this simple trick is so effective. You’re not making the model smarter; you’re giving it room to use the ability it already has. By spreading the problem across several steps instead of one leap, you let each token of reasoning inform the next, turning a hard single jump into a series of easier small ones. It’s the same reason we humans reach for scratch paper: thinking on paper is easier than thinking all at once.
One honest caution: chain-of-thought reduces mistakes, but it doesn’t eliminate them. The model can still reason its way confidently to a wrong answer. The reasoning looks sound but contains a slip. So for anything that matters, read the steps and check them. Visible reasoning helps because it lets you see where the model went wrong, not just that it did.
Chain-of-thought is a workhorse technique for reasoning. We’ve now covered several valuable prompting moves: structure, examples, and reasoning. In the next article, I’ll pull the most important techniques together into one compact toolkit: a small set of patterns that handle most real prompting tasks.
Next in the series: The 7 Prompt Patterns That Cover 90% of Cases: a practical prompting toolkit in one place.