Ask a language model a multi-step puzzle and it may blurt out a wrong answer. Ask it to work through the problem step by step and the answer often improves. This technique is called chain-of-thought (CoT) prompting, and it is one of the best-known prompting methods. This article explains how it works, when it helps and where it falls short.
What is chain-of-thought prompting?
Chain-of-thought prompting encourages a model to write out intermediate reasoning steps before giving the final answer. Researchers showed in 2022 that providing examples with worked reasoning improved performance on arithmetic, logic and common-sense tasks for large models.
The core idea: a model generates text one token at a time. When it writes the steps, those steps become part of its context and guide the later tokens, which helps with problems that require several linked calculations or deductions.
A quick example
Without steps:
A shop sells pens at 12 rupees each and notebooks at 45 rupees each. Ram buys 3 pens and 2 notebooks and pays with a 200-rupee note. How much change does he get? Answer only with the number.
A model may answer quickly and sometimes incorrectly.
With steps:
Solve the following step by step, then give the final answer on a new line starting with "Answer:".
>
A shop sells pens at 12 rupees each and notebooks at 45 rupees each. Ram buys 3 pens and 2 notebooks and pays with a 200-rupee note. How much change does he get?
A good response walks through the work: pens cost 3 × 12 = 36, notebooks 2 × 45 = 90, total 126, change 200 − 126 = 74. Answer: 74.
Two ways to trigger it
- Zero-shot CoT: simply add an instruction such as "Think step by step" or "Show your reasoning before answering".
- Few-shot CoT: include one or more example problems with detailed worked solutions so the model imitates that style.
For applications, a useful pattern is to separate reasoning from the final result, for example by asking for Reasoning: followed by Answer: so your code can extract just the answer.
When does it help?
Step-by-step prompting tends to improve results on:
- Arithmetic and word problems
- Logic puzzles and multi-step deductions
- Planning tasks with several constraints
- Debugging, where you trace what the code does
- Comparing options against several criteria
It matters less for simple lookups, short factual questions, rewriting or style tasks, where there is little reasoning to unfold. In those cases it only adds length.
A code example
Here is a Python function that asks for structured reasoning and extracts only the final answer:
import re
import anthropic
client = anthropic.Anthropic()
def solve(question: str) -> tuple[str, str]:
prompt = (
"Solve the problem. First write your reasoning under 'Reasoning:'. "
"Then write only the final result on a line starting with 'Answer:'.\n\n"
f"Problem: {question}"
)
reply = client.messages.create(
model="claude-sonnet-5-5",
max_tokens=600,
messages=[{"role": "user", "content": prompt}],
).content[0].text
match = re.search(r"Answer:\s*(.+)", reply)
answer = match.group(1).strip() if match else "NOT FOUND"
return reply, answer
reasoning, answer = solve(
"Pens cost 12 rupees, notebooks 45. Buy 3 pens and 2 notebooks and pay 200. Change?"
)
print(answer)You can show the reasoning to users, log it for debugging, or hide it, depending on your product.
Limits and cautions
- Reasoning can look convincing and still be wrong. A tidy chain of steps is not proof. Check calculations independently, especially in finance or engineering.
- The written reasoning may not reflect the model's true internal process. Treat it as helpful text, not a perfect explanation.
- It costs more. Longer outputs mean more tokens, more time and more money.
- Some newer models reason internally and may need less explicit prompting. Test with and without it for your model.
- Use tools for exact maths. For precise calculation, let the model call a calculator or run code instead of doing arithmetic in text.
Practical tips
- Ask for step-by-step reasoning only on tasks that need it.
- Request a clearly marked final answer for easy parsing.
- Combine with a few examples when you need a specific reasoning style.
- Verify high-stakes results with code or a second check.
- Measure: run a set of test questions with and without CoT and compare accuracy.
Frequently Asked Questions
Does chain-of-thought make a model smarter?
It does not change the model, but it gives the model room to work through a problem in text, which often improves accuracy on multi-step tasks.
Should I always say "think step by step"?
No. For simple questions it only makes answers longer. Use it where the task has several stages.
Can I hide the reasoning from users?
Yes. Ask for reasoning and answer in separate sections and display only the answer, but keep the reasoning for logging and debugging if useful.
Is the explanation always the real cause of the answer?
Not necessarily. It is generated text that usually supports the answer, so rely on verification, not on the story alone.
Conclusion
Chain-of-thought prompting asks a model to show its work, and for multi-step problems that often leads to better answers. Use it selectively, mark the final answer clearly, and always verify anything important with independent checks.