Chain-of-thought prompting

Chain-of-thought prompting means asking an AI model to write out its steps before it answers, the way you would work out a sum on paper. Asked for an invoice total, it first lists the lines, adds them up and then gives the amount. On questions that take several steps, the answer is right more often. The price is a longer answer: more waiting, and a higher bill.

Where does the term come from?

The name comes from Wei et al. (Google, January 2022). They put eight worked examples in the prompt, each with its intermediate steps written out, a form of few-shot prompting. On GSM8K, a set of maths word problems, Google's PaLM model with 540 billion parameters went from 17.9% correct with ordinary examples to 56.9% with chains of thought, a new best score at the time.

Four months later Kojima et al. showed that one sentence can do the job without any examples. Adding "Let's think step by step" raised the accuracy of OpenAI's text-davinci-002 on the MultiArith sums from 17.7% to 78.7%. This is called zero-shot chain of thought (see zero-shot prompting).

How do you use it?

• Worked examples: show one or more questions with the steps and the answer, in the format you want back.
• One instruction: ask the model to think step by step before it answers.
• Several chains: let the model answer the same question several times and take the most common answer. Wang et al. (2022) call this self-consistency; with PaLM it added 17.9 percentage points on GSM8K.
• Steps apart from the answer: Anthropic suggests tags such as <thinking> and <answer>, so your software can show only the answer.

Is it built into models now?

Reasoning models are trained to write such a chain before they answer, without being asked (see reasoning in AI). Anthropic added this as extended thinking with Claude 3.7 Sonnet in February 2025. With Claude Opus 4.6 in February 2026 came adaptive thinking: the model decides how much to think, steered by an effort level, and Claude Opus 4.7 added the level xhigh in April 2026. In September 2026 thinking is on by default in Claude Sonnet 5 and always on in Claude Opus 5.5, and in both the thinking text stays hidden unless you ask for a summary (thinking).

For these models an explicit chain-of-thought prompt matters less. OpenAI calls asking its reasoning models to "think step by step" unnecessary (reasoning best practices). Anthropic's prompting guide (September 2026) treats manual chain-of-thought prompting as the fallback for when thinking is off.

Where does it go wrong?

• It needs a large model. Wei et al. saw gains only from about 100 billion parameters; smaller models wrote fluent but illogical chains and scored lower than without them.
• Easy questions gain little. The gains were larger on harder problems, and negative or very small on sums that take one step.
• The written chain is not always the real reasoning. When Anthropic slipped models a hint about the answer, Claude 3.7 Sonnet mentioned the hint it used 25% of the time and DeepSeek-R1 39% of the time (Anthropic, April 2025).
• The steps cost tokens. They are output, the expensive part of the bill, and Anthropic bills thinking as output tokens even when you do not see the text.

Example

In the example from Wei et al., the prompt holds one solved question: Roger has 5 tennis balls and buys 2 cans of 3, and the worked answer reads "2 cans of 3 tennis balls each is 6 tennis balls. 5 + 6 = 11." The new question: a cafeteria had 23 apples, used 20 for lunch and bought 6 more. With an example that gave only the answer, the model replied 27. With the worked example, it first wrote 23 - 20 = 3, then 3 + 6 = 9, and answered 9.