What Is Chain-of-Thought Prompting? Chain-of-Thought (CoT) prompting is a technique where the model is guided to write out the intermediate reasoning steps that lead to an answer, instead of producing the answer directly.

In simple terms: you ask the model to "show its work", like a student solving a math problem on paper.

Example: Direct Answer vs Chain of Thought Problem:

A coding bootcamp batch starts with 60 students. 15% leave in the first month. Of the remaining students, one-third choose the evening batch. How many students are in the day batch? Direct prompt:

[problem] Give only the final number. A model answering directly must produce the number without writing any of the intermediate calculations. On multi-step problems like this, direct answers are more likely to contain mistakes, such as applying "one-third" to the original 60 instead of to the remaining students.

Chain-of-thought prompt:

[problem] Solve it step by step, then give the final answer. Possible output:

Step 1: 15% of 60 students leave: 60 × 0.15 = 9. Step 2: Remaining students: 60 − 9 = 51. Step 3: One-third choose the evening batch: 51 ÷ 3 = 17. Step 4: Day batch: 51 − 17 = 34.

Final answer: 34 Example Explanation Each step uses the result of the previous one. Writing "Remaining students: 51" makes it far more likely that the next step applies "one-third" to 51 rather than 60. The final answer is built on visible intermediate results, and you can check every step.

Where the Technique Comes From Chain-of-thought prompting was introduced in the 2022 research paper "Chain-of-Thought Prompting Elicits Reasoning in Large Language Models" by Wei et al. The authors included a few examples in the prompt where each answer was preceded by step-by-step reasoning, and showed large improvements on arithmetic, commonsense, and symbolic reasoning benchmarks.

Two findings from that research are useful to remember:

The improvement was largest on multi-step problems. In their experiments, the benefit appeared mainly in large models; smaller models often produced fluent but illogical reasoning that did not help. Later work showed that a simple instruction such as "Let's think step by step" can trigger similar behavior without examples (Lesson 40).

Why Chain of Thought Works Recall from Lesson 2 that a model generates text one token at a time, and each new token is predicted from everything already in the context.

The Reasoning Becomes Part of the Context When the model writes "Remaining students: 51", that text is now in the context. The next prediction can build on it directly. Without written steps, the model must arrive at the final answer in a single prediction with no intermediate results to work from.

A useful way to think about it:

Written reasoning acts as the model's scratch paper. Each step it writes becomes information it can use for the next step.

Complex Problems Become a Series of Simple Ones Each individual step ("60 × 0.15 = 9") is a simple, common pattern that the model is likely to get right. A complex problem solved in one jump requires the model to handle all of the steps implicitly at once, which is more error-prone.

Errors Become Visible When an answer is wrong, the reasoning shows where it went wrong: a misread condition, a wrong formula, or an arithmetic slip. This makes debugging prompts much easier.

The Basic Patterns Pattern 1: Ask for Step-by-Step Reasoning [problem] Think through this step by step before giving your final answer. This is zero-shot CoT, covered in Lesson 40.

Pattern 2: Show Examples With Reasoning Q: A course costs ₹2,000 with a 10% discount. What is the price after discount? A: 10% of 2,000 is 200. 2,000 − 200 = 1,800. The answer is ₹1,800.

Q: [new problem] A: This is few-shot CoT, the original form of the technique, covered in Lesson 41.

Pattern 3: Specify the Steps When you know how the problem should be solved, describe the steps explicitly:

Decide whether the student qualifies for a certificate.

1. Check attendance against the 75% requirement.
2. Check whether all 5 assignments were submitted.
3. Check whether the project score is at least 60.
4. The student qualifies only if all three checks pass. Show the result of each check, then give the decision. This structured form is often the most reliable in real applications, because it guides the reasoning toward the correct procedure.

Separating Reasoning From the Final Answer In applications, you usually need to extract the final answer reliably, while optionally keeping the reasoning for review. Use delimiters for each part (Lesson 37).

Solve the problem below.

Write your step-by-step reasoning inside tags. Then write only the final numeric answer inside tags.

Problem: A coding bootcamp batch starts with 60 students. 15% leave in the first month. Of the remaining students, one-third choose the evening batch. How many students are in the day batch? Possible output:

15% of 60 is 9, so 51 students remain. One-third of 51 is 17, who choose the evening batch. 51 − 17 = 34 students are in the day batch. 34 Why This Structure Helps The reasoning comes before the answer, so the answer is generated with the reasoning in context (Lesson 10, Tip 4). The answer is easy to extract with code. The reasoning can be stored for debugging or shown to users when explanations are useful, such as in a tutoring application. Chain of Thought in Python The following example asks for reasoning and an answer in separate tags, extracts the answer, and checks it against a value computed in code.

Requirements: pip install openai and an OPENAI_API_KEY environment variable.

import re

from openai import OpenAI

client = OpenAI() MODEL = "gpt-5.4" # Replace with a model available in your account

INSTRUCTIONS = """Solve the problem in . Write your step-by-step reasoning inside tags. Then write only the final number, with no units, inside tags."""

def extract_tag(text: str, tag: str) -> str | None: match = re.search(rf"<{tag}>\s*(.?)\s</{tag}>", text, re.DOTALL) return match.group(1) if match else None

def solve(problem: str) -> tuple[str | None, str | None]: response = client.responses.create( model=MODEL, instructions=INSTRUCTIONS, input=f"\n{problem}\n", ) text = response.output_text return extract_tag(text, "reasoning"), extract_tag(text, "answer")

problem = ( "A coding bootcamp batch starts with 60 students. 15% leave in the first " "month. Of the remaining students, one-third choose the evening batch. " "How many students are in the day batch?" )

reasoning, answer = solve(problem) print("Reasoning:\n", reasoning) print("Answer:", answer)

Verify with code

remaining = 60 - 60 * 15 // 100 expected = remaining - remaining // 3 print("Correct" if answer == str(expected) else f"Mismatch: expected {expected}") Code Explanation INSTRUCTIONS Requests reasoning first and a bare number in . "With no units" prevents answers like 34 students, which would complicate comparison.

extract_tag() The same tag-extraction function from Lesson 37. It returns None when a tag is missing, so your code can detect malformed responses.

solve() Returns both parts separately. Applications can log the reasoning while using only the answer.

Verification The expected answer is computed in Python with integer arithmetic: 60 * 15 // 100 is 9, so 51 remain; 51 // 3 is 17; and 51 - 17 is 34. When exact correctness matters and the calculation can be expressed in code, verifying (or simply computing) in code is more reliable than trusting the model (Lesson 24).

Chain of Thought Beyond Math Chain of thought is not only for arithmetic. It helps any task that requires several dependent decisions.

Applying Multiple Rules Decide whether the refund request is eligible.

Policy:

• Refunds within 7 days of purchase
• Only if less than 20% of the course is completed
• Not available for courses bought with a 100% scholarship coupon

Request: Bought 4 days ago with a 50% coupon. Completed 12% of the course.

Check each policy condition one by one, then give the decision. Possible output:

1. Within 7 days: bought 4 days ago → met.
2. Less than 20% completed: 12% → met.
3. Not a 100% scholarship coupon: used a 50% coupon → met.

Decision: Eligible Debugging Code The function below returns the wrong total. Trace its execution step by step for the input [10, 20, 30], showing the value of each variable after each line, then identify the bug. Tracing step by step is a form of chain of thought that is especially useful for code.

Multi-Step Classification First identify what the customer is asking us to do. Then decide which category that request belongs to. Then give the category. Chain of Thought and Modern Reasoning Models Some newer models are trained to reason internally before producing a visible answer. These are often called reasoning models. For them:

Explicit "think step by step" instructions may be unnecessary, and some providers advise against them, because the model already reasons internally. Clear problem statements, constraints, and success criteria remain important. You may not be able to see the model's internal reasoning, or you may only receive a summary of it, depending on the provider. For standard (non-reasoning) models, explicit chain-of-thought prompting remains a valuable technique. Check your provider's documentation for the model you use, and compare results with and without CoT instructions on your own test set (Lesson 43).

Costs and Trade-Offs Benefit Cost Higher accuracy on multi-step tasks More output tokens, which cost more Visible, checkable reasoning Longer response time Easier debugging Must extract the answer from a longer response For simple tasks, these costs bring no benefit. Lesson 43 explains how to decide when chain of thought is worth using, and Lesson 44 covers its limitations, including the fact that written reasoning is not always a faithful description of how the model actually reached its answer.

Common Mistakes Mistake 1: Answer Before Reasoning What it is: Asking for "the answer, then an explanation".

Why it happens: It feels natural to state the conclusion first.

What it causes: The answer is generated before any reasoning exists, so the explanation only justifies an answer already chosen.

How to fix it: Always place reasoning before the final answer.

Mistake 2: No Clear Final Answer Marker What it is: Asking for step-by-step reasoning without specifying where the final answer goes.

Why it happens: The reasoning is the focus during development.

What it causes: Code cannot reliably find the answer in a long response.

How to fix it: Use a final-answer tag or a fixed line such as Final answer:.

Mistake 3: Using CoT for Every Task What it is: Adding "think step by step" to translations, simple extractions, or clear classifications.

Why it happens: CoT is known to improve accuracy, so it is applied everywhere.

What it causes: Higher cost and latency without improvement, and sometimes worse results (Lesson 43).

How to fix it: Use CoT for multi-step tasks and measure its benefit.

Mistake 4: Trusting Reasoning Because It Looks Logical What it is: Accepting an answer because the steps read well.

Why it happens: Clear reasoning feels convincing.

What it causes: Errors in a single step, such as an arithmetic slip, pass unnoticed.

How to fix it: Verify important results, ideally with code for calculations.

Mistake 5: Showing Raw Reasoning to Users by Default What it is: Displaying the full reasoning text in every response.

Why it happens: It is the easiest thing to display.

What it causes: Long, cluttered responses, and sometimes exposure of internal rules written in the reasoning.

How to fix it: Separate reasoning and answer with tags, and decide deliberately what users should see.

Best Practices Use chain of thought for multi-step problems. Arithmetic, multi-rule decisions, logic, tracing, and planning.

Put reasoning before the answer. The answer should be generated from the reasoning, not the other way around.

Separate reasoning and answer with delimiters. Extract the answer reliably in code.

Specify the steps when you know the procedure. Structured reasoning is more reliable than open-ended reasoning.

Verify critical results. Use code for calculations; review reasoning for logical errors.

Check model-specific guidance. Reasoning models may not need explicit CoT instructions.

Measure the benefit. Compare accuracy, cost, and latency with and without CoT.

Try It Yourself Exercise 1: Direct vs Step-by-Step Write five multi-step word problems from your own context (fees, discounts, schedules, batch sizes). Ask a model to solve each one twice: once with "Give only the final number" and once with step-by-step reasoning. Verify every answer with Python and compare the accuracy of the two approaches.

Exercise 2: Structured Reasoning for Rules Write a prompt that decides whether a job applicant meets four requirements (degree, years of experience, a required skill, and location). Use Pattern 3 to specify the checks. Test it with five applicants, including one who fails only the last requirement.

Exercise 3: Extract and Verify Modify the Python example so that solve() raises an error when either tag is missing, and so that the answer is converted to an integer before comparison. Test it with a problem whose answer is not a whole number and decide how your code should handle that case.