Zero-Shot Chain-of-Thought Prompting

What Is Zero-Shot Chain-of-Thought? Zero-shot chain-of-thought (zero-shot CoT) is chain-of-thought prompting (Lesson 39) without any reasoning examples. Instead of showing worked solutions, you add a short instruction that tells the model to reason step by step before answering.

The best-known version adds one sentence:

Let's think step by step. Example Prompt:

Q: A student watches 3 lessons on Monday, twice as many on Tuesday, and 4 fewer on Wednesday than on Tuesday. The course has 20 lessons. How many lessons are left after Wednesday?

A: Let's think step by step. Possible output:

Monday: 3 lessons. Tuesday: twice as many as Monday = 6 lessons. Wednesday: 4 fewer than Tuesday = 2 lessons. Total watched: 3 + 6 + 2 = 11 lessons. Lessons left: 20 − 11 = 9.

The answer is 9. Example Explanation No example problems were provided. The single trigger sentence was enough to make the model write intermediate steps, and each step built on the previous one.

Where It Comes From Zero-shot CoT was introduced in the 2022 paper "Large Language Models are Zero-Shot Reasoners" by Kojima et al. The authors showed that adding "Let's think step by step" substantially improved the accuracy of large models on arithmetic and logical reasoning benchmarks, compared with standard zero-shot prompting, without writing any task-specific examples.

Why It Mattered Few-shot CoT (Lesson 41) requires carefully written reasoning examples for each type of task. Zero-shot CoT showed that a single, task-independent instruction can produce much of the benefit, which makes reasoning prompts far cheaper to create.

Why a Single Sentence Works As explained in Lesson 2, the model continues text in the way that best fits its context.

After A:, the most likely continuation is usually a short answer. After A: Let's think step by step., the most likely continuation is a sequence of reasoning steps, because during training the model has seen many texts where this phrase is followed by a worked solution. The sentence changes what kind of text is most likely to come next. Once the steps are written, they become part of the context and support the final answer (Lesson 39).

The Two-Stage Method The original research used two prompts:

Stage 1 — Reasoning extraction:

Q: [problem] A: Let's think step by step. The model writes its reasoning.

Stage 2 — Answer extraction:

Q: [problem] A: Let's think step by step. [reasoning from stage 1] Therefore, the answer (arabic numerals) is The model completes the sentence with only the answer.

Why Two Stages? Free-form reasoning can end in many ways: "So she has 9 left", "The answer is nine", or a final sentence with several numbers. The second stage forces a short, predictable answer that code can extract.

In Modern Applications With instruction-following chat models, you can often combine both stages into one request by asking for the reasoning and then a clearly marked final answer, using a tag or fixed label (Lesson 39). The two-stage method is still useful when:

The reasoning must stay free-form and you do not want formatting instructions to interfere with it. You need a very strict answer format, such as a single number or label. You are working with models that follow output formats less reliably. Implementing Two-Stage Zero-Shot CoT in Python Requirements: pip install openai and an OPENAI_API_KEY environment variable.

from openai import OpenAI

client = OpenAI() MODEL = "gpt-5.4" # Replace with a model available in your account

def ask(text: str) -> str: response = client.responses.create(model=MODEL, input=text) return response.output_text.strip()

def zero_shot_cot(question: str) -> tuple[str, str]: # Stage 1: generate reasoning reasoning_prompt = f"Q: {question}\nA: Let's think step by step." reasoning = ask(reasoning_prompt)

# Stage 2: extract a short final answer answer_prompt = ( f"{reasoning_prompt} {reasoning}\n" "Therefore, the final answer, as a number only, is" ) answer = ask(answer_prompt) return reasoning, answer

question = ( "A student watches 3 lessons on Monday, twice as many on Tuesday, and 4 fewer " "on Wednesday than on Tuesday. The course has 20 lessons. How many lessons are " "left after Wednesday?" )

reasoning, answer = zero_shot_cot(question) print("Reasoning:\n", reasoning) print("Answer:", answer) Code Explanation ask() Sends a single input string with no separate instructions. Zero-shot CoT relies only on the phrasing of the prompt itself.

Stage 1 The prompt ends with "Let's think step by step.", which invites a reasoning continuation.

Stage 2 The original question, the trigger phrase, and the generated reasoning are combined, followed by an unfinished sentence: "Therefore, the final answer, as a number only, is". The model completes it with the answer. "As a number only" plays the same role as "(arabic numerals)" in the original method: it constrains the answer format.

Why the reasoning is included in stage 2 Each request is independent (Lesson 2). The second request must contain the reasoning so that the answer is based on it.

Validating the Answer Chat models may still answer "9." or "9 lessons". Clean and validate the result:

import re

def parse_number(text: str) -> float | None: match = re.search(r"-?\d+(?:.\d+)?", text.replace(",", "")) return float(match.group()) if match else None This removes thousands separators and extracts the first number, returning None if none is found. For multi-number answers, a stricter format with tags is safer.

Variations of the Trigger Phrase "Let's think step by step" is not the only option, and it is not always the best.

Variation Notes Let's think step by step. The original zero-shot CoT trigger Let's work this out in a step by step way to be sure we have the right answer. Found by the Automatic Prompt Engineer research (Zhou et al., 2022) to perform better than the original phrase on the benchmarks they tested (Lesson 64) Let's first understand the problem and devise a plan to solve the problem. Then, let's carry out the plan and solve the problem step by step. From Plan-and-Solve Prompting (Wang et al., 2023), designed to reduce missing-step errors Think through this carefully before answering. A natural instruction suited to chat models Practical Guidance Differences between phrasings depend on the model and the task. A phrase that helped one model on one benchmark may not help another model on your task. For instruction-following models, a clear instruction that also specifies the output format is usually more useful than any particular "magic" phrase. Measure alternatives on your own test set before choosing one (Lesson 43). Plan-and-Solve: A Structured Zero-Shot Variant Free-form step-by-step reasoning sometimes skips steps or misunderstands the problem. Plan-and-Solve prompting addresses this by asking the model to plan first, then execute the plan.

Problem: A course fee is ₹8,000. Students who pay in full get a 10% discount. GST of 18% is applied to the discounted amount. A student pays in full. How much does the student pay?

First, understand the problem and list the values given. Then, make a plan: write the calculation steps needed, in order. Then, carry out the plan step by step, showing each calculation. Finally, give the answer on a line starting with "Answer:". Possible output:

Values: fee ₹8,000; full-payment discount 10%; GST 18% on discounted amount.

Plan:

1. Calculate the discount.
2. Subtract it to get the discounted fee.
3. Calculate GST on the discounted fee.
4. Add GST to the discounted fee.

Execution:

1. 10% of 8,000 = 800.
2. 8,000 − 800 = 7,200.
3. 18% of 7,200 = 1,296.
4. 7,200 + 1,296 = 8,496.

Answer: ₹8,496 Why Planning Helps Listing the values forces the model to read every part of the problem, reducing missed details. Writing the plan before calculating makes the order of operations explicit. Here, the plan correctly applies GST to the discounted amount, not to the original fee. Execution steps follow the plan, so errors are easy to locate. Domain-Specific Zero-Shot CoT For real applications, the most effective zero-shot CoT prompts describe the steps of the specific task, rather than using a generic phrase.

Generic Decide whether this ticket should be escalated. Let's think step by step. Domain-Specific Decide whether this ticket should be escalated.

Reason through these questions in order:

1. Is a paying customer currently unable to access something they paid for?
2. Is there a deadline within 48 hours mentioned in the ticket?
3. Has the customer already contacted support about this issue before?

Escalate if question 1 is yes and either question 2 or 3 is yes. Write your answer to each question, then "Decision: Escalate" or "Decision: Normal". Why Domain Steps Are Better The generic version leaves the model to decide what matters. The domain version tells it exactly which facts to check and how to combine them. This is still zero-shot (no examples), but the reasoning is guided by your business rules. This combines zero-shot CoT with conditional instructions (Lesson 35).

Zero-Shot CoT vs Few-Shot CoT Aspect Zero-Shot CoT Few-Shot CoT Effort to create Very low Requires writing correct reasoning examples Control over reasoning style Low (generic) to medium (with defined steps) High: examples show the exact reasoning pattern Token cost Lower prompt; reasoning output only Higher prompt (examples) plus reasoning output Risk of copying None Examples' reasoning patterns may be copied inappropriately Typical use First approach; general reasoning When a specific reasoning format or method is required A practical approach is to start with zero-shot CoT, add domain-specific steps if needed, and move to few-shot CoT only if the reasoning still goes wrong in ways examples can fix (Lesson 41).

Common Mistakes Mistake 1: Treating the Trigger Phrase as Magic What it is: Believing one specific phrase is required for reasoning.

Why it happens: The phrase is famous from research.

What it causes: Overlooking clearer, task-specific instructions that work better.

How to fix it: Use the phrase as a starting point; prefer domain-specific steps for real tasks.

Mistake 2: No Answer Format What it is: Asking for step-by-step reasoning with no defined final answer.

Why it happens: The focus is on getting reasoning.

What it causes: Answers buried in text, written in different ways each time.

How to fix it: Use the two-stage method or a clear answer marker.

Mistake 3: Using Zero-Shot CoT for Simple Tasks What it is: Adding "Let's think step by step" to tasks that need no reasoning.

Why it happens: It is easy to add to every prompt.

What it causes: Longer, slower, more expensive responses without benefit.

How to fix it: Reserve it for multi-step problems (Lesson 43).

Mistake 4: Adding CoT Triggers to Reasoning Models Without Testing What it is: Adding "think step by step" to models that already reason internally.

Why it happens: Habit from working with earlier models.

What it causes: Possibly no improvement, extra output, or, according to some provider guidance, worse results.

How to fix it: Follow the provider's guidance and test with and without the instruction.

Best Practices Start with zero-shot CoT for reasoning tasks. It is the cheapest way to add reasoning.

Prefer task-specific steps over generic phrases. Tell the model what to check and in what order.

Use planning for problems with many values or steps. List values, plan, execute, answer.

Always define how the final answer is marked. A tag, a label, or a second extraction stage.

Validate extracted answers. Clean and check them in code.

Compare phrasings on your own data. Research results do not automatically transfer to your model and task.

Try It Yourself Exercise 1: Compare Trigger Phrases Write 10 multi-step word problems with known answers. Run each with three variations: no trigger, "Let's think step by step.", and the Plan-and-Solve instruction. Use parse_number() to extract answers and compare accuracy.

Exercise 2: Generic vs Domain-Specific Write a prompt that decides whether a student should receive a fee waiver based on three rules of your choice. Create a generic zero-shot CoT version and a domain-specific version with explicit questions. Test both on eight cases, including two borderline ones.

Exercise 3: Single-Stage vs Two-Stage Modify the Python example to create a single-stage version that asks for reasoning followed by Answer: in one request. Run both versions on the same 10 problems and compare accuracy, total tokens used (from response.usage), and how often the answer could be extracted without errors.