Zero-Shot vs Few-Shot Prompting

The Core Difference Zero-Shot Few-Shot How the task is communicated Described in instructions Described and demonstrated with examples What the model relies on Its training plus your words Its training, your words, and the pattern in the examples Main strength Simple, cheap, flexible Precise format, boundaries, and style Main weakness Ambiguity in boundaries, formats, and style Copying, leakage, bias, and higher cost Lessons 16–25 covered zero-shot prompting, and Lessons 26–32 covered few-shot prompting. This lesson compares them directly: the same task solved both ways, a fair method for testing which is better, how to read the results, and how to combine the two approaches.

The Same Task, Two Ways Task Classify customer messages for a coding course platform into Doubt, Request, Complaint, or Feedback.

Zero-Shot Version Classify the student message as one of: Doubt, Request, Complaint, Feedback.

Definitions:

• Doubt: a question about course content or code
• Request: asking for something to be done or provided
• Complaint: dissatisfaction with a service, person, or experience
• Feedback: an opinion or suggestion without a request or complaint

Reply with the category only.

Message: Is it possible to get the session recording? Few-Shot Version Classify the student message as one of: Doubt, Request, Complaint, Feedback.

Definitions:

• Doubt: a question about course content or code
• Request: asking for something to be done or provided
• Complaint: dissatisfaction with a service, person, or experience
• Feedback: an opinion or suggestion without a request or complaint

Examples:

Message: Can you explain how Django signals work? Category: Doubt

Message: Can you share the slides from today's session? Category: Request

Message: The mentor didn't join the live session again. Category: Complaint

Message: Maybe add more SQL exercises in the next batch. Category: Feedback

Reply with the category only.

Message: Is it possible to get the session recording? Category: What Changes Aspect Zero-Shot Few-Shot Prompt length Shorter Several times longer Boundary between Doubt and Request Defined only in words Shown with two "Can you..." examples Output format Defined by "category only" Also shown by example outputs Effort to build Write definitions Write definitions and choose, verify, and balance examples For the question about the recording, both versions will often answer Request. The difference appears across many borderline inputs, which is why the comparison must be measured, not judged from one example.

Comparison Across Key Dimensions Dimension Zero-Shot Few-Shot Accuracy on common, clearly defined tasks Often already high Similar; may add little Accuracy on subtle boundaries Can be inconsistent Usually better with good boundary examples Custom or unusual formats Harder to get exactly right Strong Style and voice matching Generic Strong Risk of copying and leakage None from examples Present; must be managed Risk of label bias Lower Present if examples are unbalanced Token cost per request Lower Higher Time to first version Minutes Longer: examples must be collected and verified Maintenance Update instructions Update instructions and examples Adapting to new categories Edit definitions Edit definitions and add examples Needs labeled data No A small amount Reading the Table Neither approach is better overall. Zero-shot is the efficient default. Few-shot is a targeted improvement for format, boundary, and style problems, at the price of more tokens and more maintenance.

Comparing Them Fairly Intuition is unreliable here: a few-shot prompt often looks more thorough, but that does not mean it performs better. Compare the two with a controlled experiment.

Rules for a Fair Comparison Same test set. Both versions must be evaluated on identical inputs. Separate test data. Test inputs must never appear among the examples (Lesson 28). Same instructions. The few-shot version should be the zero-shot version plus examples, so the only difference is the examples. Same model and settings. Otherwise you are comparing models, not techniques. Include borderline cases. Differences usually appear there. Measure cost as well as accuracy. Count tokens per request. Repeat runs for borderline inputs. One run can be misleading because of sampling (Lesson 2). Comparison Script Requirements: pip install openai and an OPENAI_API_KEY environment variable.

from openai import OpenAI

client = OpenAI() MODEL = "gpt-5.4" # Replace with a model available in your account

LABELS = {"Doubt", "Request", "Complaint", "Feedback"}

BASE_INSTRUCTIONS = """Classify the student message as one of: Doubt, Request, Complaint, Feedback.

Definitions:

• Doubt: a question about course content or code
• Request: asking for something to be done or provided
• Complaint: dissatisfaction with a service, person, or experience
• Feedback: an opinion or suggestion without a request or complaint """

EXAMPLES = """ Examples:

Message: Can you explain how Django signals work? Category: Doubt

Message: Can you share the slides from today's session? Category: Request

Message: The mentor didn't join the live session again. Category: Complaint

Message: Maybe add more SQL exercises in the next batch. Category: Feedback """

ENDING = "\nReply with the category only."

PROMPTS = { "zero-shot": BASE_INSTRUCTIONS + ENDING, "few-shot": BASE_INSTRUCTIONS + EXAMPLES + ENDING, }

TEST_SET = [ ("Is it possible to get the session recording?", "Request"), ("Why does my loop print the last item twice?", "Doubt"), ("Great session, but please upload the notes.", "Request"), ("Nobody has fixed the broken quiz for a week.", "Complaint"), ("I liked the pace of today's class.", "Feedback"), ("Could you clarify what a foreign key is?", "Doubt"), ("The new dashboard is confusing, honestly.", "Feedback"), ("Third time asking: when will my refund arrive?", "Complaint"), ]

def evaluate(name: str, instructions: str) -> None: correct, input_tokens = 0, 0 for message, expected in TEST_SET: response = client.responses.create( model=MODEL, instructions=instructions, input=f"Message: {message}\nCategory:", ) predicted = response.output_text.strip() input_tokens += response.usage.input_tokens if predicted == expected: correct += 1 else: print(f" [{name}] expected {expected}, got {predicted!r}: {message}") average = input_tokens / len(TEST_SET) print(f"{name}: {correct}/{len(TEST_SET)} correct, " f"average input tokens per request: {average:.0f}")

for name, instructions in PROMPTS.items(): evaluate(name, instructions) Code Explanation BASE_INSTRUCTIONS, EXAMPLES, ENDING The prompts are assembled from shared parts. The zero-shot version is BASE + ENDING; the few-shot version inserts EXAMPLES between them. This guarantees that the only difference between the two prompts is the examples, which is what makes the comparison fair.

TEST_SET Labeled inputs that do not appear among the examples, including borderline cases such as "Great session, but please upload the notes." (feedback plus a request) and "The new dashboard is confusing, honestly." (opinion or complaint?).

response.usage.input_tokens The Responses API reports how many input tokens each request used. Summing these gives the real token cost of each version, including the examples.

Failure output Printing each misclassification shows where the versions differ, which is more useful than accuracy alone.

Notes on the Test Set Some labels in this test set reflect decisions that must be stated somewhere. "The new dashboard is confusing, honestly." is labeled Feedback here, but someone could argue for Complaint. If both versions disagree with your expected label on such cases, the problem is not zero-shot versus few-shot: it is a missing rule. Add the rule to the shared base instructions so both versions benefit.

Note: Eight test cases are enough to demonstrate the method, not to make a decision. For real decisions, use a larger test set of real inputs, typically dozens to hundreds, depending on how important the task is (Lessons 115 and 119).

Interpreting the Results Result Interpretation Action Both versions are accurate The task is well specified in words Keep zero-shot; it is cheaper Few-shot is clearly better on borderline cases Examples define boundaries that words did not Keep few-shot, or turn the lessons from examples into rules and retest zero-shot Zero-shot is better Examples are misleading, biased, or unrepresentative Fix the examples (Lesson 32) or keep zero-shot Both fail on the same inputs A rule or piece of information is missing Fix the shared instructions or context Both are inconsistent across runs Borderline cases are under-specified Add boundary rules, then reconsider examples Converting Example Lessons Into Rules Sometimes the examples work because they reveal a rule you had not written down, such as "Can you explain..." being a Doubt but "Can you share..." being a Request. Writing that rule explicitly ("Asking for an explanation is a Doubt; asking for a resource or action is a Request") may let the zero-shot version perform as well as the few-shot version, at lower cost. Always test after converting.

The Hybrid Approach Zero-shot and few-shot are not an either/or choice. Many effective prompts combine them.

Pattern 1: Zero-Shot Instructions Plus Targeted Examples Write a complete zero-shot prompt, then add only one or two examples for the specific boundary that remains difficult.

[Complete definitions and rules]

Example of a difficult case: Message: Great session, but please upload the notes. Category: Request (a request takes priority over feedback) This gives most of the benefit of few-shot prompting at a fraction of the token cost.

Note: If you show an explanation in parentheses like this, make clear that the real output must contain only the label, or the model may copy the explanation (Lesson 32, Problem 6).

Pattern 2: Zero-Shot With a Format Example Use instructions for the task and a single example only to show the exact output format (Lesson 27, one-shot for fixed templates).

Pattern 3: Zero-Shot by Default, Dynamic Examples for Hard Inputs Use a zero-shot prompt for most inputs. When a cheap check identifies an input as difficult, for example because the zero-shot model returned an invalid label or a "NeedsReview" category, retry with a few-shot prompt that includes the most similar examples (Lesson 28). Most requests stay cheap, and only hard cases pay for examples.

A Typical Path in a Real Project

1. Write a clear zero-shot prompt with definitions and rules ↓
2. Build a test set from real inputs, including borderline cases ↓
3. Evaluate zero-shot ↓
4. Fix shared problems (missing rules, missing context) ↓
5. Still failing on boundaries, formats, or style? ├── No → Deploy zero-shot └── Yes → Add targeted examples (hybrid or few-shot) ↓
   1. Compare on the same test set, including cost ↓
   2. Deploy the version that meets the criteria at the lowest cost Common Mistakes Mistake 1: Comparing Different Instructions What it is: The few-shot version also has better instructions than the zero-shot version.

Why it happens: Both prompts are written separately.

What it causes: You cannot tell whether improvements came from the examples or the instructions.

How to fix it: Build both versions from the same base instructions and add only the examples.

Mistake 2: Judging From a Handful of Inputs What it is: Choosing an approach after trying three or four messages.

Why it happens: Quick tests feel convincing.

What it causes: Decisions based on chance.

How to fix it: Use a test set large enough to include many borderline cases.

Mistake 3: Ignoring Cost What it is: Choosing few-shot for a small accuracy gain on a high-volume task.

Why it happens: Accuracy is measured; cost is not.

What it causes: Large ongoing costs for little benefit.

How to fix it: Measure input tokens per request and estimate monthly cost (Lesson 23).

Mistake 4: Never Revisiting the Choice What it is: Keeping a few-shot prompt after the model is upgraded, or a zero-shot prompt after requirements become more subtle.

Why it happens: The original decision is assumed to be permanent.

What it causes: Paying for examples that are no longer needed, or tolerating errors that examples would fix.

How to fix it: Re-run the comparison when the model, the requirements, or the input distribution changes.

Best Practices Start with zero-shot and measure. It is the baseline every other approach must beat.

Make the comparison controlled. Same base instructions, same model, same test set.

Focus on borderline cases. That is where the two approaches differ.

Measure cost alongside accuracy. Use reported token usage.

Look for missing rules first. Failures shared by both versions are not solved by examples.

Use hybrid prompts. Add only the examples that fix specific remaining problems.

Re-evaluate after changes. Model updates and new requirements can change the answer.

Try It Yourself Exercise 1: Run the Comparison Run the comparison script from this lesson with at least 20 test messages of your own, including 8 borderline ones. Record accuracy and average input tokens for both versions, and decide which you would deploy.

Exercise 2: Convert Examples Into Rules Look at the inputs where few-shot beat zero-shot. For each, write a rule that captures what the examples taught. Add the rules to the shared base instructions and run the comparison again. Check whether zero-shot now matches few-shot.

Exercise 3: Build a Hybrid Prompt Starting from the zero-shot version, add only one example targeting the most common zero-shot error. Compare three versions (zero-shot, hybrid, full few-shot) on accuracy and token cost.