Zero-Shot Summarization

What Is Summarization? Summarization is the task of producing a shorter version of a text that keeps its most important information.

Zero-shot summarization means asking a model to summarize by describing the summary you want, without showing example summaries.

Example Summarize the announcement below in 2 sentences for students.

From 1 December, all GOCOURSE live classes will move from Zoom to our own classroom platform. Students will receive login details by email on 25 November. Recorded sessions from previous batches will remain available in the course library. Students facing login issues should contact support@gocourse.example before 30 November. Possible output:

From 1 December, live classes will move from Zoom to GOCOURSE's own classroom platform, with login details emailed on 25 November. Past recordings stay in the course library, and login problems should be reported to support before 30 November. Example Explanation The summary keeps the change (Zoom to own platform), the key dates (1 December, 25 November, 30 November), and the actions students need to take. It omits the full email address, which is less important for a 2-sentence summary but would matter in a different summary. What counts as "important" depends on the purpose. The audience ("for students") pushes the model to focus on what students must know and do. Why Summarization Needs Careful Prompting "Summarize this" seems simple, but it leaves the most important decisions to the model:

Decision Why It Matters How long? A summary can be one line or one page For whom? A manager, a customer, and a developer need different details For what purpose? Deciding, acting, studying, or just staying informed What to focus on? Decisions, risks, numbers, action items, or the overall idea In what format? Paragraph, bullet points, table, or headline A model that does not know these answers produces a "general" summary that is reasonable but often not useful for your specific situation.

Extractive vs Abstractive Summaries There are two basic ways to summarize.

Type How It Works Advantages Risks Extractive Selects important sentences or phrases from the original Wording stays faithful to the source Can be choppy; may include unnecessary details within selected sentences Abstractive Writes new sentences that express the main ideas Fluent, compact, adaptable to audience May change meaning or add unsupported information LLMs produce abstractive summaries by default. You can request extractive behavior when exact wording matters:

Select the 3 sentences from the text that best capture its main points. Copy them exactly as written, in their original order. When to use extractive: legal text, policies, quotes, or any situation where rewording could change the meaning.

When to use abstractive: news, meeting notes, articles, and any situation where readability matters more than exact wording.

Controlling the Summary

1. Length Use units the model can follow precisely (Lesson 8):

Instruction Precision in one sentence High in 3 bullet points High in 2 short paragraphs High in under 100 words Approximate about 20% of the original length Low; models are not good at estimating ratios

1. Audience Summarize the release notes for customers who are not technical. Summarize the release notes for the developers who will maintain this code. The first summary focuses on visible changes and benefits; the second on technical changes, breaking changes, and migration steps.

2. Focus (Query-Focused Summarization) Sometimes you only need the parts of a text related to one topic.

Summarize only what the meeting notes say about the mobile app release. Ignore other topics. If the notes say nothing about the mobile app, reply "No discussion of the mobile app." This is called query-focused summarization. It is often more useful than a general summary, because long documents usually cover many topics, and you rarely need all of them.

1. Format Format Instruction Good For TL;DR One sentence capturing the main point. Previews, notifications Bullet points 3 to 5 bullets, one fact per bullet. Quick scanning Executive summary A 3-sentence paragraph: situation, key finding, recommended action. Decision makers Action items List each action as: Owner — Task — Due date. Meeting notes Structured Sections: Problem, Changes, Impact, Next Steps. Reports, incidents Faithfulness: The Most Important Quality A summary is faithful when everything it says is supported by the original text. Unfaithful summaries are dangerous because readers usually trust a summary instead of checking the source.

Common Faithfulness Problems Problem Example Added information The source says "classes move to a new platform"; the summary adds "for better video quality" Changed certainty "We are considering a price change" becomes "Prices will change" Changed numbers "about 300 students" becomes "350 students" Merged facts Two separate deadlines become one Wrong attribution A suggestion by one person is attributed to another Prompt Instructions That Improve Faithfulness Rules:

• Include only information stated in the text. Do not add reasons, opinions, or background that the text does not contain.
• Keep all numbers, dates, and names exactly as written.
• Keep the level of certainty: if the text says something is planned or possible, do not present it as decided. Why These Rules Help Abstractive summarization rewrites the content, and while rewriting, the model may fill in plausible connections, such as a reason for a change. The rules explicitly forbid additions and require preserving the exact details where errors are most harmful: numbers, dates, names, and certainty.

Summarization in Code With a Number Check The following example summarizes text with configurable focus and length, and then checks whether every number in the summary appears in the source. Numbers are one of the easiest faithfulness errors to detect automatically.

Requirements: pip install openai and an OPENAI_API_KEY environment variable.

import re

from openai import OpenAI

client = OpenAI() MODEL = "gpt-5.4" # Replace with a model available in your account

NUMBER_PATTERN = re.compile(r"\d+(?:[.,]\d+)*%?")

def summarize(text: str, audience: str, bullets: int = 3, focus: str = "") -> str: focus_line = f"Focus only on: {focus}.\n" if focus else "" instructions = ( f"Summarize the text in for {audience}.\n" f"{focus_line}" f"Write exactly {bullets} bullet points, one main fact per bullet.\n" "Include only information stated in the text. Do not add reasons or opinions.\n" "Keep all numbers, dates, and names exactly as written.\n" "Return only the bullet points." ) response = client.responses.create( model=MODEL, instructions=instructions, input=f"\n{text}\n", ) return response.output_text.strip()

def unsupported_numbers(source: str, summary: str) -> list[str]: """Return numbers that appear in the summary but not in the source.""" source_numbers = set(NUMBER_PATTERN.findall(source)) return [n for n in NUMBER_PATTERN.findall(summary) if n not in source_numbers]

report = """In October, 1,240 students enrolled across all courses, up from 980 in September. The Django course had the highest completion rate at 72%. Support tickets rose to 310, mainly due to payment failures during a 2-day gateway outage. The team plans to add a backup payment provider in November."""

summary = summarize(report, audience="the company's founders", bullets=3) print(summary)

problems = unsupported_numbers(report, summary) if problems: print("Warning: numbers not found in source:", problems) Possible output:

• Enrollments grew to 1,240 in October from 980 in September.
• The Django course had the highest completion rate at 72%.
• Support tickets rose to 310 due to payment failures during a 2-day outage; a backup payment provider is planned for November. Code Explanation summarize() parameters

audience adapts the summary to its readers. bullets controls length using a count the model can follow exactly. focus enables query-focused summarization. When it is empty, no focus line is added, so the prompt stays clean. Faithfulness rules in instructions The rules forbid added reasons and opinions and require exact numbers, dates, and names.

NUMBER_PATTERN

\d+ matches digits. (?:[.,]\d+)* allows thousands separators and decimals, such as 1,240 or 3.5. %? includes an optional percent sign, so 72% is matched as one value. unsupported_numbers() Finds every number in the summary and reports those that do not appear in the source. If the model wrote 1,300 instead of 1,240, or calculated a value such as "26% growth" that is not stated in the source, the check flags it for review.

Limitations of the Check This check only verifies numbers. It cannot detect changed meaning, wrong attribution, or added reasons. Numbers written as words ("two days" vs "2-day") are also not compared. It is a cheap safety net, not a complete faithfulness test. A second, model-based check is shown next.

Checking a Summary With a Second Prompt You can ask a model to compare a summary against its source. This applies the "separate creating from judging" principle (Lesson 10, Tip 6).

Compare the summary with the source text.

For each sentence or bullet in the summary, decide:

• SUPPORTED: fully supported by the source
• UNSUPPORTED: contains information not in the source, or changes its meaning

Return one line per bullet: : —

[original text]

[summary to check] Why It Helps The checking prompt has a single job: verify. It does not need to write fluently or decide what is important, so it can focus on comparing claims against the source. Unsupported bullets can then be revised or flagged for human review.

Summarizing Different Kinds of Content Content What the Prompt Should Emphasize Meeting notes Decisions, action items with owners and dates, open questions Customer reviews Common themes, frequency, representative complaints and praise Technical documentation Purpose, requirements, key steps, warnings Research or long articles Main question, method, findings, limitations Email threads Current status, latest decision, what is required from the reader Incident reports What happened, impact, cause, fix, prevention Example: Email Thread Summary Summarize the email thread in for someone joining the conversation now.

Use this structure: Current status: Decisions made: Waiting on:

Base the summary on the latest messages when earlier messages were corrected or overridden. The final rule matters: in email threads, early messages often contain plans that were later changed. Without this rule, a summary may report an outdated decision.

Long Documents When a document is too long to summarize well in one prompt, use the map-reduce approach from Lesson 15: summarize sections separately, then combine the section summaries.

Common Mistakes Mistake 1: "Summarize This" With No Purpose What it is: A summary request without audience, length, or focus.

Why it happens: Summarization seems self-explanatory.

What it causes: A generic summary that may omit exactly what you needed.

How to fix it: State who reads it, why, how long it should be, and what to focus on.

Mistake 2: Trusting Summaries Without Checking Facts What it is: Using a summary in place of the source for decisions.

Why it happens: Summaries read smoothly and confidently.

What it causes: Added reasons, changed numbers, or changed certainty mislead the reader.

How to fix it: Add faithfulness rules, check numbers in code, and verify important points against the source.

Mistake 3: Losing Certainty Levels What it is: "Planned", "proposed", or "being considered" becomes "will" or "has".

Why it happens: Shorter phrasing tends to drop qualifiers.

What it causes: Readers believe tentative ideas are final decisions.

How to fix it: Instruct the model to preserve certainty and attribution.

Mistake 4: Ratio-Based Length Limits What it is: Asking for "10% of the original length".

Why it happens: It seems like a precise instruction.

What it causes: Unpredictable lengths, because models estimate ratios poorly.

How to fix it: Use counts of sentences, bullets, or words.

Mistake 5: Summarizing Outdated Parts of Threads or Documents What it is: Summarizing an entire history equally, including superseded information.

Why it happens: The model treats all text as equally current.

What it causes: Summaries that report decisions that were later reversed.

How to fix it: Tell the model to prioritize the latest information and note changes.

Best Practices Define audience, purpose, length, focus, and format. These five decisions determine whether a summary is useful.

Use counts for length. Sentences, bullets, and paragraphs are followed more precisely than ratios.

Prefer focused summaries over general ones. Ask for the information you will actually use.

Protect faithfulness explicitly. Forbid additions; preserve numbers, names, dates, and certainty.

Use extractive summaries when wording matters. Legal and policy text should not be paraphrased loosely.

Verify automatically where possible. Check numbers in code and use a second prompt to check claims.

Choose structures that match the content. Action items for meetings, status and decisions for threads, impact and cause for incidents.

Try It Yourself Exercise 1: Same Text, Different Audiences Take a product update announcement and summarize it three times: for customers, for the support team, and for developers. Compare which details each summary keeps and drops.

Exercise 2: Test Faithfulness Write a short text that contains tentative language ("we are considering", "might", "proposed"), three numbers, and two named people. Summarize it with and without the faithfulness rules from this lesson. Check whether certainty, numbers, and attributions are preserved in each version.

Exercise 3: Extend the Number Check Improve unsupported_numbers() so it also detects numbers that appear in the source but are missing from a summary when you required all numbers to be kept. Test it with a summary that drops one figure.