Build a Text Summarizer

The Goal

A summarizer takes a document of any length and returns a short, accurate summary. Short documents need one request. Long documents do not fit comfortably in one request, and a single summary of a very long text loses detail from the middle. The project therefore has two paths, chosen by the length of the input: a direct summary for short text, and a map-reduce summary for long text, in which each part is summarized and the partial summaries are combined.

This project reuses the chunking and map-reduce ideas from Lesson 118, and it packages them as a function with a clear contract. It also makes the number of model calls visible, because the cost of a summary depends directly on how many calls it takes.

What You Will Build

A summarize function that:

1. returns a summary of any text,
2. makes one call for text that fits within a single part, and one call per part plus a combining call for longer text,
3. reports how many model calls it made, so the cost of each summary is known,
4. runs with a stand-in client for testing, and with the live client in production.

Project Structure

summarizer/
├── summarizer.py    # chunking, the two paths, and the summarize function
└── demo.py          # a stand-in client, and one short and one long document

Step 1: Splitting the Document

Splitting is the first decision, and it must respect paragraph boundaries where possible. The chunker packs whole paragraphs into parts no larger than the limit, and it cuts a paragraph only when the paragraph alone exceeds the limit:

"""summarizer.py: summarize a document of any length, with one or several model calls."""

from dataclasses import dataclass

MAX_PART = 6000


def chunk(text: str, max_chars: int = MAX_PART) -> list[str]:
    parts: list[str] = []
    current = ""
    for paragraph in (p.strip() for p in text.split("\n\n")):
        if not paragraph:
            continue
        for piece in (paragraph[i:i + max_chars] for i in range(0, len(paragraph), max_chars)):
            candidate = f"{current}\n\n{piece}" if current else piece
            if len(candidate) <= max_chars:
                current = candidate
            else:
                parts.append(current)
                current = piece
    if current:
        parts.append(current)
    return parts

The slicing for oversized paragraphs is a fallback that can split mid-sentence. It is kept because a single long paragraph must still be summarizable, and the next step gives each part a title so Claude knows where it sits.

Step 2: One Call to the Model

Each request sends one instruction and one piece of text, wrapped in tags so the text is clearly data (Lesson 37). A small helper extracts the answer from the text blocks:

INSTRUCTIONS = {
    "short": "Summarize the document in three bullet points. Keep every number and name.",
    "part": "This is part {i} of {n} of a longer document. Summarize this part in at most "
            "120 words, listing every decision, name, and number.",
    "reduce": "Combine these partial summaries, in order, into one summary of at most 200 words. "
              "Preserve every number and name, and say so if a part seems to be missing.",
}


@dataclass
class Summary:
    text: str
    model_calls: int


def ask(client, instruction: str, text: str) -> str:
    response = client.messages.create(
        model="claude-sonnet-5-5",
        max_tokens=400,
        messages=[{"role": "user", "content": f"{instruction}\n\n<document>\n{text}\n</document>"}],
    )
    return "".join(block.text for block in response.content if block.type == "text").strip()

Step 3: Choosing the Path

The function measures the input and chooses the path. A document within one part is summarized directly. A longer document is split, each part is summarized in turn, and the partial summaries are combined. The number of model calls is counted as the work happens:

def summarize(client, text: str) -> Summary:
    parts = chunk(text)
    if len(parts) == 1:
        return Summary(ask(client, INSTRUCTIONS["short"], parts[0]), model_calls=1)

    partials = [
        ask(client, INSTRUCTIONS["part"].format(i=index, n=len(parts)), part)
        for index, part in enumerate(parts, start=1)
    ]
    combined = "\n\n".join(f"Part {index}: {text}" for index, text in enumerate(partials, start=1))
    final = ask(client, INSTRUCTIONS["reduce"], combined)
    return Summary(final, model_calls=len(parts) + 1)

How It Works

1. The document is split once, and the number of parts decides the path. A document that fits in one part skips the map-reduce machinery entirely.
2. Each part is summarized with its position, so the model knows the summary is one piece of a larger whole.
3. The partial summaries are combined in their original order, which keeps the sequence of the document intact (Lesson 118 explained why order matters).
4. The call count is returned with the summary. A summary of a long document costs several calls, and the count is the figure a cost report needs (Lesson 186).

Step 4: A Stand-In and a Demonstration

The demonstration needs a client that returns a predictable answer. The stand-in numbers its replies, so the output shows exactly which calls were made:

"""demo.py: summarize a short and a long document with a stand-in client."""

from types import SimpleNamespace

from summarizer import summarize


class NumberedClient:
    def __init__(self):
        self.calls = 0
        self.messages = self

    def create(self, **request):
        self.calls += 1
        return SimpleNamespace(content=[SimpleNamespace(type="text", text=f"summary #{self.calls}")])


if __name__ == "__main__":
    short_document = "The course has six modules. Each module ends with a short quiz."
    long_document = "\n\n".join(["a" * 4000, "b" * 4000, "c" * 4000])

    client = NumberedClient()
    result = summarize(client, short_document)
    print(f"short: {result.text} ({result.model_calls} model call)")

    client = NumberedClient()
    result = summarize(client, long_document)
    print(f"long: {result.text} ({result.model_calls} model calls, 3 parts)")

Output:

short: summary #1 (1 model call)
long: summary #4 (4 model calls, 3 parts)

How It Works

1. The short document is one part, so the function makes one call and reports one model call.
2. The long document has three parts of four thousand characters each. Two parts cannot share a six-thousand-character limit, so each stands alone, giving three parts.
3. Three part summaries and one combining call make four calls, which the output reports as four model calls, and the final text is the fourth numbered reply.
4. The stand-in's counter shows the call sequence, which is how a test confirms the number of requests without any network access.

Going Live

For production, the stand-in is replaced by the SDK client, and the function is called in the same way:

from anthropic import Anthropic

result = summarize(Anthropic(), document_text)
print(result.text)

The live path uses the same instructions and the same chunking, so behavior is predictable. Where a team wants faster long summaries, the part calls can be made concurrently, using the asynchronous patterns from Lessons 74 and 75, while the combining call still waits for all of them.

Checking the Output

A summary is a model output, and its quality is judged with the methods from Lesson 110. A useful check for this project compares the numbers and names in each summary with those in the source, since omitted or altered figures are the most serious summary errors. A summary that drops a number should be flagged for review, not accepted.