Breaking Large Tasks into Smaller Prompting Tasks
What Is Task Decomposition? Task decomposition is the practice of splitting a large, complex task into smaller subtasks, each handled by its own focused prompt.
In simple terms: instead of asking the model to do everything at once, you ask it to do one well-defined part at a time, and then combine the results.
Example One large prompt:
Read these 40 customer reviews, find the main complaints, group them into themes, count how often each theme appears, write a report for management with recommendations, and also draft a reply to each negative review. Decomposed into subtasks:
- For each review: extract the complaints, if any.
- Group all extracted complaints into themes.
- Count the complaints per theme (in code).
- Write a management report from the themes and counts.
- For each negative review: draft a reply. Each subtask is small enough to describe precisely, test independently, and check before moving on.
Why Large Tasks Fail in a Single Prompt
Too Many Requirements Compete for Attention A prompt with many different tasks has many instructions, constraints, and formats. The model may handle some parts thoroughly and others superficially, or skip some entirely (Lesson 11, mistake 1.2).
Errors Compound Invisibly In a single response, an early mistake (for example, misreading a review) affects everything after it: the themes, the counts, the report. Because all steps happen inside one response, you cannot see or correct the intermediate result.
Output Length Limits A task that requires a long output, such as 40 individual replies plus a report, may exceed the model's output token limit and be cut off (Lesson 2).
Different Subtasks Need Different Settings Extracting complaints needs strict, consistent output. Writing a management report needs well-structured prose. Drafting customer replies needs a warm tone. A single prompt must use one set of instructions for all of them.
Debugging Is Difficult When a combined output is wrong, it is hard to tell which part of the process failed. With separate steps, each intermediate output can be inspected.
Benefits of Breaking Tasks Down Benefit Why It Happens Higher accuracy Each prompt focuses on one job with clear instructions Visible intermediate results You can inspect and correct each step's output Easier testing Each step can be tested with its own inputs and expected outputs Reusability Steps such as "extract complaints" can be reused in other workflows Flexibility Each step can use the most suitable instructions, format, or even model Mixing models and code Exact operations (counting, sorting) can be done in code between steps How to Decompose a Task Step 1: Write the End Goal State what the final result must be.
Goal: A one-page management report on the main problems customers mention in reviews, with counts and recommended actions. Step 2: List What Must Exist Before the Goal Can Be Produced Work backwards. The report requires themes and counts. Themes require extracted complaints. Extracted complaints require reading each review.
Report ← themes + counts ← grouped complaints ← extracted complaints ← reviews Step 3: Identify the Type of Each Step Step Type Done By Extract complaints from each review Language understanding Model Group complaints into themes Language understanding Model Count complaints per theme Exact computation Code Write the report Language generation Model Step 4: Define the Input and Output of Each Step Each step's output becomes the next step's input, so outputs should be structured and predictable.
Step Input Output Extract One review JSON list of complaint phrases (possibly empty) Group All complaint phrases JSON mapping theme → list of complaints Count Theme mapping Theme → number (Python) Report Themes and counts Markdown report Step 5: Decide What to Check Between Steps For example: verify the extraction returned valid JSON for every review, and check that grouping did not drop complaints.
Decomposition Patterns Pattern 1: Sequential (Pipeline) Each step uses the output of the previous step.
Raw text → Clean → Extract → Analyze → Write Example: Writing a technical blog post
Generate an outline for the topic. Review and edit the outline (you or a model). Write each section from the approved outline. Check the full draft for technical errors. When to use: When later steps depend on decisions made earlier.
Pattern 2: Parallel (Map) The same prompt is applied independently to many items.
Review 1 → Extract Review 2 → Extract Review 3 → Extract ... When to use: When items are independent, such as classifying tickets, summarizing individual documents, or translating separate strings.
Why it helps: Each item gets full attention, and one problematic item does not affect the others.
Pattern 3: Map-Reduce (Divide and Combine) Items or document sections are processed separately (map), then the partial results are combined (reduce).
Section 1 → Summary 1 ┐ Section 2 → Summary 2 ├→ Combined summary Section 3 → Summary 3 ┘ When to use: Documents too long for one prompt, or analyses across many items.
Pattern 4: Routing (Conditional) A first prompt decides which kind of input it is, and the input is then sent to a specialized prompt.
Ticket → Classify → Billing? → Billing reply prompt → Technical? → Troubleshooting prompt → Other? → General reply prompt When to use: When different input types need different handling. Each specialized prompt stays simple because it handles only one type.
Example: Summarizing a Long Document With Map-Reduce The following Python example summarizes a document that may be too long to summarize well in one prompt.
Requirements: pip install openai and an OPENAI_API_KEY environment variable.
from openai import OpenAI
client = OpenAI() MODEL = "gpt-5.4" # Replace with a model available in your account
SECTION_INSTRUCTIONS = """Summarize the document section in
.- Write 3 to 5 bullet points.
- Keep specific facts: numbers, dates, names of features or policies.
- Do not add information that is not in the section."""
FINAL_INSTRUCTIONS = """You are given summaries of consecutive sections of one document in . Write a single summary of the whole document for a busy manager:
- One short paragraph describing the overall purpose
- Then up to 7 bullet points with the most important facts and decisions
- Merge duplicate points and keep the original order of topics
- Do not add information that is not in the summaries"""
def split_into_chunks(text: str, max_words: int = 800) -> list[str]: """Split text into chunks of whole paragraphs, each up to about max_words.""" chunks, current, current_words = [], [], 0 for paragraph in text.split("\n\n"): words = len(paragraph.split()) if current and current_words + words > max_words: chunks.append("\n\n".join(current)) current, current_words = [], 0 current.append(paragraph) current_words += words if current: chunks.append("\n\n".join(current)) return chunks
def ask(instructions: str, input_text: str) -> str: response = client.responses.create( model=MODEL, instructions=instructions, input=input_text, ) return response.output_text
def summarize_long_document(text: str) -> str: section_summaries = [] for number, chunk in enumerate(split_into_chunks(text), start=1): summary = ask(SECTION_INSTRUCTIONS, f"
\n{chunk}\n") section_summaries.append(f"Section {number}:\n{summary}")combined = "\n\n".join(section_summaries) return ask(FINAL_INSTRUCTIONS, f"\n{combined}\n")
with open("policy_document.txt", encoding="utf-8") as file: print(summarize_long_document(file.read())) Code Explanation split_into_chunks
Splits the document at blank lines ("\n\n"), which usually separate paragraphs. Adds paragraphs to the current chunk until adding the next one would exceed max_words, then starts a new chunk. Splitting at paragraph boundaries keeps related sentences together. Splitting at a fixed number of words could cut a sentence or a table in half and lose its meaning. A single paragraph longer than max_words becomes its own chunk. For documents with very long paragraphs, you would add further splitting by sentences. SECTION_INSTRUCTIONS (the map step)
Each chunk is summarized independently. "Keep specific facts" is important: if each section summary is vague, the final summary cannot recover the details. "Do not add information" reduces the chance of invented content entering the pipeline, where it would be passed on as if it were fact. FINAL_INSTRUCTIONS (the reduce step)
Combines section summaries into one result for a specific audience. "Merge duplicate points" is needed because related information may appear in several sections. "Keep the original order of topics" preserves the document's structure. ask A small helper that sends instructions and input in one call. Each step is a separate, independent request, as explained in Lesson 2.
Section labels Section {number}: tells the reduce step that the summaries are ordered parts of the same document.
What Happens If You Change Parts of This Design Change Effect Chunks much larger Fewer calls, but each section summary may lose details Chunks much smaller More calls, higher cost, and summaries of fragments with too little context Remove "keep specific facts" Final summary becomes vague; details disappear in the map step Skip the reduce step and just join section summaries Long, repetitive result without an overall picture Decomposition in Interactive Chat You do not need code to decompose tasks. In a chat, you can guide the model through steps across several messages.
Message 1:
I want to write a blog post titled "Python Virtual Environments for Beginners". Create an outline with 5 to 7 section headings and one sentence describing each section. Do not write the article yet. Message 2 (after reviewing the outline):
Good. Replace section 6 with a section on requirements.txt. Now write only section 1, in about 200 words, for complete beginners. Message 3:
Now write section 2 in the same style. Why This Works Better Than One Request You approve the structure before any writing happens, so you do not have to rewrite a full article because of a bad outline. Each section receives focused attention. You can correct problems early, before they spread to later sections. Passing Information Between Steps Because each request is independent, a later step only knows what you include in its prompt.
Include What the Next Step Needs If step 3 must follow decisions made in step 1, include those decisions in step 3's prompt. For example, when writing section 4 of an article in a separate request, include the outline and possibly a short summary of the earlier sections, so the style and content remain consistent.
Use Structured Outputs Between Steps Plain prose is hard for the next step (or your code) to process reliably. Structured outputs such as JSON or labeled lines make it clear what each piece of information is.
Validate Before Passing Results On A wrong intermediate result becomes the input of every following step. Check intermediate outputs in code:
Is the JSON valid? Does every input item have a result? Are values within the allowed set? The details of designing, connecting, and debugging multi-step workflows are covered in the Prompt Chaining lessons (Lessons 61–63 and 70–74).
When Not to Break a Task Down Decomposition has costs. It is not always the right choice.
Situation Why a Single Prompt May Be Better The task is simple Extra steps add cost and latency with no improvement Steps depend heavily on shared context Splitting may lose connections between parts Low latency is required Each step adds another request and waiting time A single prompt already meets the success criteria No problem to solve Rule of thumb: Start with one prompt. Decompose when you observe a specific problem: skipped parts, compounding errors, cut-off output, or results that are hard to debug.
Common Mistakes Mistake 1: Splitting Without Defining Step Outputs What it is: Breaking a task into steps but letting each step return free-form text.
Why it happens: Defining formats for every step seems like extra work.
What it causes: Later steps receive inconsistent input, and code cannot process intermediate results.
How to fix it: Define a clear output format for each step, especially when another step or code uses it.
Mistake 2: Losing Context Between Steps What it is: A later step lacks information it needs from earlier steps or from the original request.
Why it happens: Forgetting that each request is independent.
What it causes: Inconsistent style, contradictions, or ignored decisions.
How to fix it: Include the necessary context in each step's prompt: the original goal, relevant decisions, and required facts.
Mistake 3: No Checks Between Steps What it is: Passing each step's output directly to the next without validation.
Why it happens: The pipeline works on test examples.
What it causes: An error in one step silently spreads through all following steps.
How to fix it: Validate structure and completeness after each step and handle failures, such as by retrying.
Mistake 4: Using the Model for Steps That Code Should Do What it is: Asking the model to count, sort, deduplicate exactly, or calculate totals.
Why it happens: The whole workflow is designed as a sequence of prompts.
What it causes: Small numerical errors in an otherwise good pipeline.
How to fix it: Use code for exact operations between model steps.
Mistake 5: Over-Decomposing What it is: Splitting a simple task into many tiny steps.
Why it happens: Applying decomposition as a rule rather than a solution.
What it causes: Higher cost, slower results, and more places for errors, without better quality.
How to fix it: Split only where a step has a distinct purpose, format, or failure mode.
Best Practices Work backwards from the goal. Identify what must exist before the final result can be produced.
Give each step one clear job. A step that does one thing is easier to prompt, test, and debug.
Define inputs and outputs for every step. Structured outputs make steps easy to connect and validate.
Use code for exact operations. Counting, sorting, filtering, and arithmetic belong in code.
Validate intermediate results. Catch errors before they spread.
Choose the right pattern. Sequential for dependent steps, parallel for independent items, map-reduce for long content, routing for different input types.
Pass only the context each step needs. Enough to keep steps consistent, but not so much that the step loses focus.
Start with one prompt and decompose when needed. Decomposition should solve an observed problem.
Try It Yourself Exercise 1: Decompose a Task on Paper Break this request into steps and fill in a table with each step's type (model or code), input, and output:
From our last 100 support tickets, find the 5 most common problems, estimate which ones take the most time to resolve, and write an FAQ entry for each. Exercise 2: Outline-First Writing In a chat tool, write an article in three ways:
One prompt asking for the full article. An outline first, then the full article from the approved outline. An outline first, then each section in a separate message. Compare structure, depth, and how much rewriting each approach required.
Exercise 3: Build a Map-Reduce Summarizer Run the Python example in this lesson on a long document of your own, such as a policy, a long article, or documentation. Then:
Print each section summary and check whether important facts were kept. Change max_words to 300 and 1,500 and compare the final summaries. Add a check that warns you if any section summary is empty.