Combining Prompting Techniques for Complex Tasks

Why Combine Techniques? Each technique in this tutorial solves a specific problem:

Clear instructions and context solve misunderstanding (Lessons 5–6). Few-shot examples solve format, boundary, and style problems (Lesson 26). Chain-of-thought solves multi-step reasoning errors (Lesson 39). Self-consistency solves random reasoning errors (Lesson 45). Retrieval and context solve missing knowledge (Lesson 19). Code execution solves calculation errors (Lesson 58). Tools solve the need for live data and actions (Lesson 78). Chains solve complexity, by giving each step one job (Lesson 61). Validation solves undetected errors between steps (Lesson 73). Real tasks usually contain several of these problems at once. A student asking, "How much will I pay for the Django REST APIs course if I pay in full, and can I still get a refund if I change my mind next week?" needs facts from a policy, an exact calculation, a judgment about eligibility, and a clear, friendly reply.

No single technique handles all of that well. Combining techniques means assigning each part of the task to the technique that solves that part's problem.

The Core Principle: Match Techniques to Problems Do not ask "Which advanced technique should I use?" Ask "What can go wrong in this part of the task, and which technique prevents that?"

Problem-to-Technique Map If This Part of the Task... It Risks... Use Depends on facts the model may not know Invented or outdated facts Context or retrieval (Lessons 6, 19, 101) Involves exact numbers or dates Calculation errors Code / PAL (Lesson 58) Requires combining several rules or steps Reasoning errors Chain-of-thought (Lesson 39) Has borderline cases Inconsistent decisions Rules plus boundary examples (Lessons 18, 30) Gives different answers on repeated runs Random errors Self-consistency, possibly adaptive (Lessons 45–46) Needs live data or actions Stale data, unsafe actions Tool calling with validation and confirmation (Lesson 78) Has several distinct jobs Overloaded prompts Prompt chain (Lessons 61–63) Produces data used by later steps Errors spreading Validation gates (Lesson 73) Needs a specific format Unparseable output Output format, schema, or examples (Lessons 8, 31) Must follow a style or persona Generic or inconsistent tone Role/persona and style examples (Lessons 34, 26) Processes untrusted input Manipulation Delimiters and data-handling rules (Lesson 37) Use this map part by part, not for the task as a whole.

A Worked Example: A Fee and Policy Assistant The Task A student asks: "How much will I pay for the Django REST APIs course if I pay in full, and can I still get a refund if I change my mind next week?"

Requirements:

• Use the current policy documents, not general knowledge.
• The amount must be exactly correct.
• The refund answer must follow the policy rules.
• The reply must be friendly, short, and suitable for students. Step 1: Break the Task Into Parts and Identify Risks Part Risk Technique Find the relevant policy text Model does not know the platform's policy Retrieval of policy sections Read the fee, discount, and GST from the policy Misreading or inventing values Structured extraction with schema and validation Calculate the amount Arithmetic errors Code Decide refund eligibility "next week" Rule misapplied; missing information Chain-of-thought over stated rules, with a rule to ask for missing details Write the reply Wrong tone; changed numbers Persona, format rules, and a check that numbers match Step 2: Arrange the Parts as a Chain Question ↓ [Retrieve] relevant policy sections (code or search) ↓ [Extract] fee facts as JSON (model) → [Validate] values appear in policy (code) ↓ [Compute] total amount (code) ↓ [Reason] refund eligibility step by step from refund rules (model, CoT) ↓ [Write] reply in assistant persona, using computed amount and decision (model) ↓ [Check] reply numbers match computed values (code) Each technique appears exactly where its problem occurs, and nowhere else. The final reply step does not calculate anything, and the calculation step does not write prose.

Step 3: A Testable Skeleton The following skeleton shows how the parts connect. To keep it runnable without an API, two model steps are replaced by simple stand-ins: retrieval uses word overlap, and extraction uses regular expressions on a known policy format. In a real system, these would be a search over your documents (Lessons 101–106) and a model-based extraction step (Lesson 22).

import re

POLICY_SECTIONS = { "fees": "The Django REST APIs course fee is 4999 rupees. Students who pay in full " "get 10% off. GST of 18% is added after any discount.", "refunds": "Refunds are available within 7 days of purchase if less than 20% of " "the course is completed.", "certificates": "Certificates are issued after all modules and the final project " "are completed.", }

def retrieve(question: str, k: int = 1) -> dict[str, str]: """Stand-in for retrieval: rank policy sections by shared words.""" words = set(re.findall(r"[a-z]+", question.lower())) ranked = sorted( POLICY_SECTIONS.items(), key=lambda item: len(words & set(re.findall(r"[a-z]+", item[1].lower()))), reverse=True, ) return dict(ranked[:k])

def extract_fee_facts(policy_text: str) -> dict[str, int]: """Stand-in for a model extraction step that returns structured facts.""" return { "fee": int(re.search(r"fee is (\d+)", policy_text).group(1)), "discount_percent": int(re.search(r"(\d+)% off", policy_text).group(1)), "gst_percent": int(re.search(r"GST of (\d+)%", policy_text).group(1)), }

def facts_are_grounded(facts: dict[str, int], policy_text: str) -> bool: """Validation gate: every extracted value must appear in the policy text.""" return all(str(value) in policy_text for value in facts.values())

def compute_total(facts: dict[str, int], pays_in_full: bool) -> float: """Exact calculation in code: discount first (if eligible), then GST.""" amount = facts["fee"] if pays_in_full: amount = amount * (100 - facts["discount_percent"]) / 100 return round(amount * (100 + facts["gst_percent"]) / 100, 2)

question = "How much will I pay for the Django REST APIs course fee if I pay in full?" context = retrieve(question) facts = extract_fee_facts(context["fees"]) print(list(context)) print(facts, facts_are_grounded(facts, context["fees"])) print(compute_total(facts, pays_in_full=True), compute_total(facts, pays_in_full=False)) Output:

['fees'] {'fee': 4999, 'discount_percent': 10, 'gst_percent': 18} True 5308.94 5898.82 Code Explanation retrieve() Selects the policy section that shares the most words with the question. Here it correctly selects the fee section. A second retrieval for the refund question would select the refund section.

extract_fee_facts() Returns the fee facts as a dictionary of integers, which is the interface a model-based extraction step would also provide. Because later steps depend only on this structure, the stand-in can be replaced by a model call without changing anything else (Lesson 62, contracts).

facts_are_grounded() The validation gate: each extracted value must appear in the policy text (Lesson 73). An invented fee would stop the chain here.

compute_total() Applies the policy's order of operations exactly: a 10% discount for paying in full, then 18% GST. ₹4,999 × 0.9 = ₹4,499.10, and × 1.18 = ₹5,308.94. Without the full-payment discount: ₹4,999 × 1.18 = ₹5,898.82. The model never performs this arithmetic.

Step 4: The Model Steps The reasoning and writing steps use focused prompts:

Refund reasoning (chain-of-thought over stated rules):

Using only the refund rules in , decide whether a student who buys today and asks for a refund "next week" would be eligible.

Check each condition step by step inside tags. If eligibility depends on information not provided (for example, exact days or course progress), say what is missing. Then give the decision inside tags: Eligible, Not eligible, or Depends. Reply writing (persona, format, and fixed facts):

You are the friendly enrollment assistant for an online course platform. Write a reply of at most 80 words to the student's question in .

Use exactly these facts, without changing any number: Total if paid in full: ₹5,308.94 (includes 10% full-payment discount and 18% GST) Refund decision: Depends Refund rule: within 7 days of purchase and less than 20% of the course completed

If the refund decision is "Depends", explain the two conditions simply. Why the refund decision is "Depends": "Next week" could mean 5 days or 10 days after purchase, and course progress is unknown. A reasoning step instructed to identify missing information should not guess. The reply then explains the conditions instead of making a promise.

Step 5: Final Check in Code Before sending the reply, check that every amount in it matches the computed values (Lesson 20, number checks; Lesson 73, cross-step validation). If the reply contains a different amount, send it back for revision or to a person.

Common Combinations Some combinations appear so often that they are worth recognizing as patterns.

Combination Purpose Example Retrieval + grounded QA + evidence check Answer from documents without invention Policy assistant (Lessons 19, 70) Extraction + code computation + explanation Exact numbers in natural language Fee calculator, chart questions (Lessons 58, 76) CoT + self-consistency (adaptive) Reliable decisions on difficult cases Eligibility checks (Lesson 46) Few-shot + structured output + validation Consistent, parseable data Ticket classification at scale (Lessons 30, 31) ReAct / tool calling + confirmation Live data and safe actions Enrollment assistant (Lessons 59, 78) Generate + evaluate + select Best of several open-ended options Design decisions (Lesson 48) Chain + validation + human escalation Complex workflows with safety nets Support automation (Lessons 61, 73) When Combining Becomes Over-Engineering Every technique adds cost, latency, and maintenance. Combining techniques is only worthwhile when each one solves a problem you have observed or can clearly expect.

Warning Signs Techniques are added because they are "advanced", not because a failure required them. A self-consistency step runs on inputs where samples always agree (Lesson 46). Chain-of-thought is used for simple lookups (Lesson 43). A chain has steps that never change the result. Nobody can explain why a particular step exists. How to Avoid It Start simple: one clear prompt with context and a defined output (Lesson 10, Tip 1). Test on realistic inputs and record failures (Lesson 12). Classify each failure using the problem-to-technique map. Add only the technique that addresses that failure. Measure again, including cost and latency (Lesson 66). Remove techniques that do not improve results (ablation, Lesson 66). Common Mistakes Mistake 1: Applying One Technique to the Whole Task What it is: Using chain-of-thought or self-consistency for an entire multi-part task.

Why it happens: The technique worked well elsewhere.

What it causes: The technique helps one part, while other parts still fail, for example reasoning improves but numbers are still miscalculated.

How to fix it: Break the task into parts and choose techniques per part.

Mistake 2: Stacking Techniques Without Measurement What it is: Adding retrieval, few-shot, CoT, self-consistency, and verification all at once.

Why it happens: More techniques seem safer.

What it causes: High cost and latency, and no knowledge of which techniques matter.

How to fix it: Add techniques one at a time in response to observed failures, and measure each addition.

Mistake 3: Letting the Model Do Exact Work Inside a Combined System What it is: A sophisticated chain where the model still calculates totals or applies exact rules.

Why it happens: Code steps are overlooked when designing with prompts.

What it causes: The weakest point of the system is an avoidable calculation error.

How to fix it: Use code for every exact operation.

Mistake 4: No Interfaces Between Techniques What it is: Techniques are connected with free-form text.

Why it happens: Each part was developed separately.

What it causes: Later steps misread earlier outputs, and parts cannot be replaced independently.

How to fix it: Define structured contracts between steps (Lessons 62, 71).

Best Practices Diagnose before choosing. Identify what can go wrong in each part of the task.

Map problems to techniques, part by part. Use the problem-to-technique map.

Use code for exact operations and models for language. This single rule removes many errors.

Connect parts through structured interfaces. Validated JSON, tags, and clear contracts.

Add techniques in response to failures, and measure each one. Keep only what helps.

Keep humans in the loop for uncertain or high-stakes outcomes. "Depends" and escalation are valid results.

Try It Yourself Exercise 1: Diagnose a Task For this request, list each part, its main risk, and the technique you would use: "Read this week's 200 student feedback messages, tell me the three most common complaints with percentages, and draft a reply template for each complaint."

Exercise 2: Complete the Fee Assistant Extend the skeleton in this lesson: add a refund-reasoning step with a model, a reply-writing step with the persona prompt, and a final check that every amount in the reply matches compute_total(). Test it with questions about paying in full, paying in installments (no discount), and refunds after 3 days and after 10 days.

Exercise 3: Remove What Does Not Help Take a multi-technique prompt or chain you have built. Run an ablation (Lesson 66): remove one technique at a time and measure accuracy, cost, and latency on the same test set. Decide which techniques to keep, and write one sentence justifying each.