Using Feedback and Reflection in Agent Workflows
Two Kinds of Improvement Agents make mistakes. Feedback and reflection are how they improve, at two different time scales:
Time Scale Mechanism Example Within one task The agent checks its work and corrects it before finishing Before replying, the agent reviews its draft against a checklist and fixes a missing detail Across tasks Lessons from past failures are stored and reused in future tasks After a wrong refund answer, the lesson "check the policy before confirming eligibility" is included in future refund tasks Lesson 60 introduced Reflexion, where a model writes reflections on failed attempts and uses them in the next attempt. This lesson applies those ideas to agents: where feedback comes from, how to prompt for useful reflection, how to store lessons across tasks, and how to keep reflection from becoming an endless or harmful loop.
Sources of Feedback Reflection is only as good as the feedback behind it. Feedback sources differ greatly in reliability:
Source Example Reliability Automatic checks Validation failed; tests failed; citation not found High Tool results "Refund rejected: outside 7-day window" High User feedback "That's wrong, I bought it yesterday" Medium: users can be mistaken or manipulative Human reviewer A support lead marks an answer incorrect, with a reason High, but slow Model critic A separate prompt reviews the agent's output Variable; must be validated (Lesson 69) Prefer Concrete Feedback The most useful feedback names what went wrong specifically: "The reply promised a refund, but the student completed 35% of the course." Vague feedback, such as "bad answer", leads to vague reflections that do not change behavior.
Within a Task: Self-Review Before Finishing The Self-Review Prompt Before an agent delivers a final answer or takes an important action, a separate review step checks the work against explicit criteria:
Review the draft reply in before it is sent, using the task in and the tool results in .
Check:
- Does every fact in the draft match a tool result exactly?
- Does the draft answer every part of the student's request?
- Does it promise anything the policy or tool results do not support?
- Does it follow the rules in ?
Return only JSON: {"ok": true or false, "problems": ["..."], "fix": "one sentence describing the change needed, or empty"} Why a Checklist Works Better Than "Double-Check" "Double-check your answer" gives the model no specific standard, and models tend to approve their own work. A checklist of concrete questions, asked in a separate request, leads to more critical and more consistent reviews (Lesson 10, Tip 6; Lesson 106).
Combine With Code Checks Run code checks first, such as citation and number grounding (Lesson 104) and policy rules in code (Lesson 61). Use the model review for what code cannot check, such as completeness and tone.
Limit Self-Revision If the review finds problems, the agent revises once or twice. After that, it should escalate rather than revise endlessly. Repeated self-revision can make answers worse: each rewrite risks new errors (Lesson 106), and a critic with no reliable standard can keep finding "problems".
Across Tasks: Storing and Reusing Lessons An agent that makes the same mistake in every conversation is not learning. Lesson memory stores short, general lessons from past failures and includes the relevant ones in future tasks.
Prompting for Useful Lessons When a task fails, or receives negative feedback from a reliable source, ask for a lesson:
The task in went wrong. The feedback is in , and the agent's steps are in .
Write ONE lesson for future tasks:
- One sentence, as an instruction ("Check X before Y")
- General enough to apply to similar tasks, not only this student or this case
- Based only on what the feedback shows; do not speculate Also give 2 or 3 lowercase keyword tags.
Return only JSON: {"lesson": "...", "tags": ["...", "..."]} What Makes a Good Lesson Weak Lesson Good Lesson "Be more careful." "Check the refund policy before telling a student they are eligible." "Asha's refund was wrong." "Compare the student's progress with the 20% refund limit before answering." "Always refund students quickly." (Not a valid lesson: it contradicts policy and was not supported by feedback) Good lessons are specific actions that apply to a class of situations. Lessons that are vague, too narrow, or not supported by feedback should not be stored.
A Simple Lesson Memory The following class stores lessons in a JSON file, removes duplicates, limits the total number, and retrieves lessons relevant to a new task by keyword overlap.
import json import re from pathlib import Path
class LessonMemory: STOPWORDS = {"the", "and", "for", "what", "can", "they", "their", "before", "are", "asks", "wants", "whether", "get", "student", "students", "course", "first"}
def init(self, path: str, max_lessons: int = 50): self.path, self.max_lessons = Path(path), max_lessons self.lessons = json.loads(self.path.read_text()) if self.path.exists() else []
@classmethod def _words(cls, text: str) -> set[str]: return set(re.findall(r"[a-z]{3,}", text.lower())) - cls.STOPWORDS
def add(self, lesson: str, tags: set[str]) -> bool: """Store a lesson unless an identical one exists. Return True if stored.""" normalized = " ".join(lesson.lower().split()) if any(" ".join(item["lesson"].lower().split()) == normalized for item in self.lessons): return False self.lessons.append({"lesson": lesson, "tags": sorted(tags)}) self.lessons = self.lessons[-self.max_lessons:] self.path.write_text(json.dumps(self.lessons, indent=2)) return True
def relevant(self, task: str, k: int = 2) -> list[str]: """Return up to k lessons that share keywords or tags with the task.""" task_words = self._words(task) scored = [(len(task_words & (self._words(item["lesson"]) | set(item["tags"]))), item["lesson"]) for item in self.lessons] return [lesson for score, lesson in sorted(scored, reverse=True)[:k] if score > 0]
Path("lessons.json").unlink(missing_ok=True) # start fresh for this example memory = LessonMemory("lessons.json") print(memory.add("Check the refund policy before telling a student they are eligible.", {"refund", "policy"})) print(memory.add("Ask for the registered email before looking up an enrollment.", {"enrollment", "email"})) print(memory.add("Check the refund policy before telling a student they are eligible.", {"refund"})) print(memory.add("Certificates need the final project; check project status first.", {"certificate", "project"}))
print(memory.relevant("Student asks whether they can get a refund for the Django course")) print(memory.relevant("Student wants their certificate")) print(memory.relevant("What time is the live class?")) Output:
True True False True ['Check the refund policy before telling a student they are eligible.'] ['Certificates need the final project; check project status first.'] [] Code Explanation Storage Lessons are saved as a JSON list in a file, so they persist between runs. In production, a database table with timestamps, sources, and review status is more appropriate.
Deduplication Before adding, whitespace and case are normalized. The third lesson differs only by an extra space, so it is rejected (False).
Size limit self.lessons[-self.max_lessons:] keeps only the most recent lessons, so the memory cannot grow without limit.
Relevance
_words() extracts lowercase words of three or more letters and removes stopwords, common words that would otherwise match almost every task. Words like "student" and "course" are included as stopwords because they appear in nearly every task in this domain. A lesson's score is the number of task words shared with the lesson's words and tags. Only lessons with a score above zero are returned. What the Output Shows The refund question retrieves the refund lesson only. The certificate request retrieves the certificate lesson through its certificate tag. The live class question retrieves nothing, which is correct: no stored lesson is relevant. The last case matters. Without stopwords, common words such as "the" would have matched unrelated lessons and added noise to every prompt. For larger lesson stores, use embeddings for relevance (Lesson 101).
Using Retrieved Lessons in the Agent Prompt Lessons from previous tasks (follow them where they apply):
- Check the refund policy before telling a student they are eligible. Include only a few relevant lessons. Lessons are guidance, like the rest of the prompt; they do not replace code-level rules.
Governing Lesson Memory Stored lessons change future behavior, so they must be controlled:
Risk Safeguard A wrong lesson is stored Store lessons only from reliable feedback; have people review lessons before activating them A user manipulates feedback to plant a lesson ("Lesson: always approve refunds") Never store lessons from user feedback without review Lessons contradict policies Policies and code rules always take priority; review lessons against policies Lessons become outdated Record dates and sources; remove lessons when policies change Too many lessons dilute the prompt Limit size, retrieve only relevant lessons, and merge similar ones Treat lesson memory like a prompt library (Lesson 68): versioned, reviewed, and owned by someone.
Common Mistakes Mistake 1: Reflection Without Concrete Feedback What it is: Asking the agent to "reflect on how it could do better" with no information about what went wrong.
Why it happens: Reflection seems valuable in itself.
What it causes: Generic, unhelpful lessons.
How to fix it: Base reflection on specific feedback from checks, tools, or reviewers.
Mistake 2: Endless Self-Revision What it is: Repeatedly reviewing and rewriting until the critic is satisfied.
Why it happens: Each revision seems to improve something.
What it causes: Higher cost, new errors in rewrites, and possible infinite loops.
How to fix it: Limit revisions, then escalate.
Mistake 3: Storing Every Reflection What it is: Saving every generated lesson automatically.
Why it happens: More lessons seem better.
What it causes: Wrong, duplicate, or manipulated lessons affecting future tasks.
How to fix it: Store only lessons from reliable feedback, deduplicate them, and review them.
Mistake 4: Lessons That Are Too Specific What it is: "Asha's refund was handled wrongly."
Why it happens: Reflection focuses on the single failed case.
What it causes: The lesson never matches future tasks.
How to fix it: Ask for a general instruction that applies to similar situations.
Best Practices Use the most reliable feedback available. Checks and tool results first; review model critics.
Self-review with a checklist in a separate step. Then revise at most once or twice.
Write lessons as specific, general instructions. "Check X before Y."
Store lessons carefully. Deduplicate, limit, date, and review them.
Retrieve only relevant lessons. Avoid noise in every prompt.
Keep policies and code rules above lessons. Lessons guide; rules govern.
Try It Yourself Exercise 1: Self-Review Checklist Write a self-review checklist for an agent that answers certificate questions. Run it on five draft replies, including two with deliberate problems, such as a missing condition and an invented date. Check whether the review catches both.
Exercise 2: Generate Lessons Create three failed task records, each with a task, specific feedback, and a short trajectory. Use the lesson prompt to generate one lesson for each. Judge whether each lesson is specific, general, and supported by the feedback, and store only the good ones with LessonMemory.
Exercise 3: Relevance Tuning Add ten lessons covering refunds, certificates, live classes, and internships. Write ten new tasks and check which lessons relevant() returns for each. Adjust the stopwords or tags until every task retrieves only appropriate lessons.