Reflexion: Improving Agents Through Feedback
What Is Reflexion? Reflexion is a technique in which a model improves its results over several attempts by reflecting on its own failures in words and using those reflections in the next attempt.
The cycle looks like this:
Attempt the task ↓ Evaluate the result (tests, checks, or a judge) ↓ Failed? → Write a reflection: what went wrong and what to do differently ↓ Store the reflection in memory ↓ Attempt again, with the reflections included in the prompt Reflexion was introduced in the 2023 paper "Reflexion: Language Agents with Verbal Reinforcement Learning" by Shinn et al. The authors reported substantial improvements on several tasks, including code generation benchmarks such as HumanEval, compared with the same models without reflection.
What "Verbal Reinforcement" Means In traditional reinforcement learning, a system improves by adjusting its internal parameters based on rewards and penalties. Reflexion does not change the model at all. Instead, the "learning" is stored as text: written lessons such as "I forgot to handle empty input; next time, check for an empty string first." These lessons are placed in the prompt for the next attempt.
Why Reflexion Works Errors Become Specific Guidance A simple retry ("try again") gives the model no information about what went wrong, so it often repeats the same mistake. A reflection turns a failure into a specific instruction for the next attempt.
Approach Information for the Next Attempt Retry without feedback None Retry with the raw error What failed, but not why or how to fix it Retry with a reflection What failed, why, and what to do differently Analysis Is a Separate Step Writing a reflection is a different task from writing the solution. The model looks at the failed attempt and the feedback, and focuses only on diagnosing the problem. This applies the principle of separating creation from judgment (Lesson 10, Tip 6).
Memory Accumulates Reflections from several failed attempts are kept together. The model can avoid all previously identified mistakes, not only the most recent one.
The Three Components The Reflexion framework has three roles, which can all be played by the same model with different prompts.
The Actor Produces the attempt: code, an answer, or a sequence of actions. Its prompt includes the task and any reflections from earlier attempts.
The Evaluator Judges whether the attempt succeeded and produces feedback. Evaluators can be:
Evaluator Type Example Reliability Exact checks Unit tests, schema validation, a calculation High Heuristics "The agent repeated the same action three times" Medium Model judge A prompt that rates the answer against criteria Variable
- The Self-Reflection Model Reads the attempt and the evaluator's feedback, and writes a short reflection: what went wrong and what should be done differently. The reflection is stored in memory.
The Importance of the Evaluator Reflexion is only as good as its feedback. With unit tests, the feedback is precise and trustworthy. With a weak model-based evaluator, the reflection may "fix" problems that do not exist or miss real ones. This is why Reflexion is especially effective for code, where tests give reliable feedback.
A Worked Example: Writing a Function The Task Write a Python function slugify(text) that converts a course title into a URL slug: lowercase words separated by single hyphens, with symbols removed. The Tests slugify("Python for Beginners") → "python-for-beginners" slugify(" Django REST ") → "django-rest" slugify("C++ & Data Structures!") → "c-data-structures" slugify("") → "" Attempt 1 def slugify(text): return text.lower().replace(" ", "-") Evaluator feedback (from running the tests):
slugify(" Django REST ") returned '--django---rest--', expected 'django-rest' slugify("C++ & Data Structures!") returned 'c++-&-data-structures!', expected 'c-data-structures' Reflection My function replaced each space with a hyphen, so leading, trailing, and repeated spaces became extra hyphens. It also kept symbols such as +, &, and !. Next time: extract only the words made of letters and digits, then join them with single hyphens. This removes symbols and handles extra spaces automatically. Attempt 2 (With the Reflection in the Prompt) import re
def slugify(text): words = re.findall(r"[a-z0-9]+", text.lower()) return "-".join(words) Evaluator feedback: all tests pass.
Example Explanation The test failures showed what was wrong. The reflection explained why (replacing spaces one by one) and proposed a different approach (extracting words). The second attempt applied the strategy, not just a patch for one failing case.
Implementing Reflexion in Python Step 1: An Evaluator That Runs Tests The evaluator runs the generated function against test cases in a separate process with a time limit, and reports each failure in detail.
import os import subprocess import sys import tempfile
TESTS = [ ('slugify("Python for Beginners")', "python-for-beginners"), ('slugify(" Django REST ")', "django-rest"), ('slugify("C++ & Data Structures!")', "c-data-structures"), ('slugify("")', ""), ]
def build_test_script(code: str, tests: list[tuple[str, str]]) -> str: """Combine candidate code with checks that print a message for every failure.""" lines = [code, "", "failures = []"] for call, expected in tests: lines += [ "try:", f" result = {call}", "except Exception as error:", " result = f'raised {type(error).name}: {error}'", f"if result != {expected!r}:", f" failures.append({call!r} + ' returned ' + repr(result) + {', expected ' + repr(expected)!r})", ] lines += ["print('\n'.join(failures))", "raise SystemExit(1 if failures else 0)"] return "\n".join(lines)
def run_tests(code: str, tests: list[tuple[str, str]], timeout: int = 5) -> tuple[bool, str]: with tempfile.TemporaryDirectory() as workdir: path = os.path.join(workdir, "candidate.py") with open(path, "w", encoding="utf-8") as file: file.write(build_test_script(code, tests)) try: result = subprocess.run( [sys.executable, "-I", path], capture_output=True, text=True, timeout=timeout, cwd=workdir, ) except subprocess.TimeoutExpired: return False, "Timed out: the code may contain an infinite loop." if result.returncode == 0: return True, "All tests passed." return False, (result.stdout + result.stderr).strip()
attempt_1 = 'def slugify(text):\n return text.lower().replace(" ", "-")\n' attempt_2 = 'import re\n\ndef slugify(text):\n return "-".join(re.findall(r"[a-z0-9]+", text.lower()))\n'
print(run_tests(attempt_1, TESTS)) print(run_tests(attempt_2, TESTS)) Output:
(False, 'slugify(" Django REST ") returned \'--django---rest--\', expected \'django-rest\'\nslugify("C++ & Data Structures!") returned \'c++-&-data-structures!\', expected \'c-data-structures\'') (True, 'All tests passed.') Code Explanation build_test_script() Generates a Python script that contains the candidate code followed by one check per test:
Each call runs inside try/except, so an exception becomes a readable failure message instead of stopping all tests. {expected!r} inserts the expected value as a Python literal (with quotes), so string comparison works correctly. Every failure is collected and printed, and the script exits with status 1 if any test failed. run_tests()
Writes the script to a temporary directory and runs it in a separate, isolated Python process (-I) with a time limit, as in Lesson 58. Returns (True, ...) on success, or (False, failure messages) on failure. Detailed failure messages are what make good reflections possible. Warning: As explained in Lesson 58, a separate process with a timeout is not a full sandbox. Generated code can still access files and the network. Use a proper sandbox, such as an isolated container with no network or secrets, for anything beyond local experiments.
Step 2: The Actor, the Reflector, and the Loop Requirements: pip install openai and an OPENAI_API_KEY environment variable.
import re
from openai import OpenAI
client = OpenAI() MODEL = "gpt-5.4" # Replace with a model available in your account
TASK = ( "Write a Python function slugify(text) that converts a course title into a URL " "slug: lowercase words separated by single hyphens, with symbols removed." )
ACTOR_INSTRUCTIONS = """Write Python code that completes the task in . If contains lessons from earlier failed attempts, follow them. Return only the code in a `python code block."""
REFLECT_INSTRUCTIONS = """The code in failed the tests shown in . In 2 to 4 sentences, explain why it failed and describe a different approach that would avoid this kind of failure in general. Do not write code."""
def call(instructions: str, text: str) -> str: response = client.responses.create(model=MODEL, instructions=instructions, input=text) return response.output_text
def extract_code(text: str) -> str: match = re.search(r"(?:python)?\s*\n(.*?)", text, re.DOTALL) return match.group(1) if match else text
def reflexion(task: str, tests: list[tuple[str, str]], max_attempts: int = 3) -> str | None: reflections: list[str] = [] for attempt in range(1, max_attempts + 1): memory = "\n".join(f"- {r}" for r in reflections) or "(none)" code = extract_code(call( ACTOR_INSTRUCTIONS, f"\n{task}\n\n\n{memory}\n", )) passed, feedback = run_tests(code, tests) print(f"Attempt {attempt}: {'passed' if passed else 'failed'}") if passed: return code reflection = call( REFLECT_INSTRUCTIONS, f"\n{code}\n\n\n{feedback}\n", ).strip() reflections.append(reflection) print("Reflection:", reflection) return None
final_code = reflexion(TASK, TESTS) print(final_code or "No passing solution within the attempt limit.") Code Explanation ACTOR_INSTRUCTIONS The actor receives the task and all stored reflections, and is told to follow them.
REFLECT_INSTRUCTIONS
The reflector sees only the failed code and the test feedback. It must explain why the code failed and propose a different approach in general, which helps avoid fixes that only patch individual test cases (Lesson 12, overfitting). "Do not write code" keeps the reflection short and focused on strategy; the actor writes the code. reflections (memory) A list that grows with each failed attempt. Every new attempt sees all previous lessons.
max_attempts A strict limit on the number of cycles, which controls cost and prevents endless loops.
Return value The passing code, or None if no attempt passed. Your application must handle the None case, for example by asking a developer to review.
Reflexion Beyond Code The same loop works for other tasks where failures can be detected.
Task Evaluator Example Reflection JSON extraction Schema validation (Lesson 22) "I returned dates as '15 Nov'; the schema requires YYYY-MM-DD." Summarization Number check and faithfulness check (Lesson 20) "I included a reason that is not in the source; I must only use stated facts." SQL generation Run the query on a test database "I joined on the wrong column; enrollments link to students through student_id." ReAct agent tasks Task success check "I searched for the course name with a typo; I should look up the catalog list first." The key requirement is the same in every case: an evaluator that gives specific, trustworthy feedback.
Limitations Weak Evaluators Produce Misleading Reflections If a model judge wrongly reports a correct answer as wrong, the reflection invents a problem, and the next attempt may become worse.
Reflections Can Be Wrong The model may misdiagnose the cause of a failure. Several attempts and good test coverage reduce this risk.
Passing Tests Is Not the Same as Being Correct If the tests are incomplete, Reflexion produces code that passes the tests but fails on other inputs. The quality of the result is limited by the quality of the tests.
Cost Each failed attempt costs an actor call and a reflection call. Use attempt limits and reserve Reflexion for tasks where automatic improvement is worth the cost.
Common Mistakes Mistake 1: Retrying Without Reflection What it is: Calling the model again after a failure with the same prompt.
Why it happens: It is the simplest retry logic.
What it causes: The same mistake is often repeated.
How to fix it: Include specific feedback and a reflection in the next attempt.
Mistake 2: Vague Feedback What it is: Telling the reflector only "Tests failed."
Why it happens: The evaluator reports only pass or fail.
What it causes: Reflections based on guesses.
How to fix it: Report which inputs failed, the actual output, and the expected output.
Mistake 3: Reflections That Patch Individual Cases What it is: Reflections such as "Add a special case for 'C++'."
Why it happens: The reflector focuses on the visible failures.
What it causes: Code that passes the tests but fails on similar inputs.
How to fix it: Ask for the general cause and a general approach.
Mistake 4: Running Generated Code Unsafely What it is: Executing generated code in the application's own process.
Why it happens: It is easy to call exec().
What it causes: Security risks from buggy or malicious code.
How to fix it: Run code in a separate, isolated sandbox with time and resource limits.
Mistake 5: No Attempt Limit What it is: Looping until the tests pass.
Why it happens: It seems the model will eventually succeed.
What it causes: Unbounded costs on tasks the model cannot solve.
How to fix it: Set a maximum number of attempts and handle failure explicitly.
Best Practices Use the most reliable evaluator available. Tests and exact checks first; model judges only when necessary.
Give detailed feedback. Inputs, actual outputs, and expected outputs.
Ask reflections for causes and general strategies. Not patches for individual cases.
Keep reflections short and accumulate them. Each attempt should benefit from all earlier lessons.
Limit attempts. Control cost and handle failure gracefully.
Run generated code in a sandbox. Never in your main process.
Make the tests good. Reflexion can only be as correct as the evaluator allows.
Try It Yourself Exercise 1: Run the Evaluator Write three different versions of slugify() by hand, including one that fails only the empty-string test. Run each through run_tests() and read the feedback messages. Decide whether each message gives enough information to fix the code.
Exercise 2: A New Reflexion Task Use reflexion() for a different function, such as is_valid_indian_mobile(number) that accepts 10-digit numbers starting with 6, 7, 8, or 9, optionally prefixed with +91. Write at least six tests, including edge cases, and observe how many attempts are needed.
Exercise 3: Reflexion With a Schema Evaluator Adapt the loop to a JSON extraction task. Use a Pydantic model (Lesson 22) as the evaluator: when validation fails, pass the validation error message to the reflector. Test it on course enrollment messages with unusual date formats.