Build a Code Explanation Prompt Workflow
Project Goal Build a tool that produces a code explanation report for a Python file, such as one a new team member would read on their first day, and that checks the explanations against what the code actually does. The workflow:
Outlines the file in code: functions, classes, line ranges, arguments, and docstrings, using Python's ast module (Lesson 90) Explains each function with a structured prompt and numbered lines (Lesson 92) Asks the model to predict outputs for sample calls Runs the code with the same calls in a separate process and compares predictions with real results Writes a Markdown report that highlights every difference between predicted and actual behavior Step 4 is what makes this workflow valuable. As Lesson 92 showed, a model often explains what code is meant to do rather than what it does. Comparing predictions with real execution turns that weakness into a bug-finding tool.
Requirements Requirement Details Python 3.10 or later Packages pip install openai API key OPENAI_API_KEY environment variable Reuse llm.py from Project 128 Warning: This workflow runs the code being explained. Use it only on your own, trusted code. The separate process limits time and isolates some settings, but it is not a sandbox (Lesson 58).
Project Structure code_explainer/ ├── llm.py # call_model(instructions, prompt) ├── explain.py # outline, execution, prompts, report └── main.py # explain a file and save report.md Step 1: Outline the File in Code explain.py (first part)
import ast import json import os import subprocess import sys import tempfile
def number_lines(code: str) -> str: return "\n".join(f"{i:>3}: {line}" for i, line in enumerate(code.splitlines(), start=1))
def outline_code(code: str) -> list[dict]: """List top-level functions and classes with their line ranges, arguments, and docstrings.""" tree = ast.parse(code) items = [] for node in tree.body: if isinstance(node, (ast.FunctionDef, ast.ClassDef)): items.append({ "name": node.name, "kind": "class" if isinstance(node, ast.ClassDef) else "function", "lines": f"{node.lineno}-{node.end_lineno}", "args": [a.arg for a in node.args.args] if isinstance(node, ast.FunctionDef) else [], "docstring": (ast.get_docstring(node) or "").split("\n")[0], }) return items
def source_of(code: str, item: dict) -> str: start, end = map(int, item["lines"].split("-")) return "\n".join(code.splitlines()[start - 1:end]) Code Explanation ast.parse() reads the code into a syntax tree without running it. tree.body contains only top-level statements, so nested helper functions are not listed separately. lineno and end_lineno give each definition's first and last line, which source_of() uses to cut out that function's code. ast.get_docstring() returns the docstring; only its first line is kept as a short description of the intended behavior. Outlining in code is exact and free. The model is used only for what code cannot do: explaining meaning.
Step 2: Run the Real Code def actual_outputs(code: str, function: str, calls: list[str], timeout: int = 5) -> list[str]: """Run function(call) for each argument string in a separate process; return repr results.""" script = code + "\n\nimport json\nresults = []\n" for call in calls: script += (f"try:\n results.append(repr({function}({call})))\n" f"except Exception as error:\n results.append('raises ' + type(error).name)\n") script += "print(json.dumps(results))\n" with tempfile.TemporaryDirectory() as workdir: path = os.path.join(workdir, "run.py") with open(path, "w", encoding="utf-8") as file: file.write(script) result = subprocess.run([sys.executable, "-I", path], capture_output=True, text=True, timeout=timeout, cwd=workdir) return json.loads(result.stdout) Code Explanation A small script is built: the file's code, followed by one try block per call. repr() records exact Python values, such as 40.0 or 'AB', so they can be compared with predictions. Exceptions are recorded as raises ZeroDivisionError, the same format the model is asked to use. The script runs in a separate process in a temporary directory with a time limit, using isolated mode (-I), as in Lessons 58 and 60. Step 3: Explain and Predict EXPLAIN = """Explain the Python function in for {reader}. Sections: Purpose (1-2 sentences), Inputs and output, Step by step (reference line numbers), Edge cases and possible problems. Describe what the code actually does, not what its name suggests."""
PREDICT = """For the function in , predict the exact Python repr of the return value for each call in , or "raises ". Base predictions only on the code. Return only a JSON list of strings, one per call, in order."""
def explain_function(code: str, item: dict, calls: list[str], call_model, reader: str = "a junior developer") -> dict: numbered = number_lines(source_of(code, item)) explanation = call_model(EXPLAIN.format(reader=reader), f"\n{numbered}\n") predicted = json.loads(call_model( PREDICT, f"\n{numbered}\n\n\n{json.dumps(calls)}\n")) actual = actual_outputs(code, item["name"], calls) mismatches = [{"call": c, "predicted": p, "actual": a} for c, p, a in zip(calls, predicted, actual) if p != a] return {"name": item["name"], "explanation": explanation, "mismatches": mismatches} Design Decisions The explanation prompt follows Lesson 92: a reader, fixed sections, line references, and the instruction to describe actual behavior. The prediction prompt is separate, so the model commits to specific values that can be checked. Mismatches list every call where the prediction differs from reality. Each one means that either the model misunderstood the code, or the code does not do what it appears to do, which is often a bug. Step 4: The Report def report(results: list[dict]) -> str: lines = ["# Code Explanation Report", ""] for result in results: lines += [f"## {result['name']}", "", result["explanation"].strip(), ""] if result["mismatches"]: lines.append("Behavior differs from the explanation's predictions:") lines += [f"- {m['call']}: predicted {m['predicted']}, actual {m['actual']}" for m in result["mismatches"]] lines.append("") return "\n".join(lines).rstrip() + "\n" Mismatches are placed directly under the explanation they contradict, so a reader sees immediately where the explanation should not be trusted.
Step 5: Testing Without the API CODE = '''def average_progress(students): """Return the average progress of active students.""" total = 0 for s in students: if s["active"]: total += s["progress"] return total / len(students)
def initials(name): """Return the initials of a full name.""" return "".join(part[0].upper() for part in name.split()) '''
items = outline_code(CODE) print(items)
calls = ['[{"active": True, "progress": 80}, {"active": False, "progress": 20}]', '[]'] print(actual_outputs(CODE, "average_progress", calls))
def simulated_model(instructions: str, prompt: str) -> str: if instructions.startswith("For the function"): return json.dumps(["80.0", "0"]) # predictions based on the docstring's intent return "Purpose: Returns the average progress of active students."
print(report([explain_function(CODE, items[0], calls, simulated_model)])) Output:
[{'name': 'average_progress', 'kind': 'function', 'lines': '1-7', 'args': ['students'], 'docstring': 'Return the average progress of active students.'}, {'name': 'initials', 'kind': 'function', 'lines': '10-12', 'args': ['name'], 'docstring': 'Return the initials of a full name.'}] ['40.0', 'raises ZeroDivisionError']
Code Explanation Report
average_progress
Purpose: Returns the average progress of active students.
Behavior differs from the explanation's predictions:
[{"active": True, "progress": 80}, {"active": False, "progress": 20}]: predicted 80.0, actual 40.0[]: predicted 0, actual raises ZeroDivisionError What the Test Confirms The outline finds both functions with correct line ranges and docstrings. Real execution shows the actual behavior: 40.0 (the code divides by all students) and a crash on an empty list. The report places both bugs directly below the explanation. The simulated explanation matched the docstring's intent, exactly the mistake a real model often makes, and the workflow catches it. Step 6: The Command-Line Tool main.py
import json import sys
from explain import explain_function, outline_code, report from llm import call_model
path, calls_path = sys.argv[1], sys.argv[2] code = open(path, encoding="utf-8").read() sample_calls = json.load(open(calls_path, encoding="utf-8")) # {"function_name": ["args", ...]}
results = [explain_function(code, item, sample_calls.get(item["name"], []), call_model) for item in outline_code(code) if item["kind"] == "function"]
with open("report.md", "w", encoding="utf-8") as file: file.write(report(results)) print(f"Explained {len(results)} functions; " f"{sum(bool(r['mismatches']) for r in results)} have behavior mismatches.") Sample Calls File { "average_progress": ["[{\"active\": True, \"progress\": 80}]", "[]"], "initials": ["\"Priya Raman\"", "\"\""] } Each value is the argument text that is inserted into a Python call, such as initials("Priya Raman"). The file itself is JSON, but the text inside each string is Python code, so it must use Python syntax: True and None, not JSON's true and null. If you wrote true inside an argument string, the call would fail with raises NameError and appear as a false mismatch in the report.
Choose sample calls that cover normal inputs, boundaries, and edge cases such as empty values (Lesson 94). The initials("") call, for example, returns an empty string, which may or may not be the behavior you want.
Common Mistakes Mistake 1: Trusting Explanations Without Execution What it is: Publishing model explanations of code without checking behavior.
Why it happens: Explanations read well.
What it causes: Documentation that describes intended behavior and hides bugs.
How to fix it: Compare predicted and actual outputs for representative calls.
Mistake 2: Running Untrusted Code What it is: Using the workflow on code from unknown sources.
Why it happens: The tool works on any file.
What it causes: Harmful code runs with your permissions.
How to fix it: Use it only on trusted code, or run it inside a real sandbox.
Mistake 3: Only Normal Sample Calls What it is: Testing explanations with typical inputs only.
Why it happens: Typical inputs are easy to write.
What it causes: Edge-case bugs, such as division by zero, go unnoticed.
How to fix it: Include empty, boundary, and unusual inputs.
Best Practices Outline code with ast, not with the model. Exact structure at no cost.
Ask for predictions as well as explanations. Predictions can be checked.
Run code in a separate process with a time limit. And only trusted code.
Report mismatches next to the explanations. Make doubts visible.
Treat every mismatch as a possible bug. Investigate before trusting either side.
Try It Yourself Exercise 1: Explain Your Own Module Run the workflow on a utility module from one of your projects, with at least three sample calls per function, including edge cases. Read the report and investigate every mismatch.
Exercise 2: Explain for Different Readers Run the workflow twice with reader set to "a junior developer" and "a senior reviewer". Compare the explanations, and decide which sections each reader actually needs.
Exercise 3: Classes and Methods Extend outline_code() to list methods inside classes, using each class node's body. Adapt actual_outputs() to call a method on a new instance, and test it with a small class from your own code.