Prompting for Code Generation

From Single Prompts to a Reliable Process Lesson 90 provided prompt recipes for common code tasks. This lesson, the first in the developer section, focuses on the process of generating code you can trust and maintain. Experienced developers rarely accept generated code from a single prompt. They follow a loop:

1. Specify → what exactly the code must do
2. Contextualize → what the code must fit into
3. Define the interface → names, inputs, outputs, errors
4. Write tests → how correctness will be checked
5. Generate → small units, one at a time
6. Verify → static checks, tests, linting
7. Iterate → feed failures back with specifics
8. Review and own → understand it before merging Each step addresses a specific way generated code goes wrong. Skipping a step usually means discovering its problem later, when it is more expensive to fix.

Step 1: Specify What the Code Must Do A specification is a precise description of required behavior. Generated code can only be as good as its specification: vague requests produce code that solves a slightly different problem (Lesson 5).

A Specification Template Purpose: one sentence describing what the code does and why Inputs: each input, its type, and its valid range Outputs: the return value and its type Rules: the exact behavior, including formulas and business rules Edge cases: empty input, zero, boundaries, missing values Errors: which invalid inputs raise which exceptions Constraints: performance limits, libraries allowed, compatibility Example Specification Purpose: Calculate the amount a student pays for a course. Inputs: fee (int, rupees, >= 0), discount_percent (int, 0–100), gst_percent (int, 0–28) Output: int, the final amount in rupees, rounded to the nearest rupee Rules: apply the discount to the fee first, then add GST to the discounted amount Edge cases: discount_percent = 100 gives 0; gst_percent = 0 gives the discounted fee Errors: raise ValueError for a negative fee or percentages outside their ranges Constraints: standard library only; Python 3.12 Why This Matters Notice the rule "apply the discount first, then add GST". Without it, a model might add GST first, which produces a different amount. The specification removes the ambiguity before any code is written. Writing specifications also often reveals decisions you had not made yet, such as how to round.

Step 2: Provide Context Code does not exist alone. It must fit into a project with existing structure and conventions. The model knows none of this unless you provide it (Lesson 6).

Context to Include Context Example Versions Python 3.12, Django 5.1 Project structure Business logic in services.py; views are thin Conventions Type hints required; errors as custom exception classes Existing interfaces The function signatures and models the new code must use Related code A similar existing function to follow as a pattern Allowed dependencies Standard library plus requests; nothing else Include Only Relevant Code Paste the specific models, functions, and helpers the new code interacts with, not the entire codebase. Irrelevant code distracts the model and increases cost (Lesson 14).

Step 3: Define the Interface First The interface is how other code uses yours: function names, parameters, types, return values, and exceptions. Defining it before generating the implementation keeps generated code compatible with the rest of the project.

def course_total(fee: int, discount_percent: int, gst_percent: int) -> int: """Return the final amount in rupees after discount and GST.

Raises: ValueError: if fee is negative or a percentage is out of range. """ Why Interface First Works The model cannot invent a different function name or parameter order. Other developers can start writing code that calls the function before it is implemented. Tests can be written against the interface before the implementation exists. Step 4: Write Tests Before Generating Tests turn the specification into executable checks. Writing them before generation is the core idea of test-driven development (TDD). With generated code, it has an additional benefit: the tests define correctness independently of the model.

Tests From the Specification import pytest

from pricing import course_total

def test_discount_then_gst(): assert course_total(4999, 10, 18) == 5309 # 4999 × 0.9 = 4499.1; × 1.18 = 5308.938 → 5309

def test_no_discount(): assert course_total(1000, 0, 18) == 1180

def test_full_discount(): assert course_total(4999, 100, 18) == 0

def test_invalid_percent(): with pytest.raises(ValueError): course_total(1000, 120, 18) Should the Model Write the Tests? A model can draft tests from the specification (Lesson 94), which saves time. But tests generated by the same model that writes the code may share its misunderstanding. Review every expected value yourself, especially calculated ones, as done in the comments above.

Step 5: Generate in Small Units The Generation Prompt With the specification, context, interface, and tests ready, the prompt is precise:

Implement the function below for a Python 3.12 project.

{specification}

{function_signature_and_docstring}

{tests}

Requirements:

• The implementation must make all tests pass.
• Use only the standard library.
• Do not change the function signature.
• Return only the implementation in a `python code block. Why Small Units? Large Request ("build the enrollment module") Small Units (one function at a time) Many design decisions made implicitly Each decision visible and reviewable Hard to test and review Each unit tested separately Errors in one part affect others Errors are isolated Output may be truncated Output stays short For larger features, first ask for a plan, a list of modules and functions with their interfaces, review it, then generate each unit (Lesson 15).

Step 6: Verify Automatically Verification has layers, from fastest to most thorough.

Layer 1: Static Checks Without Running the Code Static checks examine code without executing it, so they are safe even for untrusted code. The following checker confirms that generated code parses, uses the required function signature, and avoids modules and functions your project does not allow.

import ast

FORBIDDEN_MODULES = {"os", "subprocess", "shutil", "socket", "requests"} FORBIDDEN_CALLS = {"eval", "exec", "import", "open"}

def check_generated_code(code: str, required_function: str, required_args: list[str]) -> list[str]: """Return a list of problems found without running the code.""" issues = [] try: tree = ast.parse(code) except SyntaxError as error: return [f"Syntax error on line {error.lineno}: {error.msg}"]

for node in ast.walk(tree): if isinstance(node, ast.Import): for alias in node.names: if alias.name.split(".")[0] in FORBIDDEN_MODULES: issues.append(f"Forbidden import: {alias.name}") elif isinstance(node, ast.ImportFrom) and node.module and node.module.split(".")[0] in FORBIDDEN_MODULES: issues.append(f"Forbidden import: {node.module}") elif isinstance(node, ast.Call) and isinstance(node.func, ast.Name) and node.func.id in FORBIDDEN_CALLS: issues.append(f"Forbidden call: {node.func.id}()")

functions = {n.name: n for n in tree.body if isinstance(n, ast.FunctionDef)} if required_function not in functions: issues.append(f"Missing function: {required_function}") else: args = [a.arg for a in functions[required_function].args.args] if args != required_args: issues.append(f"{required_function} has arguments {args}, expected {required_args}") return issues

good = ( "from decimal import Decimal\n\n" "def apply_discount(price, percent):\n" " return price * (100 - percent) / 100\n" ) bad = ( "import os\n\n" "def apply_discount(amount):\n" " os.system('ls')\n" " return eval('amount * 0.9')\n" )

print(check_generated_code(good, "apply_discount", ["price", "percent"])) print(check_generated_code(bad, "apply_discount", ["price", "percent"])) Output:

[] ['Forbidden import: os', 'Forbidden call: eval()', "apply_discount has arguments ['amount'], expected ['price', 'percent']"] Code Explanation ast.parse() converts code into a syntax tree without running it. Invalid code is reported immediately. ast.walk(tree) visits every node. Import nodes are checked against FORBIDDEN_MODULES, and calls to built-in functions such as eval are checked against FORBIDDEN_CALLS. Top-level functions are collected from tree.body. The required function must exist with exactly the expected argument names, in order, so it matches the interface from Step 3. The bad example imports os, calls eval, and changes the signature. All three problems are reported before the code is ever run. The forbidden lists are examples. Adjust them to your project: a data-processing module may legitimately need os.path, while a pricing function should not touch the file system.

Layer 2: Run the Tests Run the tests from Step 4 against the generated code. Because generated code is untrusted until reviewed, run it in an isolated environment with a time limit (Lessons 58 and 60).

Layer 3: Linters and Type Checkers Tools such as Ruff (a fast Python linter) and mypy (a static type checker) catch unused imports, likely bugs, and type mismatches. Run the same tools on generated code that you run on human-written code.

Step 7: Iterate With Specific Feedback When verification fails, send the specific failures back with the code, as in Reflexion (Lesson 60):

Your implementation failed these checks:

test_discount_then_gst: expected 5309, got 5308

Explain the cause in one sentence, then return a corrected implementation in a `python code block. Do not change the function signature or the tests. Why "Do Not Change the Tests" When tests fail, a model may "fix" the problem by weakening the test, for example by changing the expected value. The tests represent the specification; the implementation must change, not the definition of correctness. If you believe a test is wrong, decide that yourself.

Limit Iterations Set a maximum number of attempts, for example three. If the code still fails, the specification may be ambiguous or the task too large; revisit Steps 1 and 5 rather than continuing to retry.

Step 8: Review and Own the Code Passing tests is necessary but not sufficient. Before merging generated code:

Review Question Why Do I understand every line? You are responsible for maintaining it Does it handle errors as the project expects? Consistency and reliability Is it secure for the inputs it will receive? Tests rarely cover attacks (Lesson 90, Recipe 8) Does it introduce new dependencies? Maintenance and security cost Is it consistent with project conventions? Readability for the team Could it reproduce licensed code verbatim? Follow your organization's policy on generated code Note: Treat generated code like code from a new team member: helpful and often good, but reviewed before it becomes part of the project.

A Complete Workflow Example Specification: course_total — discount first, then GST, rounded to nearest rupee Context: Python 3.12, standard library only, pricing.py module Interface: def course_total(fee: int, discount_percent: int, gst_percent: int) -> int Tests: 4 tests, expected values verified by hand Generate: prompt with spec, interface, tests Static check: check_generated_code() → no issues Run tests: 3 pass, 1 fails (rounding) Iterate: failure sent back → corrected implementation Run tests: 4 pass Lint and types: ruff and mypy pass Review: understood, no new dependencies, consistent errors → merge Each step took little time, and each caught or prevented a different kind of problem.

Common Mistakes Mistake 1: Generating Before Specifying What it is: Asking for code with a one-line description.

Why it happens: It is fast to start.

What it causes: Code that makes hidden decisions, such as the order of discount and tax, that may be wrong.

How to fix it: Write the specification first, including rules and edge cases.

Mistake 2: Generating Large Features at Once What it is: Asking for an entire module or application in one prompt.

Why it happens: It seems efficient.

What it causes: Unreviewable code, truncated output, and hidden design decisions.

How to fix it: Plan first, then generate one unit at a time.

Mistake 3: Letting the Model Change the Tests What it is: Accepting fixes that modify expected test values.

Why it happens: The test suite passes afterwards.

What it causes: Incorrect code with tests that confirm the error.

How to fix it: Treat tests as the specification; change them only by your own decision.

Mistake 4: Running Generated Code Unsafely What it is: Executing generated code directly in your development environment or server.

Why it happens: It is convenient.

What it causes: Risk of harmful operations, by mistake or through manipulated input.

How to fix it: Use static checks first, and run code in an isolated environment.

Mistake 5: Merging Code You Do Not Understand What it is: Merging because tests pass.

Why it happens: Time pressure.

What it causes: Code no one on the team can maintain or debug.

How to fix it: Review and understand every line before merging.

Best Practices Specify before generating. Inputs, outputs, rules, edge cases, errors, constraints.

Provide focused context. Versions, conventions, and the specific code the new code interacts with.

Define interfaces first. Keep generated code compatible with the project.

Write and verify tests first. Tests define correctness independently of the model.

Generate small units. Plan larger features before implementing them.

Verify in layers. Static checks, isolated test runs, linters, and type checkers.

Iterate with specific failures and limited attempts. Never weaken the tests.

Review and own the result. Understand, secure, and maintain it like your own code.

Try It Yourself Exercise 1: Specification and Tests First Write a specification for a function days_until_deadline(today: date, deadline: date) -> int that returns the number of days remaining, or 0 if the deadline has passed. Write four tests, including the deadline being today and being in the past, before generating anything.

Exercise 2: Generate and Verify Use the generation prompt from Step 5 with your specification, interface, and tests. Run check_generated_code() on the result, then run the tests in an isolated environment. Record every failure and how many iterations were needed.

Exercise 3: Tune the Static Checker Adapt FORBIDDEN_MODULES and FORBIDDEN_CALLS for a Django view module in your project. Add a check that reports any function longer than 40 lines, using each function node's lineno and end_lineno attributes. Test the checker on three pieces of generated code.