Separating System Instructions From User Data

Unit 3 covered writing an individual prompt well: how to phrase instructions clearly, how to structure input so the model understands what is being asked, and how to iterate on wording until responses became consistent. This unit treats prompts as software artifacts that need to be organized, reused, versioned, and tested across a real application, not just written well once. The starting point for that is a distinction that matters far more once a prompt lives inside a codebase than it does in a one-off playground experiment: the separation between system instructions and user data.

What the Separation Actually Is

Every request to the Responses API (or the Chat Completions API) is built from discrete pieces. The SDK lets you assign each piece a role — typically system (sometimes called developer in newer API versions), user, and assistant. In application code, this maps onto two categories that matter for engineering purposes:

• System instructions: text that your application controls. It describes the assistant's role, the rules it must follow, the output format it must produce, and any constraints on behavior. This text is the same (or changes only through a deliberate deploy) regardless of who is using the application or what they typed.
• User data: text that originates outside your application's control — a customer's support message, the contents of an uploaded document, a search query, a form field. This text changes on every request and is, from the application's point of view, untrusted.

The distinction is not about how the model chooses to weigh the two categories internally — the model is trained to give system-role text more authority, but that is a probabilistic tendency, not a hard guarantee. The distinction matters because of what it does to your code: it gives you one place to change behavior (the system instructions) that is independent of the data flowing through the application, and it gives you a clear boundary for where trusted, developer-authored text ends and untrusted, user-supplied text begins.

Why This Matters for Application Code

Consider what happens without this separation. A common but fragile pattern is to build a single string by concatenating instructions and user input:

# Fragile: instructions and data are the same string
prompt = f"""You are a support assistant. Answer questions only about
our billing policy. If asked about anything else, decline politely.

Customer question: {user_message}
"""

This works while user_message is well-behaved. The problem is architectural, not just a security concern (Unit 23 covers prompt injection and related risks in depth — this lesson only needs the engineering consequence, not the attack theory). When instructions and data are fused into one string:

• You cannot swap the instructions independently of the data path. Every place that builds this string has to be found and edited together.
• You cannot log, test, or version the instructions separately from a specific request, because there is no separately addressable "instructions" object — just a string that happened to have some data spliced into it.
• You cannot easily reuse the same instructions across multiple call sites (a CLI tool, a web handler, a background job) without recopying the f-string template into each one.

Using the API's role structure fixes all three, because it gives you two independent values instead of one merged string:

from openai import OpenAI

client = OpenAI()

SYSTEM_INSTRUCTIONS = """You are a support assistant. Answer questions only
about our billing policy. If asked about anything else, decline politely
and suggest the customer contact general support."""

def answer_billing_question(user_message: str) -> str:
    response = client.responses.create(
        model="gpt-5.6-terra",
        instructions=SYSTEM_INSTRUCTIONS,
        input=user_message,
    )
    return response.output_text

SYSTEM_INSTRUCTIONS is now a module-level constant: a single, addressable object. It can be imported, unit tested for its wording, diffed in a pull request, and swapped for SYSTEM_INSTRUCTIONS_V2 without touching the function that handles user input. user_message remains a plain function argument — nothing about it needs to change no matter how the instructions evolve.

How the SDK Represents This

The Responses API exposes this separation through the instructions parameter alongside input. Under the hood, instructions is sent as a message with a system/developer role, placed ahead of the conversation content. When you need more than a single instructions string — for example, a multi-turn conversation — the same separation continues through the role field on each message:

response = client.responses.create(
    model="gpt-5.6-terra",
    instructions=SYSTEM_INSTRUCTIONS,
    input=[
        {"role": "user", "content": "Why was I charged twice this month?"},
        {"role": "assistant", "content": "Let me look into that. Can you share the invoice date?"},
        {"role": "user", "content": "It was on the 14th."},
    ],
)

Every dictionary in the input list has an explicit role. This is the same principle applied across a whole conversation: the application knows, structurally, which turns are the assistant's own prior output, which are user-supplied, and — via the top-level instructions argument — which text is the developer's standing policy. None of this depends on string formatting conventions inside a single blob of text.

Note: Some SDK versions and some older Chat Completions-style code represent the system message as the first element of a messages list with "role": "system" instead of a separate instructions parameter. The role-based separation is the same concept; only the surface API shape differs. Check the SDK version your project pins before assuming one shape.

A Practical Pattern: Wrapping the Boundary in a Function

Because the separation is a codebase concern, not just an API-call concern, it pays to enforce it at a function boundary rather than trusting every call site to remember it:

from dataclasses import dataclass
from openai import OpenAI

client = OpenAI()

@dataclass(frozen=True)
class PromptRequest:
    instructions: str
    user_input: str

def run_prompt(request: PromptRequest, model: str = "gpt-5.6-terra") -> str:
    response = client.responses.create(
        model=model,
        instructions=request.instructions,
        input=request.user_input,
    )
    return response.output_text

The PromptRequest dataclass makes the separation explicit in the type system: it is impossible to construct a request without deciding what belongs in instructions and what belongs in user_input. This is a small amount of structure, but it prevents the most common regression — a future contributor who, under time pressure, string-formats a user value directly into what should have been the fixed instructions text.

When Some Blending Is Legitimate

The separation does not mean user data can never appear near instructions. Two common, legitimate cases:

1. Few-shot examples derived from real data (covered in Lesson 4) are still developer-authored — you selected and curated them — even though their content looks like user data. They belong in the instructions or in a clearly labeled examples section, not mixed anonymously into live user input.
2. Structured user context passed as data, not as instructions. For example, passing a customer's subscription tier so the model can tailor its answer:

def answer_with_context(user_message: str, subscription_tier: str) -> str:
    instructions = (
        "You are a support assistant. Use the customer's subscription "
        "tier only to determine which features to mention as available."
    )
    input_payload = (
        f"Customer subscription tier: {subscription_tier}\n"
        f"Customer question: {user_message}"
    )
    response = client.responses.create(
        model="gpt-5.6-terra",
        instructions=instructions,
        input=input_payload,
    )
    return response.output_text

Here, subscription_tier and user_message are both data — they both vary per request — but they are still kept out of the instructions string, which remains fixed policy text. The rule is not "nothing but the literal user message may go in input"; it is "policy that your application decides does not belong in the same variable as content that originates from outside your application."

This separation is also a security boundary, not just an organizational one — instructions in the system role carry more weight against attempts by user-supplied content to override them, which is why prompt injection defenses (covered fully in Unit 23) start from exactly this same instructions/input split. For this unit's purposes, the practical payoff is simpler: a codebase where instructions are addressable, testable, and swappable independently of the data flowing through it.

Common Mistakes

Concatenating user input into the instructions string. Writing instructions = f"You are a support bot. The user said: {user_message}" collapses the separation entirely — user_message is now inside what should be a fixed, versionable string, and every future change to instructions risks bugs where a stray { or format specifier in user text breaks the format call.

Treating role assignment as optional in multi-turn code. Some early prototypes pass a single concatenated transcript string as input instead of a list of role-tagged messages. This works for simple cases but throws away the model's ability to distinguish its own prior turns from the user's turns, which degrades multi-turn coherence as conversations get longer.

Assuming the separation is a security guarantee by itself. Placing text in the system role makes it harder, not impossible, for adversarial user input to influence behavior. Do not treat this lesson's separation as sufficient defense against prompt injection — that requires the additional measures in Unit 23.

Best Practices

Give instructions a single source of truth. Define each set of system instructions as a named constant, function, or template (Lesson 3) in one place, and import it everywhere it is used, rather than retyping similar wording at each call site.

Keep user-supplied values out of the instructions parameter. Even convenience data like a username or tier should go into input as structured context, not be spliced into the instructions string, so instructions remain a fixed, diffable artifact.

Use explicit roles for every turn in multi-turn input. Always build input as a list of {"role": ..., "content": ...} dictionaries once a conversation has more than one turn, instead of collapsing history into a single string.