Beyond Images: What Else the Sandbox Can Produce

Lesson 5 focused specifically on chart images because they are the most common generated artifact, but the sandbox's filesystem is general-purpose — code running inside it can write any kind of file: a cleaned CSV, a multi-sheet Excel workbook, a JSON export, a PDF report, or a zip archive bundling several outputs together. Any of these can be requested and retrieved through the same underlying mechanism, and building a data-analysis feature that only ever expects images will miss a large and useful category of results: "clean this data and give me back a file I can download" is one of the most practical things this tool does.

This lesson builds a general-purpose extraction pattern that works for any file type the sandbox produces, not just images, and covers the citation mechanism the model uses to reference generated files directly in its text output.

Requesting a Non-Image Artifact

from openai import OpenAI

client = OpenAI()

uploaded = client.files.create(file=open("raw_survey_responses.csv", "rb"), purpose="assistants")

response = client.responses.create(
    model="gpt-5.6-terra",
    tools=[{
        "type": "code_interpreter",
        "container": {"type": "auto", "file_ids": [uploaded.id]},
    }],
    input=(
        "raw_survey_responses.csv has inconsistent capitalization in the "
        "'country' column and some duplicate rows based on respondent_id. "
        "Clean it: normalize country names to title case, drop duplicate "
        "respondent_id rows keeping the first occurrence, and save the "
        "result as cleaned_survey_responses.csv."
    ),
)

print(response.output_text)

This is a common real-world shape for this feature: the input is messy, the transformation is well-defined, and the useful output isn't a number or a chart — it's a corrected file the user can download and use elsewhere (import into a spreadsheet, load into another system). The prompt states the exact cleaning rules rather than leaving "clean this data" open to interpretation, for the same reason discussed in Lesson 4: specific instructions produce specific, reviewable code.

A General-Purpose File Extraction Helper

Rather than writing bespoke extraction code for each file type, a single function that walks the response and downloads every generated file, regardless of type, is more maintainable:

def extract_generated_files(response, client) -> list[dict]:
    """Return a list of {filename, file_id, container_id, content} for every
    file the code interpreter produced in this response."""
    artifacts = []
    for item in response.output:
        if item.type != "code_interpreter_call":
            continue
        for result in item.outputs or []:
            if result.type in ("image", "file"):
                content = client.containers.files.content(
                    container_id=item.container_id,
                    file_id=result.file_id,
                )
                artifacts.append({
                    "filename": getattr(result, "filename", result.file_id),
                    "file_id": result.file_id,
                    "container_id": item.container_id,
                    "content": content.read(),
                })
    return artifacts

This function treats "image" and "file" result types uniformly, since both are ultimately bytes retrievable through the same container-scoped download call — the only difference that matters to calling code is what you do with the bytes afterward (display it, offer it for download, parse it further). It also defensively falls back to the file ID as a filename with getattr(result, "filename", result.file_id), since not every generated-file result is guaranteed to carry a human-readable filename.

Saving the results to disk, or handing them to a web response, follows naturally:

artifacts = extract_generated_files(response, client)
for artifact in artifacts:
    with open(artifact["filename"], "wb") as f:
        f.write(artifact["content"])
    print(f"Saved {artifact['filename']} ({len(artifact['content'])} bytes)")

Note: The discriminator values for generated-file result types ("image", "file") and the exact fields available on each (file_id, filename, container_id) are specific to the current API version. Confirm the current output schema against official documentation before relying on these exact names in production.

File Citations in the Response Text

When the model produces a file and then refers to it in its natural-language answer ("I've saved the cleaned data to cleaned_survey_responses.csv"), that reference is often backed by a structured citation embedded in the message content, not just a plain-text filename. This citation links a specific span of the response text to a specific generated file, which is useful when you want to render the answer with an inline, clickable download link rather than just a plain filename mentioned in prose.

for item in response.output:
    if item.type != "message":
        continue
    for content_block in item.content:
        annotations = getattr(content_block, "annotations", None) or []
        for annotation in annotations:
            if annotation.type == "container_file_citation":
                print("Referenced file:", annotation.filename)
                print("File ID:", annotation.file_id)

Walking the message item's content blocks and their annotations surfaces these citations. In a user-facing application, this is what you would use to turn "...saved to cleaned_survey_responses.csv" in the rendered text into an actual download link pointing at the file you already extracted with extract_generated_files, rather than relying on string-matching the filename out of the prose yourself, which is fragile.

Note: The annotation type name (container_file_citation) and its exact fields are version-specific details. Verify them against current documentation before depending on this exact string in production parsing logic.

Building a Complete Retrieval Function

Putting the pieces together, a realistic helper for a web backend might look like this:

def run_analysis_and_collect_artifacts(client, file_ids: list[str], question: str) -> dict:
    response = client.responses.create(
        model="gpt-5.6-terra",
        tools=[{
            "type": "code_interpreter",
            "container": {"type": "auto", "file_ids": file_ids},
        }],
        input=question,
    )
    return {
        "answer_text": response.output_text,
        "artifacts": extract_generated_files(response, client),
    }


def test_run_analysis_and_collect_artifacts():
    class FakeContentBlock:
        def read(self):
            return b"fake file bytes"

    class FakeOutputItem:
        type = "code_interpreter_call"
        container_id = "cnt_123"

        class Result:
            type = "file"
            file_id = "file_abc"
            filename = "result.csv"

        outputs = [Result()]

    class FakeResponse:
        output = [FakeOutputItem()]
        output_text = "Here is your cleaned file."

    class FakeContainerFiles:
        def content(self, container_id, file_id):
            assert container_id == "cnt_123"
            assert file_id == "file_abc"
            return FakeContentBlock()

    class FakeContainers:
        files = FakeContainerFiles()

    class FakeClient:
        containers = FakeContainers()

    response = FakeResponse()
    artifacts = extract_generated_files(response, FakeClient())

    assert len(artifacts) == 1
    assert artifacts[0]["filename"] == "result.csv"
    assert artifacts[0]["content"] == b"fake file bytes"
    print("PASS: run_analysis_and_collect_artifacts extracts a generated file correctly")


test_run_analysis_and_collect_artifacts()

The test constructs a chain of fake objects (FakeClient → FakeContainers → FakeContainerFiles) that mirror the exact attribute path extract_generated_files walks (client.containers.files.content(...)), plus a FakeResponse shaped like a real one. This lets the extraction logic be verified without a real API key, a real network call, or any nondeterminism from an actual model run — exactly the dependency-injection testing pattern used throughout this course. No live API call appears inside the test.

Storage and Expiration

Files generated inside a container are only retrievable while that container (and the platform's record of it) remains valid. Unlike files you explicitly upload through client.files.create, which persist until you delete them, generated artifacts have a more limited effective lifetime tied to the container and response. If your application needs to offer a "download this report" link days after the analysis ran, download and store the bytes yourself — in your own object storage, a database blob column, or a file system you control — at the time the response comes back, rather than trying to re-fetch it from OpenAI later.

Common Mistakes

Treating generated files as permanently retrievable from OpenAI's storage. They are not designed as long-term storage; persist anything a user needs later in your own infrastructure immediately after extraction.

Only handling the "image" result type and silently dropping other generated files. A cleaning or transformation request that produces a CSV or Excel file will populate a "file"-type result, not an "image"-type one — code that only checks for images will appear to work during chart-focused testing and then quietly fail to surface a generated spreadsheet.

Parsing the filename out of the response's prose text with string matching. This is fragile against phrasing changes in the model's natural-language answer. Use the structured outputs list and citation annotations instead, both of which are designed for exactly this purpose.

Best Practices

Build one general-purpose extraction function that handles every generated-file type, as shown above, rather than writing separate ad hoc code for images versus other file types in different parts of your application.

Persist generated artifacts to your own storage immediately, at the same point in your code where you extract them, rather than deferring that to a later request that assumes the container is still reachable.

Surface file citations to render inline download links when displaying the model's answer to end users, rather than showing only plain-text filenames with no way to actually retrieve the file being referenced.