Beyond Images: Giving the Model a Whole Document

Lesson 1 covered images as visual input. This lesson covers a related but distinct capability: giving the model a whole file — most commonly a PDF — as input, letting it read and reason about a document's full content, including both its text and, for a PDF specifically, its visual layout (tables, headers, embedded images) in a way plain extracted text often loses. This is directly useful for tasks like summarizing a long report, answering questions about a contract, or extracting structured data from an invoice or form delivered as a PDF rather than plain text.

Two Ways to Supply a File

Similar to the URL-versus-base64 choice Lesson 1 covered for images, a file can be supplied to a request in two ways: uploaded ahead of time through the dedicated Files API and referenced by its resulting file ID, or included directly in a request as base64-encoded data. Each has a different appropriate use case, covered in turn.

Uploading a File Through the Files API

For a file that will be referenced more than once, or that's large enough that repeatedly base64-encoding and resending it in every request would be wasteful, uploading it once through the Files API and referencing it by ID is the more efficient approach.

with open("quarterly_report.pdf", "rb") as f:
    uploaded_file = client.files.create(file=f, purpose="user_data")

print(f"Uploaded file ID: {uploaded_file.id}")

response = client.responses.create(
    model="gpt-5.6-luna",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Summarize the key financial figures in this report."},
                {"type": "input_file", "file_id": uploaded_file.id},
            ],
        }
    ],
)
print(response.output_text)

The purpose="user_data" argument tells the platform how the uploaded file is intended to be used (distinct from other purposes a Files API might support, such as fine-tuning data or batch job inputs — confirm exact supported purpose values against your SDK version's documentation). Once uploaded, uploaded_file.id is a stable reference you can use across multiple requests without re-uploading the file's bytes each time — directly useful for a feature that asks several different questions about the same document in sequence, or that revisits a previously uploaded document in a later session.

Supplying a File Inline as Base64

For a one-off request against a file that doesn't need to persist or be referenced again, a file can be included directly in the request as base64-encoded data, exactly paralleling Lesson 1's base64 image approach.

import base64

def ask_about_pdf_inline(pdf_path: str, question: str) -> str:
    with open(pdf_path, "rb") as f:
        pdf_bytes = f.read()
    base64_pdf = base64.b64encode(pdf_bytes).decode("utf-8")

    response = client.responses.create(
        model="gpt-5.6-luna",
        input=[
            {
                "role": "user",
                "content": [
                    {"type": "input_text", "text": question},
                    {
                        "type": "input_file",
                        "filename": pdf_path.split("/")[-1],
                        "file_data": f"data:application/pdf;base64,{base64_pdf}",
                    },
                ],
            }
        ],
    )
    return response.output_text

print(ask_about_pdf_inline("contract.pdf", "What is the termination notice period specified in this contract?"))

Notice the filename field alongside file_data — supplying a filename gives the model useful context about the document (its name often hints at its purpose or type) even when the content itself doesn't state it explicitly, and some platform behaviors may use the filename for format detection or logging purposes as well.

When to Upload vs. When to Inline

For an application that lets a user upload a document once and then ask several follow-up questions about it — a document Q&A feature, which this unit's project builds directly — the Files API upload approach is clearly the better fit: the file is uploaded exactly once, and every subsequent question references it by ID rather than resending the entire document's bytes on each call.

Deleting Files You No Longer Need

Files uploaded through the Files API persist on the platform until explicitly deleted (subject to the platform's own retention policies, which are worth confirming against current documentation) — meaning an application that uploads files on behalf of users should also clean them up when they're no longer needed, both for storage hygiene and, in many cases, for privacy and data-retention reasons.

def cleanup_file(file_id: str) -> None:
    try:
        client.files.delete(file_id)
        print(f"Deleted file {file_id}")
    except Exception as e:
        print(f"Failed to delete file {file_id}: {e}")

For an application processing files on behalf of users — especially anything containing potentially sensitive information, like the contracts and financial reports used as examples throughout this lesson — building an explicit deletion step into the application's lifecycle (after a session ends, after a defined retention period, immediately after a one-off analysis completes) is a meaningful privacy and data-hygiene practice worth treating as a first-class part of the feature's design, not an afterthought.

Reading PDFs vs. Extracting Text Yourself

A natural question: why not just extract the PDF's text with a Python library (pypdf, pdfplumber, and similar) and send that extracted text as a plain string, rather than sending the whole file? Both approaches work, but they have real trade-offs worth understanding.

# Approach A: extract text yourself, send as plain text
import pypdf

def extract_pdf_text(path: str) -> str:
    reader = pypdf.PdfReader(path)
    return "\n".join(page.extract_text() for page in reader.pages)

text = extract_pdf_text("report.pdf")
response = client.responses.create(model="gpt-5.6-luna", input=f"Summarize this report:\n\n{text}")

# Approach B: send the PDF itself, let the model handle extraction
with open("report.pdf", "rb") as f:
    uploaded = client.files.create(file=f, purpose="user_data")
response = client.responses.create(
    model="gpt-5.6-luna",
    input=[{"role": "user", "content": [
        {"type": "input_text", "text": "Summarize this report."},
        {"type": "input_file", "file_id": uploaded.id},
    ]}],
)

Approach A gives you full control over the extracted text (useful if you need to preprocess, chunk, or filter it before sending) but loses layout and visual information entirely — a table's row/column structure typically collapses into a confusing jumble of text when extracted naively, and any content that exists only as an image within the PDF (a scanned page, a chart, a diagram) is lost completely. Approach B lets the model's own document understanding handle layout, tables, and embedded visual content directly, generally producing better results for visually structured documents, at the cost of less direct control over exactly what content reaches the model and how it's chunked.

Note: Exactly how a PDF's layout, tables, and embedded images are interpreted when sent via input_file is a model- and platform-version-specific capability. For documents where table structure or embedded visual content matters significantly to the task, test both approaches against representative real documents before committing to one, since actual performance can vary meaningfully by document type and by the specific version of the model in use.

Handling Multi-Page and Very Large Documents

A long document — a hundred-page report, a lengthy legal contract — raises the same context-window considerations Unit 4 covered for conversation history: a document that's large enough can exceed what a single request can process at once, or can consume enough of the available context that little room remains for a useful answer.

def check_document_size_concern(file_path: str, rough_tokens_per_page: int = 500) -> str:
    """A rough heuristic for flagging documents likely to strain context limits —
    exact context window sizes are model- and version-specific (Unit 4, Lesson 6)."""
    import pypdf
    page_count = len(pypdf.PdfReader(file_path).pages)
    estimated_tokens = page_count * rough_tokens_per_page
    if estimated_tokens > 50_000:
        return f"~{page_count} pages, ~{estimated_tokens} estimated tokens — consider chunking or a targeted-question approach"
    return f"~{page_count} pages, ~{estimated_tokens} estimated tokens — likely fine as a single request"

For a genuinely very large document, the same strategies Unit 4, Lesson 6 covered for compacting conversation history apply conceptually here too: splitting the document into sections and processing them separately, using a targeted retrieval approach to find and send only the relevant pages for a specific question (a preview of the retrieval concepts Unit 10 covers in depth), or summarizing sections progressively rather than attempting to process an entire enormous document in one request.

Combining File Input With Structured Outputs

Exactly as with images in Lesson 1, file input combines directly with Unit 6's structured-output mechanism, letting an application extract a validated, typed record from a document rather than a free-text summary.

from pydantic import BaseModel

class ContractSummary(BaseModel):
    parties: list[str]
    effective_date: str
    termination_notice_days: int | None

response = client.responses.parse(
    model="gpt-5.6-luna",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Extract the parties, effective date, and termination notice period from this contract."},
                {"type": "input_file", "file_id": uploaded_file.id},
            ],
        }
    ],
    text_format=ContractSummary,
)
summary = response.output_parsed
print(f"Parties: {summary.parties}, effective {summary.effective_date}")

This is directly the pattern this unit's Lesson 5 project builds on: a document, uploaded once, queried with a schema-constrained request that returns a reliably typed, structured result — combining this lesson's file-handling mechanics with Unit 6's structured-output guarantees and Lesson 4's earlier point about applying appropriate validation to anything a model extracts, whether from text, an image, or a full document.

Testing File-Handling Code Without Real Files or API Calls

Following this course's dependency-injection testing pattern, the logic around building a file-input request and handling its response can be tested with a fake uploaded-file object and a fake client, without needing real PDF files or live API calls for every test run.

class FakeUploadedFile:
    def __init__(self, file_id: str):
        self.id = file_id

def build_file_question_input(file_id: str, question: str) -> list:
    return [
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": question},
                {"type": "input_file", "file_id": file_id},
            ],
        }
    ]

def test_build_file_question_input():
    fake_file = FakeUploadedFile("file_fake_123")
    result = build_file_question_input(fake_file.id, "Summarize this.")
    assert result[0]["content"][1]["file_id"] == "file_fake_123"
    print("PASS: build_file_question_input references the correct file ID")

test_build_file_question_input()

As with the image-handling tests in Lesson 1, this kind of structure-level test catches the common, easy mistakes in hand-built request dictionaries — a misplaced key, a wrong field name — quickly and for free, well before a live API call would surface the same mistake as a more confusing runtime error.

Listing and Retrieving Uploaded Files

Beyond uploading and deleting, the Files API typically supports listing files your application has uploaded and retrieving metadata about a specific one — useful for building an administrative view of what's currently stored, or for confirming a file still exists before referencing it in a new request.

def list_uploaded_files() -> list:
    files = client.files.list()
    for f in files.data:
        print(f"{f.id}: {f.filename}, {f.bytes} bytes, created {f.created_at}")
    return files.data

def get_file_info(file_id: str) -> dict:
    try:
        info = client.files.retrieve(file_id)
        return {"exists": True, "filename": info.filename, "bytes": info.bytes}
    except Exception:
        return {"exists": False}

get_file_info()'s pattern — attempting a retrieval and catching the failure as evidence the file no longer exists — is a practical way to guard against referencing a stale file ID a user's session might have stored from an earlier interaction, particularly relevant for a long-running application where a file could have been deleted (by a cleanup job, by the platform's own retention policy) between when its ID was first stored and when it's used again.

Multiple Questions Against the Same Uploaded Document

The efficiency case for uploading once and referencing by ID becomes concrete once a feature asks several distinct questions against the same document, each as its own separate request rather than needing to resend the file.

def ask_multiple_questions(file_id: str, questions: list[str]) -> dict[str, str]:
    answers = {}
    for question in questions:
        response = client.responses.create(
            model="gpt-5.6-luna",
            input=[{"role": "user", "content": [
                {"type": "input_text", "text": question},
                {"type": "input_file", "file_id": file_id},
            ]}],
        )
        answers[question] = response.output_text
    return answers

with open("contract.pdf", "rb") as f:
    uploaded = client.files.create(file=f, purpose="user_data")

results = ask_multiple_questions(uploaded.id, [
    "Who are the parties to this contract?",
    "What is the effective date?",
    "What is the termination notice period?",
])
for q, a in results.items():
    print(f"Q: {q}\nA: {a}\n")

Each of these three requests references the same uploaded.id rather than re-uploading the contract three times — the file's content is fetched by the platform from its stored upload each time, at a fraction of the bandwidth cost of resending the full document bytes with every question, which is precisely the efficiency case the Files API upload path exists to serve.

Combining a File With an Image in the Same Request

Since input_file and input_image are both just entries in the same content list, a single request can combine a document and an image together — useful, for instance, for a task that needs to cross-reference a written policy document against a photo of a physical situation.

response = client.responses.create(
    model="gpt-5.6-luna",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "Does the damage shown in this photo fall under the coverage described in this policy document?"},
                {"type": "input_file", "file_id": policy_file_id},
                {"type": "input_image", "image_url": "https://example.com/damage_photo.jpg"},
            ],
        }
    ],
)
print(response.output_text)

This kind of combined request — a reference document plus a visual situation to evaluate against it — is a natural fit for insurance claim review, compliance checking against a photographed scene, or any workflow where a written rule needs to be applied to a specific visual instance, and it costs no more conceptual complexity than either input type used alone, since both are simply items in the same content list processed together by the model.

Common Mistakes

Uploading a file fresh on every request when it will be queried multiple times, wasting bandwidth and upload time repeatedly re-sending identical file bytes instead of uploading once and referencing the resulting file ID across all subsequent requests.

Extracting PDF text yourself by default, without considering that layout and visual content (tables, charts, scanned pages) are lost in the process — for visually structured documents, sending the file directly and letting the model's own document understanding handle it often produces meaningfully better results.

Never deleting uploaded files, letting them accumulate indefinitely on the platform — a hygiene and, for sensitive documents, a genuine privacy concern worth addressing with an explicit cleanup step in the application's lifecycle.

Sending an extremely long document in a single request without considering context-window limits, and being surprised when the model's answer seems to miss content from earlier or later sections of a very long file.

Best Practices

Upload a file once through the Files API and reference it by ID for any feature that asks multiple questions against the same document, rather than re-encoding and resending it with every request.

Prefer sending the file itself over pre-extracted text for documents where layout, tables, or embedded visual content matter to the task, and prefer your own text extraction when you need fine control over preprocessing, chunking, or filtering before the model sees the content.

Build an explicit file-deletion step into any feature that uploads user documents, treating cleanup as a first-class part of the feature rather than an afterthought, especially for anything containing potentially sensitive information.

Flag or chunk unusually large documents rather than assuming every document fits comfortably within a single request's context, applying the same context-management thinking Unit 4 established for long conversation histories.

Combine file input with structured outputs for any extraction task against a document, applying the same schema-and-validation discipline Unit 6 established for text and Lesson 1 established for images, rather than parsing a free-text summary of the document's content.