The Problem File Search Solves

A language model only knows what was in its training data, plus whatever text you put directly into its context window. It cannot read your company's internal PDFs, your product documentation, your support tickets, or last week's meeting notes unless you hand that text to it in the prompt. That is a hard limit, not a configuration option — the model has no filesystem, no database connection, and no memory of documents it has not seen.

Retrieval-Augmented Generation (RAG) is the general technique for working around this limit. Instead of asking the model to answer purely from what it learned during training, you first retrieve the small number of document passages that are actually relevant to the user's question, insert those passages into the prompt, and then ask the model to answer using that inserted text as evidence. The model still does the reasoning and the writing, but the facts come from your documents, not from its parameters.

file_search is OpenAI's hosted implementation of this pattern, exposed as a built-in tool for the Responses API. You already saw it briefly in Unit 9, Lesson 2, where it was introduced alongside other built-in tools like web search and code interpreter. This unit treats it as its own subject: how the retrieval actually works, how to organize documents at scale, how to control what gets retrieved, and how to build production systems on top of it.

Why Not Just Paste the Document Into the Prompt?

For a single short document, you don't need file search at all — just read the file and put its text in the prompt. The problem appears at scale. If your knowledge base is 500 PDFs totaling 40,000 pages, none of that fits in any model's context window, and even if it did, you would be paying to re-process all of it on every single request, and the model's attention would be diluted across mostly irrelevant text.

RAG solves this by separating two concerns that are easy to conflate:

1. Search — finding the small subset of content relevant to this specific question, out of a much larger corpus.
2. Generation — using that small subset to produce a well-written, grounded answer.

file_search handles the first concern for you. OpenAI stores your documents in a vector store (covered in depth in Lesson 2), automatically chunks them into passages, computes embeddings for each chunk, and at query time finds the chunks whose embeddings are most similar to the embedding of the user's question. Those chunks are injected into the model's context automatically, and the model generates its answer from them.

How Retrieval Actually Works, Step by Step

Understanding the mechanism matters because it explains almost every design decision you'll make in later lessons — chunk size, metadata, and evaluation all follow from this pipeline.

1. Ingestion. You upload a document to a vector store. OpenAI splits it into chunks (roughly paragraph-to-page sized pieces) and computes an embedding vector for each chunk using an embedding model.
2. Indexing. Each chunk's embedding is stored alongside the chunk's text and any metadata you attached, in a structure optimized for fast similarity search (technically an approximate nearest-neighbor index, not a linear scan).
3. Query time. When a user asks a question, the Responses API computes an embedding for the query itself.
4. Similarity search. The system compares the query embedding against the stored chunk embeddings using a similarity metric (cosine similarity is the standard here) and retrieves the top-K most similar chunks.
5. Augmentation. The retrieved chunk text is inserted into the model's context, typically with source annotations.
6. Generation. The model reads the user's question plus the retrieved chunks and writes an answer, ideally citing which chunks it used.

The key insight is that steps 1–4 are a search problem, not a reasoning problem. Embeddings capture semantic meaning, so a query like "how do I cancel my subscription" can retrieve a chunk titled "Ending your plan" even though the words don't overlap — that's the entire value proposition of embedding-based retrieval over old-fashioned keyword search.

A Minimal File Search Example

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-5.6-terra",
    input="What is our refund policy for annual subscriptions?",
    tools=[
        {
            "type": "file_search",
            "vector_store_ids": ["vs_68f2a1c9e4b8..."],
        }
    ],
)

print(response.output_text)

Note: The exact tool configuration keys (vector_store_ids, how many stores can be attached at once, and any additional optional fields) can change between API versions. Confirm the current file_search tool schema against the official OpenAI API reference before deploying this in production.

What happens here: the tools list tells the model it has access to the file_search capability, scoped to one specific vector store by ID. When the model receives the input question, the Responses API automatically performs the retrieval steps described above — you never manually call an embedding endpoint or write similarity search code yourself. The model's response text will typically be grounded in whatever chunks were retrieved from that vector store, and the raw response object also contains structured information about which chunks were used (covered in Lesson 4).

This is the central trade-off of file_search versus the custom pipeline you'll build in Unit 10: you give up fine-grained control over the embedding model, chunking algorithm, and similarity search implementation, in exchange for not having to build or operate any of that infrastructure yourself. Unit 10, Lesson 4 compares these two approaches directly once you've seen both in detail.

When to Use File Search

file_search is the right tool when:

• You have a body of reference documents (policies, manuals, contracts, product docs) that changes occasionally but isn't rewritten on every request.
• You want grounded, citation-backed answers rather than answers purely from model memory.
• You don't want to operate your own embedding pipeline, vector database, or chunking logic.
• Your documents are the kind of static or slowly-changing text that benefits from being pre-indexed once and queried many times.

When Not to Use File Search

File search is not a universal answer to "the model doesn't know something."

• Real-time or frequently changing information (stock prices, current weather, breaking news) is a poor fit — a vector store is a snapshot, not a live feed. That's what web search (Unit 15) is for, and Lesson 9 of this unit shows how to combine the two.
• Structured lookups ("what is order #48213's shipping status") are better served by a direct database or API call than by embedding-based retrieval — embeddings are for semantic similarity, not exact key lookups.
• Small, fixed context that always fits comfortably in the prompt (a single short FAQ) doesn't need the overhead of a vector store at all; just include the text directly.
• Highly precise numerical or tabular data (financial statements, dense spreadsheets) often retrieves poorly because chunking can split a table awkwardly, losing the row/column relationships that give the numbers meaning. You'll see mitigation strategies for this in Lesson 7.

RAG vs. Fine-Tuning vs. Long Context

These three techniques are commonly confused because they all address "the model doesn't know X."

A common mistake is trying to fine-tune a model to "know" a knowledge base. Fine-tuning adjusts how the model behaves and writes, but it is not a reliable way to store retrievable facts — the model can hallucinate details even after being fine-tuned on the source material, because fine-tuning does not guarantee memorization or the ability to cite sources. RAG remains the correct tool for "answer questions using this specific set of documents."

Common Mistakes

Treating file search as a database, expecting it to return exact records by ID or field, which fails because retrieval is similarity-based, not key-based — use metadata filtering (Lesson 5) or a real database for exact lookups instead.

Assuming retrieval is free of latency and cost, which causes surprise when a file_search-enabled request takes noticeably longer or costs more than a plain text request — retrieval happens synchronously as part of the response, so factor it into your latency budget from the start.

Skipping evaluation of retrieval quality, shipping a file-search-backed assistant without ever checking whether the retrieved chunks actually answer the test questions — this hides quality problems that surface later as vague or wrong answers, discussed further in Lesson 7 and Lesson 8.

Best Practices

Keep the corpus scoped and relevant. A vector store containing only your product's documentation retrieves more precisely than one that also contains unrelated internal wikis — irrelevant content increases the chance an unrelated chunk wins the similarity search.

Log which chunks were retrieved for each response, not just the final answer, so you can debug bad answers by checking whether the retrieval step or the generation step failed.

Start with the simplest configuration before optimizing. Get a single vector store with default chunking working end-to-end first, then tune chunk size, metadata, and filtering once you have real queries to evaluate against.