Combining File Search With Web Search

Ma Mahalakshmi V Updated 16 Sep 2026
7 min read ·Lesson 84 of 224

Why These Two Tools Complement Each Other

File search retrieves from a fixed, curated corpus you control — great for stable, private, or authoritative content like internal policies and documentation, but blind to anything published after your last upload or outside your document set. Web search, covered in Unit 15, does the opposite: it reaches current public information but has no awareness of your private, internal, or proprietary content. Neither tool is a superset of the other, and many real questions genuinely need both — "how does our refund policy compare to what's now legally required in the EU" needs your internal policy document and current external regulatory information.

Attaching both tools to a single request lets the model choose which one (or both) to invoke based on the question, without you having to pre-classify every incoming query yourself.

Attaching Both Tools to One Request

from openai import OpenAI

client = OpenAI()

response = client.responses.create(
    model="gpt-5.6-terra",
    instructions=(
        "You are a research assistant with access to internal documents "
        "(via file search) and current public information (via web search). "
        "Use file search for questions about internal policy, product "
        "documentation, or company-specific information. Use web search for "
        "current events, external regulations, or publicly available facts "
        "not covered by internal documents. State which source informed "
        "each part of your answer."
    ),
    input="Does our current data retention policy meet the latest GDPR requirements?",
    tools=[
        {"type": "file_search", "vector_store_ids": ["vs_legal_policies_68f2"]},
        {"type": "web_search"},
    ],
)

print(response.output_text)

Note: The exact configuration keys for web_search (and whether it requires additional parameters beyond type) are specific to the current API version — refer to Unit 15 and official documentation for the current schema before relying on the exact shape shown here.

The tools list now contains two entries, and the model decides independently, based on the input and its own judgment, whether to invoke file_search, web_search, both, or neither. The instructions explicitly describe when each tool is appropriate — this matters more here than in a single-tool setup, because with two tools available, an under-specified prompt leaves more room for the model to default to only one of them (commonly, favoring whichever tool feels more directly relevant to surface-level keyword matches in the question) when the ideal answer actually needs both. The example question is deliberately constructed to need both: the internal policy document (via file search) and current regulatory information (via web search, since GDPR requirements can be updated by lawmakers independent of when your internal documents were last reviewed).

Reading Which Tools Were Actually Used

Exactly as in Lesson 4 and Lesson 8, the structured output tells you what actually happened, which matters more with two tools available because you can no longer assume which one (or both) fired:

def summarize_tool_usage(response):
    used_file_search = False
    used_web_search = False

    for item in response.output:
        if item.type == "file_search_call":
            used_file_search = True
        elif item.type == "web_search_call":
            used_web_search = True

    return {"file_search": used_file_search, "web_search": used_web_search}


usage = summarize_tool_usage(response)
print(usage)

Note: The exact item type name for a web search invocation (web_search_call here) is version-specific — confirm the current naming against official documentation.

This function scans the response's output items and reports which of the two tools were actually invoked for this particular request. Logging this alongside every response in a dual-tool assistant is valuable for the same reason as in Lesson 8: it turns "did the model use the right sources" from a guess into a measurable fact you can review, and over time it can reveal systematic problems — for instance, if file_search almost never fires even for questions that clearly should hit your internal documents, that's a sign your instructions or your document coverage need attention.

A Practical Pattern: Internal-First, Web as Fallback

For many applications, a stricter and more predictable pattern than "let the model freely choose" is preferable: try file search first, and only fall back to web search when file search doesn't produce a confident, evidence-backed answer. This gives you more control over cost (web search calls typically cost more and take longer) and over trust (you may want to bias toward your own vetted documents whenever they're sufficient).

def answer_with_fallback(question, vector_store_id):
    file_search_response = client.responses.create(
        model="gpt-5.6-terra",
        instructions=(
            "Answer only using information found via file search. If the "
            "documents do not contain enough information, respond with exactly: "
            "\"NOT_FOUND_INTERNALLY\""
        ),
        input=question,
        tools=[{"type": "file_search", "vector_store_ids": [vector_store_id]}],
    )

    if "NOT_FOUND_INTERNALLY" not in file_search_response.output_text:
        return file_search_response.output_text, "internal_documents"

    web_search_response = client.responses.create(
        model="gpt-5.6-terra",
        instructions=(
            "The internal knowledge base did not have an answer. Answer using "
            "current public information via web search, and note that this "
            "answer comes from external sources, not internal documents."
        ),
        input=question,
        tools=[{"type": "web_search"}],
    )

    return web_search_response.output_text, "web_search"


answer, source = answer_with_fallback(
    "What is our current password rotation policy?",
    "vs_it_security_policies_9c31",
)
print(f"[{source}] {answer}")

This function makes two sequential requests rather than one combined request: first, a file_search-only call with instructions to emit an exact sentinel string ("NOT_FOUND_INTERNALLY") when internal documents don't cover the question — the same detectable-fallback-phrase pattern from Lesson 8. Only if that sentinel appears does it make a second call using web_search. Returning a source label alongside the answer text lets calling code (and, importantly, the end user) know whether they're looking at vetted internal information or an external web result, which matters for questions like internal security policy where an external, generic answer would be actively misleading if presented as if it were your own policy.

This two-call pattern costs more in latency than a single combined-tools call when the fallback path is taken, but it buys a hard guarantee that internal documents are checked first and web content is never blended into an answer about something your internal documents were supposed to authoritatively cover — sacrificing the model's discretion for structural predictability. Whether that trade-off is worth it depends on how much your application needs to guarantee behavior versus optimize for the fewest requests.

Choosing Between Combined and Sequential Patterns

PatternBehaviorBest forTrade-off
Single call, both tools attachedModel freely decides which tool(s) to useOpen-ended research assistants; questions that often need both sources togetherLess predictable which sources get used; harder to guarantee internal-first behavior
Sequential, internal-first with fallbackFile search always tried first; web search only on explicit missCompliance-sensitive or internal-policy assistants where source trust mattersHigher latency when fallback triggers; two requests instead of one

Neither pattern is universally correct. An assistant answering "what's changed in this industry recently and how does it affect our policy" benefits from the combined pattern, since both sources are usually needed together in the same answer. An assistant answering "what is our policy on X" — where an authoritative internal answer should always take precedence when one exists — benefits from the sequential, internal-first pattern.

Common Mistakes

Attaching both tools without instructions on when to use each, leaving the model's default tool-selection behavior to decide, which can produce inconsistent results across similar questions — always describe, in instructions, what kind of question each tool is meant to answer.

Never distinguishing, in the final response to the user, whether an answer came from internal documents or the web, which can mislead users into treating an external, possibly less authoritative source as if it were your own vetted policy — always track and, where appropriate, surface the source.

Defaulting to the combined single-call pattern for compliance-sensitive assistants where a guarantee of internal-first behavior actually matters, when the sequential fallback pattern would provide that guarantee at an acceptable latency cost.

Best Practices

Write explicit, question-type-specific instructions when both tools are attached to one call, describing concretely what kind of question belongs to internal documents versus current public information.

Log which tool or tools were actually invoked for every response in a dual-tool assistant, so you can detect and correct systematic under-use of one tool over time.

Choose the internal-first sequential pattern whenever source trust or compliance matters more than minimizing request count, reserving the combined single-call pattern for open-ended assistants where blending sources in one answer is actually desirable.

0 Comments

Reviewed before they appear

No comments yet.

OpenAI SDK
Introduction to the OpenAI SDK Setting Up Python Creating an API Key Your First Call — client.responses.create() and response.output_text Understanding Billing, Credits, and What a Request Costs Why Responses Replaced Chat Completions Anatomy of a Request: model, input, and instructions Anatomy of a Response: The Typed output Array, Not Just Text Roles: User, Assistant, and Developer/System Choosing a Model, and Reading the Models Page Instead of Memorizing Names Instructions vs. Input Writing Prompts That Get Consistent Results Few-Shot Examples Reasoning Models and the reasoning Parameter Debugging a Prompt That Misbehaves Why Streaming Matters for User Experience stream=True and Iterating Over Events Handling the Event Types You Actually Care About Background Mode for Long-Running Jobs Project — Add Live Streaming to Your Chatbot The Problem With Parsing Free Text JSON Schema and Strict Mode Pydantic Models With the SDK's Parse Helpers Handling Refusals and Validation Failures Project — A Resume-to-JSON Extractor Working With input_image input_file, PDFs, and the Files API Image Generation Speech-to-Text and Text-to-Speech Project: A PDF Question-Answering Script What Function Calling Is Defining a Tool Schema The Full Loop Multiple Tools Errors, Timeouts, and Untrusted Arguments Project: A Weather Assistant Web Search File Search and Vector Stores Code Interpreter Remote MCP Servers and Connectors Project: A Research Assistant What an Embedding Is, Without the Maths Generating and Storing Embeddings Similarity Search From Scratch Hosted Vector Stores vs. Rolling Your Own A Small RAG App Over a Folder of Notes Agents vs. a Single API Call — When You Need One pip install openai Giving Agents Tools Handoffs and Multi-Agent Triage Guardrails and Approvals Tracing and Observing What Your Agent Did A Multi-Agent Support Desk Error Codes and What Each One Means Retries, Timeouts, and Backoff Rate Limits and Spend Limits Prompt Caching and Cost Optimisation The Batch API for Bulk Work Async Clients and Concurrency Moderation and Safety Best Practices Designing the App Backend With FastAPI Streaming to a Simple Frontend Deploying and a Cost/Safety Checklist Why Web Search Is Useful for Current Information Using the Web Search Tool with the Responses API Configuring Search Behavior for Application Use Cases Understanding Citations and Source Attribution Where to Go Next Building a Research Assistant with Web Search Combining Web Search with Structured Outputs Handling Conflicting or Low-Quality Web Sources Reducing Unsupported Claims with Grounded Generation Testing Freshness-Sensitive AI Answers Production Considerations for Web-Grounded Applications Understanding File Search and Retrieval-Augmented Generation Creating and Organizing Vector Stores Uploading Documents for Retrieval Connecting Vector Stores to Responses API Requests Designing Document Metadata and Filtering Strategies Building a PDF Question-Answering Application Improving Retrieval Quality With Better Document Preparation Handling Missing Evidence and Retrieval Failures Combining File Search With Web Search Building a Production Knowledge-Base Assistant What the Code Interpreter Tool Is Designed For Running Python-Based Analysis Through the OpenAI SDK Uploading Datasets for Analysis Analyzing CSV and Spreadsheet Data Generating Charts and Data Summaries Handling Generated Files and Downloadable Artifacts Building a Data-Analysis Assistant Combining Code Execution with Structured Outputs Validating Generated Calculations and Results Security and Sandbox Considerations for Code Execution Understanding Multimodal Input with the OpenAI SDK Sending Images to a Model Image Analysis from URLs and Uploaded Files Extracting Text and Information from Screenshots Building an Image-Question-Answering Application Combining Image Input with Structured Output Analyzing Multiple Images in One Request Handling Image Quality and Input Limitations Designing Multimodal Prompts for Reliable Results Building a Practical Vision-Powered Python Application Understanding Speech-to-Text and Text-to-Speech Workflows Transcribing Audio with the OpenAI SDK Working with Uploaded Audio Files Handling Timestamps and Transcription Metadata Building a Meeting Transcription Workflow Generating Spoken Responses from Text Handling Long Audio and Processing Failures Combining Audio with Text and Tool Calling Building an End-to-End Python Voice Application What Embeddings Are and When to Use Them Generating Embeddings With the OpenAI API Preparing Text for Embedding Comparing Vectors With Cosine Similarity Building a Simple Semantic Search Engine in Python Storing Embeddings in a Database Metadata Filtering for Semantic Search Chunking Strategies for Better Retrieval Evaluating Semantic Search Quality Building a Document Similarity Application When Batch Processing Makes Sense Designing Large-Volume AI Processing Pipelines Using Asynchronous Python with the OpenAI SDK Running Concurrent Requests Safely Controlling Concurrency and Avoiding Rate Limits Tracking Batch Job Progress Handling Partial Failures in Bulk Workloads Retrying Failed Items Without Duplicating Successful Work Designing Resumable AI Processing Jobs Building a Production Batch-Processing Pipeline Batch Processing Makes Sense Large-Scale AI Processing Pipelines Async Python with OpenAI SDK Safe Concurrent Requests Concurrency & Rate Limits Batch Progress Tracking Partial Failure Handling Safe Retry Handling Resumable AI Jobs Production Batch Pipeline System–User Data Separation Reusable App Instructions Prompt Templates & Variables Extraction & Classification Prompts Summarization & Transformation Prompts Explicit Output Requirements Prompt Version Management Prompt Testing & Evaluation Reusable Python Prompt Library API Key Security Secure API Key Storage Secure Secret Management Prompt Injection Prevention Trusted vs. Untrusted Content Tool Argument Validation Sensitive Data Handling Secure Logging AI Action Authorization Production AI Security Checklist Why AI Applications Need Evaluation Beyond Unit Tests Unit Testing OpenAI SDK Integration Code Mocking API Responses in Python Tests Testing Structured Outputs Against Schemas Testing Tool-Calling Workflows Building a Small Evaluation Dataset Measuring Accuracy, Consistency, and Failure Rates Regression Testing Prompts and Model Changes Human Evaluation Versus Automated Evaluation Creating a Repeatable Evaluation Pipeline AI Request Monitoring Token Cost Management Usage Metrics Design Reducing Model Calls Prompt & Context Optimization Model Selection & Optimization AI Caching Strategies Interactive Latency Optimization Usage Dashboards & Budget Alerts Performance & Cost Checklist Every API Call Starts Fresh Fixing API Statelessness Server-Side Conversation Memory Limits of Response Chaining What We're Building Conversation Memory Challenges Preparing an OpenAI SDK Application for Deployment Environment-Specific Configuration for Development and Production Deploying a Python AI Service with Docker Container Health Checks and Startup Configuration Managing Secrets in Cloud Deployments Background Workers for Long-Running AI Tasks Queues and Asynchronous Job Architectures Scaling AI Workloads Horizontally Monitoring Production Incidents and Failures Production Deployment Checklist for OpenAI SDK Applications Reusable OpenAI Service Classes AI Client Dependency Injection Typed AI Responses Python Configuration Management AI Request Decorators Centralized AI Error Handling Clean SDK Abstractions Reusable OpenAI Utilities Internal AI Python Libraries SDK Integration Maintenance Production AI Chatbot Document Q&A System Web Research Assistant Customer Support Agent AI Data Analysis Assistant Image Analysis App Meeting Transcription & Summary Semantic Document Search Multi-Tool AI Agent Production OpenAI SDK App Why "It Looked Fine When I Tested It" Isn't Enough Timing Note Status Note Pre-Decision Status Note Current Availability Note
Ask about this post
AI Ask about this post

Ask questions about Combining File Search With Web Search and get answers drawn from it.

Signed-in readers only.