What Metadata Adds to Retrieval

Similarity search alone tells you which chunks are semantically closest to a query. It has no concept of structured facts about a document — when it was published, which product it applies to, what its access level is, or what region it's relevant to. Metadata (sometimes called attributes) is how you attach those structured facts to a file so you can constrain a search beyond pure semantic similarity.

Consider a support knowledge base that contains documentation for three product tiers — Basic, Pro, and Enterprise — each with its own setup guide. A user on the Basic plan asking "how do I configure single sign-on" should never see the Enterprise SSO guide, even if it's the most semantically similar chunk in the store, because it describes a feature the user doesn't have access to. Pure similarity search cannot express that constraint; it only knows about textual and semantic closeness. Metadata filtering closes this gap by letting you say, in effect, "search only among chunks where product_tier equals basic."

Attaching Metadata at Upload Time

from openai import OpenAI

client = OpenAI()

with open("enterprise-sso-setup.pdf", "rb") as f:
    result = client.vector_stores.files.upload_and_poll(
        vector_store_id="vs_product_docs_55c1",
        file=f,
        attributes={
            "product_tier": "enterprise",
            "doc_type": "setup_guide",
            "last_updated": "2026-06-01",
        },
    )

print(result.status)

Note: The parameter name for attaching metadata (attributes here), the supported value types (strings, numbers, booleans), and any limits on the number of attributes per file are specific to the current API version — confirm these against official OpenAI documentation before designing a metadata schema around them.

This attaches a small set of key-value pairs to the file at the moment it's added to the vector store. Every chunk produced from this file inherits these attributes. The values shown are deliberately simple — a string category, another string category, and a date string — because metadata filtering typically supports a constrained set of comparison operations (equality, and sometimes range comparisons on numbers or dates), not arbitrary logic. Designing metadata as flat, simple key-value pairs rather than nested structures keeps it usable by the filtering syntax the API actually supports.

Filtering at Query Time

response = client.responses.create(
    model="gpt-5.6-terra",
    input="How do I configure single sign-on?",
    tools=[
        {
            "type": "file_search",
            "vector_store_ids": ["vs_product_docs_55c1"],
            "filters": {
                "type": "eq",
                "key": "product_tier",
                "value": "basic",
            },
        }
    ],
)

print(response.output_text)

Note: The filters parameter's structure — including supported operators like eq, and how to express compound conditions — is version-specific. Verify the current filtering syntax against official documentation before relying on the exact shape shown here.

This request restricts the file_search tool to only consider chunks whose product_tier attribute equals "basic", before similarity ranking even happens. Conceptually, filtering and similarity search apply in sequence: the filter narrows the candidate set to chunks matching the structured condition, and only within that narrowed set does the system rank by embedding similarity. This ordering matters — it's why filtering can dramatically improve precision in a mixed knowledge base: it removes entire categories of wrong-but-similar content before they ever get a chance to compete in the similarity ranking.

You determine the value of product_tier for the filter dynamically, based on application context — typically the authenticated user's actual plan, looked up from your own user database, not from anything the user typed. This is an important distinction from a security perspective, covered further below.

Combining Multiple Filter Conditions

Real filtering needs are rarely a single equality check. A typical pattern combines a tenant or access boundary with a topical constraint:

def build_support_filter(product_tier, doc_type=None):
    conditions = [
        {"type": "eq", "key": "product_tier", "value": product_tier},
    ]
    if doc_type is not None:
        conditions.append({"type": "eq", "key": "doc_type", "value": doc_type})

    if len(conditions) == 1:
        return conditions[0]

    return {"type": "and", "filters": conditions}


filter_for_request = build_support_filter("pro", doc_type="setup_guide")
print(filter_for_request)

Note: Compound filter operators such as and / or, and their exact structure, are version-specific — confirm the current syntax against official documentation before shipping compound filters to production.

This helper function builds a filter dynamically: it always includes the tenant-style product_tier condition, and optionally adds a doc_type condition when the caller specifies one, combining multiple conditions with an and wrapper only when there's more than one. Structuring filter construction as a small, testable function like this — rather than hand-writing a dictionary literal at every call site — reduces the chance of a typo silently producing a filter that matches nothing (which would then look, misleadingly, like "no evidence found" rather than "broken filter," a failure mode discussed further in Lesson 8).

You can test this logic without any API calls:

def test_build_support_filter_single_condition():
    result = build_support_filter("basic")
    assert result == {"type": "eq", "key": "product_tier", "value": "basic"}
    print("PASS: single condition returns a plain eq filter")


def test_build_support_filter_multiple_conditions():
    result = build_support_filter("enterprise", doc_type="setup_guide")
    assert result == {
        "type": "and",
        "filters": [
            {"type": "eq", "key": "product_tier", "value": "enterprise"},
            {"type": "eq", "key": "doc_type", "value": "setup_guide"},
        ],
    }
    print("PASS: multiple conditions are combined with 'and'")


test_build_support_filter_single_condition()
test_build_support_filter_multiple_conditions()

Both tests call build_support_filter directly with plain Python values and assert on the exact dictionary structure returned, with no network access involved. This validates the filter-construction logic in isolation — the part of the system you actually wrote and can get wrong — separately from whatever the live API does with that filter once submitted.

Designing a Metadata Schema

Before attaching metadata to hundreds of documents, it pays to design the schema deliberately, the same way you'd design columns for a database table. A few practical guidelines:

• Keep keys consistent across all documents in a store. If half your files use product_tier and the other half use tier, every filter you write has to account for both, which is a needless source of bugs.
• Use a small, closed set of values for categorical fields ("basic", "pro", "enterprise" — not free-text product names typed inconsistently by different authors).
• Store dates in a consistent, sortable format (ISO 8601, YYYY-MM-DD) if you plan to filter or reason about recency.
• Don't over-model. Adding ten speculative attributes that no filter ever actually uses adds maintenance burden for no retrieval benefit. Add an attribute when you have a concrete filtering need for it.

A useful mental model: metadata answers questions a human librarian would ask before handing you a document — "which edition," "which department," "is this still current" — while the embedding-based similarity search answers "which passage actually discusses what you're asking about." They are complementary, not competing, mechanisms.

Metadata Filtering Is Not Access Control

This point deserves emphasis because it's a genuinely common and serious mistake: a filters parameter on a file_search call shapes what gets retrieved for a well-behaved request, but it is enforced by your application code choosing what filter to send — nothing on the vector store itself stops a request from being sent without that filter, or with a different one, if an attacker controls how the request is constructed. If different users must be prevented from ever seeing certain documents regardless of what filter value is used, the enforceable boundary is a separate vector store scoped only to documents that user is allowed to see (as discussed in Lesson 2), not a metadata filter within a shared store containing everyone's documents.

Use metadata filtering to improve relevance within a legitimately shared corpus (all documentation a given class of user is already permitted to see). Use separate vector stores, chosen server-side based on authenticated identity, as the actual access-control boundary.

Common Mistakes

Treating metadata filters as a security boundary, letting a client-supplied value determine which documents are searchable, which allows a malicious or buggy client to simply omit or alter the filter and retrieve documents it shouldn't — always determine filter values from trusted, server-side context, and use separate vector stores for hard access boundaries.

Using inconsistent metadata keys or value casing across documents ("Enterprise" in some files, "enterprise" in others), which causes filters to silently match fewer documents than intended — enforce a schema and validate attribute values at ingestion time.

Building filter dictionaries by hand at every call site instead of through a shared, tested function, which invites typos that produce a filter matching nothing — centralize filter construction and test it independently of the API.

Best Practices

Design your metadata schema before bulk-uploading documents, treating it like a lightweight database schema: fixed keys, closed value sets for categorical fields, and consistent date formatting.

Derive filter values from authenticated server-side context, never from raw, unvalidated client input, especially for any attribute that gates sensitive content.

Log the filter actually sent with each request alongside the retrieval results, so that when an answer seems to be missing expected content, you can quickly tell whether the problem was an overly restrictive filter or a genuine gap in the knowledge base.