Two Ways to Build the Same Capability

Lessons 1 through 3 built a complete, working retrieval pipeline entirely by hand: generate embeddings, store them alongside their source text and metadata, and rank stored documents against a query using cosine similarity. Unit 9, Lesson 2 covered a different way to get the same underlying capability: create a vector store, upload documents to it, and let the platform's file search tool handle chunking, embedding, and search internally. This lesson puts those two approaches side by side directly, since choosing between them — or knowing when to combine them — is one of the most consequential decisions in designing a retrieval system.

What a Hosted Vector Store Actually Handles for You

A hosted vector store, as Unit 9, Lesson 2 introduced it, takes care of an entire pipeline internally: splitting an uploaded document into appropriately-sized chunks, generating an embedding for each chunk, storing those embeddings in a way that scales to large collections, and executing similarity search efficiently — potentially using an approximate nearest-neighbor index rather than a naive linear scan, exactly the concern Lesson 3 raised about scaling a hand-rolled search.

vector_store = client.vector_stores.create(name="Policy Documents")

with open("return_policy.pdf", "rb") as f:
    client.vector_stores.files.upload(vector_store_id=vector_store.id, file=f)

response = client.responses.create(
    model="gpt-5.6-terra",
    input="How long do I have to return an item?",
    tools=[{"type": "file_search", "vector_store_ids": [vector_store.id]}],
)

Every one of the individual steps Lessons 1 through 3 built by hand — embedding generation, chunk-level storage, similarity ranking — happens inside this call, invisibly. This is the appeal of the hosted approach: a working retrieval system in a handful of lines, with no decisions required about chunk size, storage format, or which similarity metric to use.

What Rolling Your Own Actually Requires

The custom pipeline built across Lessons 1 through 3 requires deciding and implementing every one of those same steps explicitly: choosing how to split documents into pieces before embedding them (a step this course has not covered in depth but which any hand-rolled pipeline over documents longer than a paragraph or two eventually needs to address), choosing a storage format appropriate to the collection's scale, writing and maintaining the similarity search logic itself, and handling the transition to a more sophisticated index if and when a naive linear scan stops being fast enough.

def embed_and_store(client, texts: list[str]) -> list[dict]:
    response = client.embeddings.create(model="text-embedding-4", input=texts)
    return [{"text": text, "embedding": data.embedding} for text, data in zip(texts, response.data)]

def cosine_similarity(vector_a: list[float], vector_b: list[float]) -> float:
    import math
    dot_product = sum(a * b for a, b in zip(vector_a, vector_b))
    magnitude_a = math.sqrt(sum(a * a for a in vector_a))
    magnitude_b = math.sqrt(sum(b * b for b in vector_b))
    if magnitude_a == 0 or magnitude_b == 0:
        return 0.0
    return dot_product / (magnitude_a * magnitude_b)

def semantic_search(client, query: str, stored_documents: list[dict], top_k: int = 3) -> list[dict]:
    query_vector = client.embeddings.create(model="text-embedding-4", input=query).data[0].embedding
    scored = [
        {"text": doc["text"], "score": cosine_similarity(query_vector, doc["embedding"])}
        for doc in stored_documents
    ]
    scored.sort(key=lambda d: d["score"], reverse=True)
    return scored[:top_k]

None of this is difficult, as Lessons 1 through 3 demonstrated, but every one of these functions is now something you own: you are responsible for its correctness, its performance as the collection grows, and any bugs it develops, in a way that a hosted vector store's internal implementation simply isn't your responsibility to maintain.

Comparing the Two Approaches Directly

Neither row in this table makes one approach strictly better; each is a genuine trade-off between convenience and control.

When to Choose a Hosted Vector Store

For the majority of retrieval-augmented use cases — answering questions from a company's internal documentation, building a support assistant grounded in policy documents, searching a collection of PDFs or text files — a hosted vector store is the right default. The documents are naturally file-shaped, the platform's default chunking and similarity behavior is good enough for the task, and there is no requirement to combine semantic similarity with a custom ranking signal the built-in tool doesn't expose. Choosing the hosted approach here isn't a compromise; it's simply not paying an implementation and maintenance cost for a capability that's already available and well-built.

When to Roll Your Own

A custom pipeline earns its additional complexity when a specific requirement genuinely falls outside what a hosted vector store supports: retrieval over data that isn't naturally document-shaped (structured database records, short user-generated snippets, product listings assembled from several fields), a need for a similarity metric other than whatever the platform uses internally, a need to combine semantic similarity with other ranking signals a hosted store's parameters don't expose (recency weighting, a business-specific boost, a signal from a different part of the system entirely), or a requirement to inspect, audit, or directly manipulate the embedding vectors themselves — none of which a hosted vector store is designed to expose, since it deliberately treats the underlying vectors as an internal implementation detail.

def rank_with_custom_signal(query_score: float, recency_weight: float, popularity_score: float) -> float:
    """Illustrative — combining a semantic similarity score with other signals
    that a hosted vector store's built-in ranking has no way to express."""
    return (query_score * 0.7) + (recency_weight * 0.2) + (popularity_score * 0.1)

A weighted combination like this — semantic similarity blended with recency and popularity — is exactly the kind of custom ranking logic a hosted vector store's fixed internal ranking cannot accommodate, and exactly the kind of case where the extra implementation cost of a hand-rolled pipeline is worth paying.

A Middle Ground: Starting Hosted, Migrating If Needed

A practical strategy for a new application is to start with a hosted vector store by default, given its low setup cost, and migrate to a custom pipeline only if a specific, concrete limitation is actually encountered — rather than guessing upfront that custom retrieval logic will eventually be needed and building it prematurely.

def should_consider_custom_pipeline(needs_custom_metric: bool, needs_non_document_data: bool, needs_custom_ranking_signals: bool) -> bool:
    """Illustrative decision check — a custom pipeline is worth its added cost
    only when a hosted vector store demonstrably can't meet a real requirement,
    not merely because building one yourself is possible."""
    return needs_custom_metric or needs_non_document_data or needs_custom_ranking_signals

print(should_consider_custom_pipeline(False, True, False))  # True — non-document data source

This mirrors a pattern this course has applied repeatedly to other built-versus-buy decisions — reasoning effort settings (Unit 3), image resolution (Unit 7), and the built-in tools of Unit 9 generally: default to the simpler, platform-provided option, and escalate to a custom implementation deliberately, when a specific requirement demonstrates the default genuinely isn't enough, rather than as a default starting posture.

Common Mistakes

Building a custom embeddings and retrieval pipeline by default, before checking whether a hosted vector store would already meet the actual requirements, and taking on unnecessary implementation and maintenance cost as a result.

Choosing a hosted vector store for a use case that genuinely needs a custom similarity metric or ranking signal, and then working around the tool's fixed behavior with awkward workarounds instead of building the custom pipeline the use case actually calls for.

Underestimating the ongoing maintenance cost of a hand-rolled retrieval pipeline, treating it as a one-time implementation effort rather than a system that needs to keep scaling and working correctly as the underlying collection grows and changes.

Assuming a custom pipeline will eventually be necessary and building one preemptively, rather than starting with the simpler hosted option and migrating only once a concrete limitation is actually encountered.

Best Practices

Default to a hosted vector store (Unit 9, Lesson 2) for standard document-based retrieval, reserving a custom pipeline for cases with a specific, concrete requirement the hosted tool doesn't support.

Identify the specific limitation driving a decision to build custom retrieval logic, rather than building one out of a general sense that more control is always better.

Weigh the ongoing maintenance burden of a custom pipeline honestly against its flexibility benefit, since the cost of a hand-rolled system is not just the initial implementation but everything required to keep it correct and performant over time.

Start with the simpler option and migrate only when a real limitation is encountered, rather than guessing upfront which approach a use case will eventually need.