From Vectors to a Ranked List of Results

Lessons 1 and 2 established that embeddings place similar-meaning text close together in a high-dimensional space, and covered how to generate and store a collection of them. This lesson covers what "close together" actually means numerically, and how to turn that notion of closeness into a practical, ranked search: given a query and a collection of stored documents, return the documents most similar in meaning to the query, ordered from most to least similar.

Cosine Similarity: The Standard Metric

The most common way to measure how similar two embedding vectors are is cosine similarity — a measure of the angle between two vectors, rather than the straight-line distance between them. Two vectors pointing in nearly the same direction have a cosine similarity close to 1; two vectors pointing in completely unrelated directions have a cosine similarity close to 0; two vectors pointing in opposite directions have a cosine similarity close to -1.

import math

def cosine_similarity(vector_a: list[float], vector_b: list[float]) -> float:
    dot_product = sum(a * b for a, b in zip(vector_a, vector_b))
    magnitude_a = math.sqrt(sum(a * a for a in vector_a))
    magnitude_b = math.sqrt(sum(b * b for b in vector_b))
    if magnitude_a == 0 or magnitude_b == 0:
        return 0.0
    return dot_product / (magnitude_a * magnitude_b)

vector_a = [1.0, 2.0, 3.0]
vector_b = [1.0, 2.0, 3.0]
vector_c = [-1.0, -2.0, -3.0]

print(cosine_similarity(vector_a, vector_b))  # 1.0 — identical direction
print(cosine_similarity(vector_a, vector_c))  # -1.0 — opposite direction

Cosine similarity is preferred over raw Euclidean distance (straight-line distance) for text embeddings specifically because it measures direction rather than magnitude — two embedding vectors representing the same meaning should be considered similar regardless of small differences in vector length that can arise from factors unrelated to meaning, such as text length. Measuring the angle between vectors, rather than the distance between their endpoints, is exactly what makes this the standard choice for comparing embeddings, and it's why this function, or an equivalent from a numerical library, appears in essentially every hand-built retrieval system.

Building a Simple Semantic Search Function

Combining Lesson 2's embed_and_store() pattern with cosine_similarity() produces a complete, working semantic search over a small in-memory document collection.

def embed_and_store(client, texts: list[str]) -> list[dict]:
    response = client.embeddings.create(model="text-embedding-4", input=texts)
    return [{"text": text, "embedding": data.embedding} for text, data in zip(texts, response.data)]

def semantic_search(client, query: str, stored_documents: list[dict], top_k: int = 3) -> list[dict]:
    query_response = client.embeddings.create(model="text-embedding-4", input=query)
    query_vector = query_response.data[0].embedding

    scored_documents = []
    for document in stored_documents:
        score = cosine_similarity(query_vector, document["embedding"])
        scored_documents.append({"text": document["text"], "score": score})

    scored_documents.sort(key=lambda d: d["score"], reverse=True)
    return scored_documents[:top_k]

documents = [
    "Our return policy allows returns within 30 days of purchase.",
    "Shipping typically takes 3 to 5 business days.",
    "Refunds are processed within 5 to 7 business days after we receive the item.",
    "We offer a 20% discount for orders over $100.",
]

stored_documents = embed_and_store(client, documents)
results = semantic_search(client, "How long until I get my money back?", stored_documents)

for result in results:
    print(f"{result['score']:.3f} — {result['text']}")

Notice that the query "How long until I get my money back?" shares almost no words with "Refunds are processed within 5 to 7 business days after we receive the item," yet semantic search should rank that document highest — exactly the capability Lesson 1 introduced embeddings to provide, and exactly the failure mode a literal keyword search would have missed entirely. The top_k parameter caps how many results are returned, mirroring the same "retrieve only what's actually needed" reasoning Unit 9, Lesson 2 applied to max_num_results for the built-in file search tool, since returning every stored document ranked by score, rather than just the most relevant handful, would waste context when the request that consumes these results only needs the top few.

Why This Is Worth Understanding Even With File Search Available

Unit 9, Lesson 2's file search tool performs exactly this kind of retrieval internally, and for most applications, using it directly remains simpler than building the pipeline shown here. Understanding the mechanics matters for the cases where file search's built-in behavior doesn't fit: a similarity metric other than cosine similarity for a specialized use case, retrieval over data that isn't naturally represented as uploadable documents (structured records, short user-generated snippets, product listings), or a need to combine semantic similarity with other ranking signals (recency, popularity, an explicit business rule) that file search's built-in ranking doesn't expose. Building this pipeline by hand is strictly more work, but it is also strictly more flexible — the trade-off Unit 9, Lesson 2 already flagged when it introduced file search as the platform's built-in implementation of retrieval, versus the option of building a custom one, and the same trade-off Lesson 4 examines directly by comparing this hand-rolled approach against hosted vector stores.

Combining Semantic Search With Metadata Filtering

A realistic search often needs to combine semantic similarity with hard filters based on other attributes — restricting results to a specific category, date range, or status before or after ranking by similarity.

documents_with_metadata = [
    {"text": "Our return policy allows returns within 30 days.", "category": "policy", "embedding": None},
    {"text": "Shipping typically takes 3 to 5 business days.", "category": "logistics", "embedding": None},
    {"text": "Refunds are processed within 5 to 7 business days.", "category": "policy", "embedding": None},
]

texts = [doc["text"] for doc in documents_with_metadata]
response = client.embeddings.create(model="text-embedding-4", input=texts)
for doc, data in zip(documents_with_metadata, response.data):
    doc["embedding"] = data.embedding

def semantic_search_with_filter(client, query: str, stored_documents: list[dict], category: str, top_k: int = 3) -> list[dict]:
    filtered_documents = [doc for doc in stored_documents if doc["category"] == category]
    query_vector = client.embeddings.create(model="text-embedding-4", input=query).data[0].embedding

    scored = [
        {"text": doc["text"], "score": cosine_similarity(query_vector, doc["embedding"])}
        for doc in filtered_documents
    ]
    scored.sort(key=lambda d: d["score"], reverse=True)
    return scored[:top_k]

results = semantic_search_with_filter(client, "how do refunds work", documents_with_metadata, category="policy")

Filtering to category == "policy" before computing similarity scores, rather than ranking the entire collection and filtering afterward, is both more efficient (fewer similarity computations) and more correct (a highly similar result from the wrong category never has a chance to crowd out a less similar but correctly categorized one). This combination of a hard structural filter with a soft semantic ranking is a common and useful pattern in real retrieval systems, mirroring the metadata filters option Unit 9, Lesson 2 introduced for the built-in file search tool, implemented here explicitly by hand.

Scaling Beyond a Small In-Memory Collection

The semantic_search() function above computes a similarity score against every single stored document for every single query — a linear scan that works fine for a few thousand documents but becomes noticeably slow for a collection with millions, since the cost of a single search grows directly with the size of the collection.

def semantic_search_naive_complexity_note(num_documents: int, num_queries_per_day: int) -> int:
    """Illustrative — a naive linear scan performs one similarity computation
    per stored document, per query, so total daily comparisons scale with
    the product of collection size and query volume."""
    return num_documents * num_queries_per_day

print(semantic_search_naive_complexity_note(1_000_000, 10_000))  # 10 billion comparisons per day

For a collection at this kind of scale, a specialized vector database or approximate nearest-neighbor index (structures designed specifically to avoid comparing a query against every stored vector, trading a small amount of retrieval accuracy for a large improvement in search speed) becomes necessary — which is, again, exactly the kind of underlying infrastructure Unit 9, Lesson 2's vector stores manage for you, and the exact question Lesson 4 addresses directly.

Vectorizing the Comparison With NumPy

The pure-Python cosine_similarity() function above is easy to follow but recomputes the same kind of loop-based arithmetic repeatedly; for anything beyond a small collection, using NumPy's vectorized array operations produces the same result meaningfully faster, since NumPy performs the underlying arithmetic in optimized, compiled code rather than a Python-level loop.

import numpy as np

def semantic_search_vectorized(client, query: str, stored_documents: list[dict], top_k: int = 3) -> list[dict]:
    query_vector = np.array(client.embeddings.create(model="text-embedding-4", input=query).data[0].embedding)
    document_matrix = np.array([doc["embedding"] for doc in stored_documents])

    # Cosine similarity for every stored document against the query, computed
    # as a single vectorized operation rather than one comparison at a time.
    dot_products = document_matrix @ query_vector
    document_norms = np.linalg.norm(document_matrix, axis=1)
    query_norm = np.linalg.norm(query_vector)
    scores = dot_products / (document_norms * query_norm)

    ranked_indices = np.argsort(scores)[::-1][:top_k]
    return [{"text": stored_documents[i]["text"], "score": float(scores[i])} for i in ranked_indices]

The document_matrix @ query_vector line computes the dot product between the query and every stored document's embedding in one matrix operation, replacing what would otherwise be a separate Python-level loop iteration per document — for a collection of even a few thousand documents, this difference is directly noticeable, and it becomes the only practical approach well before reaching the millions-of-documents scale discussed above. This is a reasonable middle ground between the simple, purely illustrative loop version shown earlier and a full dedicated vector database — a comparison Lesson 4 covers in more detail.

Testing Similarity Search Logic Without Real Embeddings

Following this course's dependency-injection pattern, the ranking and filtering logic can be tested with fabricated vectors, avoiding the cost and non-determinism of a real embedding call for every test run.

def test_semantic_search_ranks_by_similarity():
    stored_documents = [
        {"text": "closely related document", "embedding": [1.0, 0.0, 0.0]},
        {"text": "unrelated document", "embedding": [0.0, 1.0, 0.0]},
    ]
    query_vector = [0.9, 0.1, 0.0]

    scored = [
        {"text": doc["text"], "score": cosine_similarity(query_vector, doc["embedding"])}
        for doc in stored_documents
    ]
    scored.sort(key=lambda d: d["score"], reverse=True)

    assert scored[0]["text"] == "closely related document"
    print("PASS: semantic search ranking correctly places the more similar vector first")

test_semantic_search_ranks_by_similarity()

Using small, hand-constructed vectors with an obviously correct expected ranking (rather than real embeddings from real text) lets the ranking and filtering logic itself be verified quickly and deterministically, reserving real embedding calls for a smaller set of end-to-end tests confirming actual text produces the semantic relationships you expect.

Common Mistakes

Using raw Euclidean distance instead of cosine similarity for text embeddings, without accounting for the fact that cosine similarity's focus on direction rather than magnitude is specifically why it's the standard choice for this kind of comparison.

Ranking a full collection by similarity before applying a hard structural filter, wasting computation and risking an off-topic result outranking a correctly filtered one.

Scaling a hand-rolled linear-scan search to a very large document collection, when the cost of comparing every query against every stored vector grows directly with collection size and becomes impractical well before reaching millions of documents.

Rebuilding a custom embeddings and similarity pipeline for a use case the built-in file search tool already handles well, taking on unnecessary implementation and maintenance work.

Best Practices

Use cosine similarity as the default metric for comparing text embeddings, understanding why it's preferred over raw distance measures for this specific kind of comparison.

Apply hard structural filters before ranking by semantic similarity, rather than after, for both efficiency and correctness.

Recognize the scale at which a hand-rolled linear scan stops being practical, and reach for a dedicated vector database or the built-in file search tool once a collection grows large enough that per-query search time becomes a genuine problem.

Test ranking and filtering logic with small, hand-constructed vectors with known expected outcomes, reserving real embedding calls for a smaller set of tests that confirm actual text produces the expected semantic relationships.