Comparing Vectors With Cosine Similarity

Unit 10's similarity-search pipeline used cosine similarity to compare vectors without a deep dive into why that particular measure was chosen over the alternatives, or how to compute it efficiently once the number of vectors grows past a handful. This lesson fills in that gap: what cosine similarity actually measures geometrically, why it is the standard choice for text embeddings specifically, how it compares to other distance measures, and how to compute it correctly and efficiently in Python.

What Cosine Similarity Measures

Cosine similarity measures the angle between two vectors, ignoring their length (magnitude). Two vectors pointing in exactly the same direction have a cosine similarity of 1, regardless of whether one is twice as long as the other. Two vectors at a right angle score 0. Two vectors pointing in exactly opposite directions score -1.

The formula is the dot product of the two vectors divided by the product of their magnitudes:

cosine_similarity(a, b) = (a · b) / (|a| * |b|)

Why does ignoring magnitude matter for text embeddings? An embedding model represents meaning primarily through the direction a vector points in high-dimensional space, not through how long that vector is. Two paraphrases of the same sentence should point in nearly the same direction even if, incidentally, one embedding vector happens to have a slightly larger norm than the other. Cosine similarity is invariant to that difference in scale, which makes it a measure of "do these mean the same thing" rather than "are these vectors similar in size."

Computing It in Python

import math

def dot_product(a: list[float], b: list[float]) -> float:
    return sum(x * y for x, y in zip(a, b))

def magnitude(v: list[float]) -> float:
    return math.sqrt(sum(x * x for x in v))

def cosine_similarity(a: list[float], b: list[float]) -> float:
    """Return a value in [-1, 1]. 1 means identical direction."""
    denom = magnitude(a) * magnitude(b)
    if denom == 0:
        return 0.0
    return dot_product(a, b) / denom

vec_a = [1.0, 2.0, 0.0]
vec_b = [2.0, 4.0, 0.0]     # same direction as vec_a, different length
vec_c = [0.0, 0.0, 1.0]     # perpendicular to vec_a

print(cosine_similarity(vec_a, vec_b))  # 1.0 — same direction
print(cosine_similarity(vec_a, vec_c))  # 0.0 — unrelated direction

vec_b is exactly twice the length of vec_a but points in the identical direction, and the function correctly returns 1.0 — confirming that magnitude does not affect the result. The denom == 0 guard handles the degenerate case of a zero vector (which has no direction), returning 0.0 rather than raising a division-by-zero error; a zero vector should not normally occur from a real embedding model, but defensive code should not crash on it.

This pure-Python version is useful for understanding the mechanics, but it recomputes both magnitudes and loops in Python on every call, which is slow once you are comparing one query vector against thousands of stored vectors. For anything beyond a small demo, use a numeric library.

import numpy as np

def cosine_similarity_np(a: np.ndarray, b: np.ndarray) -> float:
    denom = np.linalg.norm(a) * np.linalg.norm(b)
    if denom == 0:
        return 0.0
    return float(np.dot(a, b) / denom)

def cosine_similarity_batch(query: np.ndarray, matrix: np.ndarray) -> np.ndarray:
    """Compare one query vector against every row of `matrix` at once.

    `matrix` has shape (n_documents, n_dimensions). Returns an array
    of n_documents similarity scores.
    """
    query_norm = query / np.linalg.norm(query)
    matrix_norms = matrix / np.linalg.norm(matrix, axis=1, keepdims=True)
    return matrix_norms @ query_norm

cosine_similarity_batch is the version that matters in practice: rather than looping over each stored vector and calling cosine_similarity_np one at a time, it normalizes every row of matrix in one vectorized operation and then computes all similarity scores with a single matrix-vector multiplication (@). NumPy executes this in optimized, compiled code rather than the Python interpreter loop, which is often 10 to 100 times faster for large collections — this difference becomes the deciding factor once a corpus has more than a few thousand vectors, well before it reaches the scale that justifies a dedicated vector database (Lesson 6).

Cosine Similarity vs. Other Distance Measures

Why do dot product and cosine similarity sometimes give the same ranking? If every vector in a collection is normalized to length 1 (unit length) before storage, then |a| * |b| in the cosine formula always equals 1, and the cosine formula reduces exactly to the plain dot product. Some vector databases and libraries normalize vectors at insertion time specifically so they can use the cheaper dot product operation internally while still producing cosine-similarity rankings. This is a performance optimization, not a different similarity concept — but it is worth knowing about, because a database's default "similarity metric" setting (dot product vs. cosine vs. Euclidean) needs to match how the vectors were prepared, or rankings will be silently wrong. Lesson 6 revisits this when configuring a real database.

Why not Euclidean distance for text? Euclidean distance is sensitive to vector length, and two embeddings of the same meaning are not guaranteed to have the same magnitude even though they point in nearly the same direction — small magnitude differences would then distort a distance-based ranking. Euclidean distance is more natural when the vectors' actual position in space (not just direction) is meaningful, such as certain geometric or clustering algorithms, but for meaning-based text comparison, cosine similarity is the established default.

Similarity Scores Are Relative, Not Absolute

A cosine similarity of 0.85 between a query and a document does not mean "85% correct" or "85% probability of relevance." It is a relative measure, useful for ranking candidates against each other, not for asserting an absolute, universal correctness threshold.

def rank_by_similarity(query_vec, candidates: dict[str, list[float]]) -> list[tuple[str, float]]:
    """Return (label, score) pairs sorted by similarity, highest first."""
    scored = [(label, cosine_similarity(query_vec, vec)) for label, vec in candidates.items()]
    return sorted(scored, key=lambda pair: pair[1], reverse=True)

def test_rank_by_similarity():
    query = [1.0, 0.0]
    candidates = {
        "close_match": [0.9, 0.1],
        "far_match": [0.0, 1.0],
        "opposite": [-1.0, 0.0],
    }
    ranked = rank_by_similarity(query, candidates)
    assert ranked[0][0] == "close_match"
    assert ranked[-1][0] == "opposite"
    print("PASS: rank_by_similarity orders candidates correctly")

test_rank_by_similarity()

This test never calls the embeddings API — it works entirely with small, hand-constructed vectors, which is the right way to test ranking logic: the correctness of rank_by_similarity does not depend on what a real embedding model produces, only on whether sorting by score works as intended. Whether a real query's top result at 0.42 similarity counts as "relevant enough" for a given application is a threshold decision that has to be calibrated against real, labeled data — exactly the evaluation process covered in Lesson 9 — rather than assumed from the raw number alone. Two different embedding models, or the same model on two different kinds of content, can produce very different "typical" similarity ranges, which is another reason a fixed universal threshold (like "always require similarity > 0.8") is a common source of bugs.

Common Mistakes

• Treating a similarity score as a percentage of correctness. As covered above, cosine similarity is a relative ranking signal calibrated per use case, not a universal confidence percentage.
• Comparing vectors from different embedding models or dimension settings. A vector from text-embedding-4 at 1536 dimensions and one from a different model (or the same model at a different dimensions setting) are not directionally comparable — mixing them produces meaningless scores, or an outright shape mismatch error.
• Recomputing magnitudes inside a tight loop over many comparisons. As shown above, doing this in pure Python for large collections is needlessly slow; vectorized NumPy operations (or a proper vector index, Lesson 6) scale far better.

Best Practices

• Normalize vectors once, at storage time, if the database's similarity metric expects it. This avoids redundant normalization work on every single query.
• Use vectorized batch comparison (NumPy or a vector database) once a collection grows past a few thousand items. Looping with a pure-Python cosine function does not scale and will become a visible bottleneck.
• Calibrate similarity thresholds against labeled examples, never by intuition. A "good enough" cutoff should come from the kind of evaluation described in Lesson 9, applied to the specific model, content type, and query style actually in use.