Cosine Similarity in Data Mining

Cosine similarity measures how similar two vectors are by looking only at the angle between them, ignoring how long either vector is:

cos(θ) = (x · y) / (||x|| × ||y||)

where x · y is the dot product and ||x||, ||y|| are the vectors' magnitudes (lengths). This lesson works through a full numeric example and, more importantly, demonstrates why ignoring vector length is exactly the right choice for comparing documents of different sizes — the single biggest reason cosine similarity is preferred over Euclidean distance in text mining.

A Worked Example

Represent two short documents as word-count vectors over the vocabulary {data, mining, fun}:

Doc1 = "data mining data"  ->  vector (2, 1, 0)
Doc2 = "data mining fun"   ->  vector (1, 1, 1)

Dot product = (2 x 1) + (1 x 1) + (0 x 1) = 2 + 1 + 0 = 3

||Doc1|| = sqrt(2^2 + 1^2 + 0^2) = sqrt(5) ≈ 2.236
||Doc2|| = sqrt(1^2 + 1^2 + 1^2) = sqrt(3) ≈ 1.732

cos(θ) = 3 / (2.236 x 1.732) = 3 / 3.873 ≈ 0.775

A cosine similarity of 0.775 (out of a maximum of 1.0) reflects that the two documents share two of their three vocabulary words, with Doc1 emphasizing "data" more heavily and Doc2 adding "fun," which Doc1 doesn't mention at all.

Why Ignoring Length Is the Actual Point

Suppose Doc1 were simply repeated twice — "data mining data data mining data" — giving the vector (4, 2, 0) instead of (2, 1, 0). Intuitively, this longer document is still "about the same things" as the original; it just says them twice. Cosine similarity confirms that intuition exactly:

Dot product = (4 x 1) + (2 x 1) + (0 x 1) = 6

||(4,2,0)|| = sqrt(16 + 4 + 0) = sqrt(20) ≈ 4.472

cos(θ) = 6 / (4.472 x 1.732) = 6 / 7.745 ≈ 0.775

The cosine similarity comes out to exactly 0.775 again — identical to the original, unrepeated version. Now compare what plain Euclidean distance does with the same two cases:

Euclidean((2,1,0), (1,1,1)) = sqrt((2-1)^2 + (1-1)^2 + (0-1)^2) = sqrt(1+0+1) = sqrt(2) ≈ 1.414
Euclidean((4,2,0), (1,1,1)) = sqrt((4-1)^2 + (2-1)^2 + (0-1)^2) = sqrt(9+1+1) = sqrt(11) ≈ 3.317

Euclidean distance more than doubles (1.414 → 3.317) purely because the document got longer, even though its actual topic and word proportions never changed. This is precisely why cosine similarity, not Euclidean distance, is the standard choice for comparing text: real documents vary enormously in length, and a similarity measure that penalizes a document for being longer — rather than judging it by what it's actually about — would rank a short, on-topic document as "more different" from a long one than it actually is.

What Cosine Similarity's Range Actually Means for Text

Mathematically, cosine similarity ranges from -1 (exactly opposite vectors) through 0 (perpendicular, no relationship) to 1 (identical direction). In text mining specifically, this range effectively collapses to 0 through 1: word counts and TF-IDF weights can never be negative, so two document vectors can never point in genuinely opposite directions — the worst case is two documents sharing no words at all, which is a dot product of 0 (perpendicular vectors, cosine similarity 0.0), not -1. The -1 end of the scale only becomes reachable with vector representations that do include negative values, such as word embeddings (Word2Vec-style vectors) rather than raw word counts or TF-IDF.

Where Cosine Similarity Is Actually Used

• Document similarity and search — comparing a search query's vector against every document's vector and returning the highest cosine similarities, the mechanism behind the information-retrieval systems covered conceptually elsewhere in this series.
• Recommender systems — comparing a user's preference vector against other users' or items' vectors, exactly the kind of similarity judgment underlying the recommendation examples in the lesson on data mining projects.
• Clustering high-dimensional, sparse data — text and other high-dimensional vectors are frequently mostly zeros (a document uses a tiny fraction of the full vocabulary), and cosine similarity, unlike Euclidean distance, isn't distorted by that sparsity the way a raw magnitude-based comparison would be, since it only cares about the direction the non-zero entries point in.

The trade-off worth remembering is that cosine similarity, like the word-count vectors it's usually computed on, has no concept of meaning: two documents using completely different words for the same topic score low, and two documents using the same word in unrelated senses score however high their raw word overlap happens to produce — a limitation inherited from the vector representation itself (bag-of-words or TF-IDF, covered in the lesson on stemming and text preprocessing), not from the cosine formula.