Embeddings: What They Are & When to Use Them

Unit 10 introduced embeddings and walked through a basic similarity-search pipeline: turn text into vectors, compare vectors, return the closest matches. This unit goes further into practical semantic-search engineering — cleaning and chunking text before embedding, storing vectors in a real database, filtering by metadata, evaluating retrieval quality with the same methodology used in Unit 13, and building two complete applications. Before going further, it is worth sharpening exactly when embeddings are the right tool for a job and when they are not, because that decision shapes every design choice in the lessons that follow.

The Geometric Intuition, Restated Briefly

An embedding is a list of floating-point numbers — a vector — produced by a model that has been trained so that texts with similar meaning end up close together in that vector space, and texts with different meaning end up far apart. Unit 10 covered the "without the maths" version of this. The one addition worth making here: no individual number in the vector corresponds to a human-readable concept like "is about finance" or "has a positive tone." The meaning is encoded in the relationships between the numbers across all dimensions at once. This matters practically because it means you cannot debug a bad embedding by inspecting individual coordinates — you can only evaluate it by measuring how well it separates similar text from dissimilar text, which is exactly the subject of Lesson 9 in this unit.

Three Questions Before Reaching for Embeddings

Before writing any embedding code, answer these three questions honestly. They save far more time than they cost.

1. Does the task actually require matching by meaning, or would exact/fuzzy text matching work?

If a user searches for "invoice #4471" and you have that exact string in your data, a database LIKE query or full-text search index will find it faster, more reliably, and more cheaply than an embedding lookup. Embeddings shine when the query and the matching document use different words for the same idea — "cancel my plan" should match a document titled "How to end your subscription" even though they share almost no words.

2. Is there enough distinguishing text to embed?

Embeddings compare meaning derived from text. A product catalog where every row is {"sku": "A1", "price": 9.99} gives an embedding model almost nothing to work with. A support ticket with three sentences of free-text description gives it plenty. If your content is mostly structured fields and very little prose, structured filtering (Lesson 7 in this unit) will outperform semantic search on its own.

3. Can the application tolerate approximate, ranked results instead of exact answers?

Semantic search returns a ranked list of "probably relevant" items, not a guaranteed correct answer. It is well suited to search, recommendation, and retrieval-for-generation (RAG). It is poorly suited to anything that needs a deterministic, auditable answer, such as "does this customer currently have an active subscription?" — that belongs in a database query, not a vector comparison.

Semantic Search vs. Alternatives

In production systems these are usually combined, not chosen exclusively. A search feature might use semantic search to find candidates and a structured filter to restrict them to "published in the last 30 days" — this combination is the subject of Lesson 7.

When Embeddings Are the Right Tool

• Semantic search over unstructured text — documentation, support tickets, articles, transcripts — where users phrase queries differently than the source text.
• Duplicate or near-duplicate detection — finding support tickets that describe the same underlying issue, or flagging near-identical job listings.
• Recommendation by similarity — "articles like this one," based on content rather than explicit tags.
• Clustering and topic discovery — grouping thousands of free-text responses into themes without predefined categories.
• Retrieval for RAG — fetching the most relevant chunks of a knowledge base to include in a prompt before generation.

When Embeddings Are NOT the Right Tool

• Exact-match lookups (order IDs, email addresses, exact filenames) — use a database index instead; it is faster and always correct.
• Numeric or range queries ("orders over $500 from last week") — this is what SQL WHERE clauses exist for.
• Small, fixed, well-labeled category sets — a simple classifier or even a lookup table will be more accurate and cheaper than nearest-neighbor search over embeddings.
• Tasks requiring multi-step reasoning over the result — embeddings retrieve relevant text; they do not reason about it. If the task is "calculate the total refund owed across these five tickets," retrieval is only the first step, not the answer.
• Situations demanding full explainability of "why did I get this exact match" — cosine similarity scores are useful signals but are not a human-auditable justification the way "this row's status column equals active" is.

A Practical Decision Helper

The three questions above can be encoded as a lightweight heuristic to apply consistently across a codebase or a team, rather than re-litigating the decision informally every time a new feature is proposed.

def recommend_search_strategy(task: dict) -> dict:
    """Suggest a search strategy based on a task description.

    `task` is expected to have boolean/str fields describing the
    matching need. This is a decision aid, not a hard rule — it
    exists to make the reasoning from this lesson explicit and
    reusable in code review.
    """
    needs_meaning_match = task.get("paraphrase_tolerant", False)
    has_enough_text = task.get("avg_text_length_chars", 0) >= 50
    needs_exact_answer = task.get("requires_deterministic_result", False)
    fixed_small_categories = task.get("fixed_category_count", None)

    if needs_exact_answer:
        return {"strategy": "structured_query", "reason": "Deterministic answer required."}

    if fixed_small_categories is not None and fixed_small_categories <= 20:
        return {"strategy": "classification", "reason": "Small, fixed category set."}

    if needs_meaning_match and has_enough_text:
        return {"strategy": "semantic_search", "reason": "Paraphrase-tolerant matching over sufficient text."}

    if needs_meaning_match and not has_enough_text:
        return {"strategy": "structured_query_with_light_text_search",
                "reason": "Meaning matters, but text is too short for reliable embeddings."}

    return {"strategy": "keyword_search", "reason": "No strong signal that meaning-based matching is needed."}

def test_recommend_search_strategy():
    exact_lookup = {"requires_deterministic_result": True}
    assert recommend_search_strategy(exact_lookup)["strategy"] == "structured_query"

    support_search = {"paraphrase_tolerant": True, "avg_text_length_chars": 300}
    assert recommend_search_strategy(support_search)["strategy"] == "semantic_search"

    tag_field = {"paraphrase_tolerant": False, "fixed_category_count": 5}
    assert recommend_search_strategy(tag_field)["strategy"] == "classification"

    print("PASS: recommend_search_strategy covers key branches")

test_recommend_search_strategy()

This function does not call any API — it is a plain decision table expressed in code, which is the point. It walks through the checks in priority order: a deterministic requirement always overrides everything else, a small fixed category set favors classification, and only after ruling those out does it recommend semantic search, and only when there is enough text to make embeddings meaningful. The test uses three representative task descriptions and fake dictionaries rather than real data, confirming each branch produces the expected recommendation. Wiring this kind of check into a design-review checklist prevents a common failure mode: reaching for embeddings by default because they feel modern, on a task a simple WHERE clause would have solved more reliably and far more cheaply.

Common Mistakes

• Defaulting to semantic search for everything. Embeddings add latency, cost, and infrastructure (a vector index, an embedding pipeline) that a keyword or structured query does not need. Reach for embeddings only when Question 1 above is genuinely "yes."
• Embedding data that carries no real semantic content. IDs, dates, and enum values compress poorly into meaningful vectors. Embed the free-text fields; filter on the structured fields directly.
• Treating similarity score as a probability of correctness. A cosine similarity of 0.82 is a relative ranking signal, not a calibrated confidence percentage — Lesson 4 covers this distinction in depth.

Best Practices

• Write down the decision, not just the code. A short comment or design note stating why semantic search was chosen over keyword search for a given feature saves a future maintainer from re-deriving the reasoning.
• Combine techniques instead of picking one. Most production search features layer structured filters, keyword matching, and semantic search together; each compensates for the others' blind spots.
• Prototype cheaply before committing. Because the strategy decision is reversible early on, test semantic search against a small, representative sample of real queries before investing in the full pipeline covered in the rest of this unit.