Handling Missing Information in RAG Prompts

Missing Information Is Normal No document collection answers every question. Users will ask about features you do not offer, details your documents never mention, and situations nobody anticipated. In a RAG system, how the assistant behaves in these moments matters as much as how it answers questions it can answer.

There are only two possible behaviors when information is missing:

Behavior Result The assistant fills the gap A plausible, confident, possibly wrong answer. Users act on it. The assistant recognizes the gap An honest answer that says what is known, what is missing, and what to do next. Language models lean toward the first behavior, because they are trained to be helpful and complete (Lesson 104). This lesson shows how to design prompts and code that reliably produce the second.

Types of Missing Information Different situations need different responses. Distinguishing them is the first step.

Type Example Best Response Not found "Is the internship paid?" and no document mentions payment Say it is not in the documents; offer a next step Partially found "Does the certificate show my grade?" Documents describe certificates but not grades Answer what is known; state what is missing Depends on user details "Can I get a refund?" without purchase date Explain the conditions, or ask for the detail Ambiguous question "When does the Django course start?" with two Django courses Ask a clarifying question Out of scope "What's the best laptop for coding?" in a policy assistant Say it is outside what the assistant covers Retrieval failure The answer exists, but the right chunk was not retrieved Looks like "not found"; fix retrieval, not the prompt The last type is important: from the prompt's point of view, a retrieval failure looks exactly like missing information. When "not found" responses are frequent for questions you know are covered, the cause is retrieval (Lessons 101–102).

Prompting for Honest Gaps A Structured Answer Format Free-text answers make missing information hard to detect in code. A structured response makes it explicit:

Answer the question in using only the sources in .

Return only JSON with these keys:

• "status": one of "answered" – the sources fully answer the question "partial" – the sources answer part of it "not_found" – the sources do not contain the answer "needs_details" – the answer depends on details the user has not provided "ambiguous" – the question could refer to more than one thing in the sources
• "answer": the answer with citations, using only the sources; "" if not_found
• "missing": what information is missing or needed; "" if status is "answered"
• "question_for_user": one short question to ask the user, if status is "needs_details" or "ambiguous"; otherwise ""

Never fill missing information with general knowledge or assumptions. Why It Works Explicit statuses force the model to classify the situation before writing, which makes it less likely to invent an answer. The missing field gives the model a legitimate place to put gaps, so they do not turn into invented sentences. Code can act on the status: show an answer, ask a question, or hand over to support. Example Outputs Partial:

{ "status": "partial", "answer": "Certificates are issued after all modules and the final project are completed [certificates#0].", "missing": "Whether the certificate shows a grade is not stated.", "question_for_user": "" } Needs details:

{ "status": "needs_details", "answer": "Refunds are available within 7 days of purchase if less than 20% of the course is completed [refund-policy#0].", "missing": "The purchase date and course progress.", "question_for_user": "When did you buy the course, and roughly how much of it have you completed?" } The Wording of "I Don't Know" A good "not found" answer is honest and helpful.

Weak Better "I don't know." "I couldn't find information about internship payments in our help documents. Please contact support@gocourse.example, and the team can confirm." "The documents don't say." "Our certificate page explains when certificates are issued, but not whether they show grades. The support team can confirm this for you." Guidelines Say what was searched, such as "our help documents", so the user understands the limit. State what is known if anything relevant was found. Give a next step: a contact, a page, or a question. Do not apologize excessively or blame the user's question. Fixed fallback texts for not_found can be written in code rather than by the model, which guarantees consistent wording and correct contact details.

Handling Missing Information in Code Two decisions are best made in code:

Before calling the model: if retrieval found nothing relevant, skip the model and return the fallback directly. This is cheaper and removes any chance of an invented answer. After calling the model: act on the status, and log gaps so the documentation can be improved. from collections import Counter

FALLBACK = "I couldn't find this in our help documents. Please contact support@gocourse.example."

def respond(question: str, retrieved: list[dict], generate, min_score: float = 0.2) -> dict: relevant = [r for r in retrieved if r["score"] >= min_score] if not relevant: return {"status": "not_found", "answer": FALLBACK, "missing": question, "model_called": False} result = generate(question, relevant) result["model_called"] = True if result["status"] == "not_found": result["answer"] = FALLBACK return result

def fake_generate(question: str, sources: list[dict]) -> dict: """Stands in for the model call, so the logic can be tested without an API.""" if "certificate" in question.lower(): return {"status": "partial", "answer": "Certificates are issued after the final project [certificates#0].", "missing": "whether certificates show grades"} return {"status": "not_found", "answer": "", "missing": "internship stipend amount"}

log = [] for question, retrieved in [ ("Does the certificate show my grade?", [{"id": "certificates#0", "score": 0.45}]), ("Is the internship paid?", [{"id": "internship#0", "score": 0.31}]), ("Do you have a Rust course?", [{"id": "courses#3", "score": 0.08}]), ]: result = respond(question, retrieved, fake_generate) log.append(result) print(question, "->", result["status"], "| model called:", result["model_called"])

print(Counter(entry["status"] for entry in log)) print([entry["missing"] for entry in log if entry["status"] != "answered"]) Output:

Does the certificate show my grade? -> partial | model called: True Is the internship paid? -> not_found | model called: True Do you have a Rust course? -> not_found | model called: False Counter({'not_found': 2, 'partial': 1}) ['whether certificates show grades', 'internship stipend amount', 'Do you have a Rust course?'] Code Explanation The relevance gate respond() keeps only retrieved chunks with a score of at least min_score. For the Rust question, the only result scored 0.08, so the model is never called, and the fallback is returned immediately.

Replacing the model's fallback text When the model returns not_found, the answer is replaced with the fixed FALLBACK text, so wording and contact details are always correct.

fake_generate() A stand-in for the real model call that returns structured results in the format from the prompt above. It lets you test the control flow without API calls. In production, it would call the model and parse the JSON (Lesson 99).

Gap logging Counter summarizes statuses, and the list of missing values shows what users asked for that the documents could not answer: grades on certificates, internship stipends, and a Rust course.

Turning Gaps Into Improvements Logged gaps are a valuable product signal:

Logged Gap Possible Action "whether certificates show grades" Add this detail to the certificate help page "internship stipend amount" Decide on and publish the policy "Do you have a Rust course?" Consider demand for a new course, or add a "courses we offer" page Review gap logs regularly. Each fix reduces future "not found" answers, and the improvements come from real user needs.

Clarifying Questions vs Conditional Answers When an answer depends on information the user has not given, there are two options:

Approach Example Best When Ask a clarifying question "When did you buy the course?" The missing detail changes the answer completely, and the user can easily provide it Give a conditional answer "If you bought it within 7 days and completed less than 20%, you're eligible." The conditions are simple, and the user can apply them without another message Many assistants combine both: a short conditional answer plus a question. Decide which you prefer, and state it in the prompt so behavior is consistent (Lesson 83, Recipes 5 and 6).

Testing Missing-Information Behavior Your evaluation set must include questions the documents cannot answer (Lesson 115):

Test Type Example Expected Status Clearly unanswerable "Do you offer a Rust course?" not_found Closely related but not covered "Does the certificate show my grade?" partial Missing user details "Can I get a refund?" needs_details Ambiguous reference "When does the Django course start?" ambiguous Tempting guess "Is the internship paid?" not_found "Tempting guess" questions are especially important: these are questions where a typical answer exists in general knowledge, so the model is most likely to invent an answer.

Common Mistakes Mistake 1: No Defined Behavior for Missing Information What it is: The prompt describes how to answer, but not what to do when it cannot.

Why it happens: Prompts are written with answerable questions in mind.

What it causes: Invented answers for unanswerable questions.

How to fix it: Define statuses, fallback wording, and next steps.

Mistake 2: Always Calling the Model What it is: Sending a question to the model even when retrieval found nothing relevant.

Why it happens: The pipeline always runs every step.

What it causes: Extra cost and a risk that the model answers from general knowledge.

How to fix it: Use a relevance gate and return the fallback directly.

Mistake 3: Unhelpful Refusals What it is: "I don't know" without context or a next step.

Why it happens: The fallback is written as briefly as possible.

What it causes: Frustrated users and repeated questions.

How to fix it: Say what was searched, what is known, and what the user can do next.

Mistake 4: Treating Retrieval Failures as Missing Documents What it is: Assuming every "not found" means the information does not exist.

Why it happens: Both look the same in the output.

What it causes: Documentation is extended unnecessarily, while the real retrieval problem remains.

How to fix it: For "not found" answers, check whether the information exists in your documents. If it does, improve retrieval.

Mistake 5: Ignoring Gap Logs What it is: Logging "not found" answers but never reviewing them.

Why it happens: Logs are created for debugging only.

What it causes: The same gaps frustrate users repeatedly.

How to fix it: Review gaps regularly and turn them into documentation or product decisions.

Best Practices Classify the situation explicitly. Answered, partial, not found, needs details, or ambiguous.

Give missing information its own field. So gaps never become invented sentences.

Gate the model with retrieval scores. No relevant sources, no model call.

Write fallback text in code. Consistent, correct, and helpful.

Choose between clarifying and conditional answers deliberately. And state the choice in the prompt.

Test with unanswerable and tempting questions. They reveal invention risk.

Use gap logs to improve documentation. Missing information is a signal, not just a failure.

Try It Yourself Exercise 1: Status Classification Write ten questions for your document collection: three answerable, two partial, two needing user details, one ambiguous, and two unanswerable. Run the structured prompt and compare the returned status with your expected status for each.

Exercise 2: Relevance Gate Connect respond() to the retriever from Lesson 101 and a real model call. Try three values of min_score, such as 0.05, 0.2, and 0.4. For each, count how many answerable questions were wrongly rejected and how many unanswerable questions reached the model.

Exercise 3: Gap Report Run 30 realistic questions through your assistant, log every missing value, and group them by topic. Write a short report with the three most common gaps and the documentation change you would make for each.