Reducing Hallucinations with Clear Instructions and Context

What Is a Hallucination? A hallucination is output from a language model that sounds confident and plausible but is false, unsupported, or invented: a policy that does not exist, a function that is not in a library, a statistic with no source, or a citation to a paper that was never written.

Hallucinations happen because language models generate the most likely continuation of text (Lesson 2). When the model lacks the needed information, the most likely continuation is often a fluent, typical-sounding answer, not "I don't know". The model is not lying; it has no built-in sense of which of its statements are grounded in fact.

Earlier lessons addressed hallucinations in specific settings: grounding answers in documents (Lesson 104), handling missing information (Lesson 105), and reducing unsupported claims (Lesson 106). This lesson brings together the instruction and context techniques that reduce hallucinations in any prompt, and shows how to measure the result.

Common Types of Hallucination Type Example Invented facts "Interns receive a stipend of ₹10,000 per month" when no stipend policy exists Invented sources Citing a documentation page, paper, or law that does not exist Invented technical details A function, parameter, or command that does not exist in a library (Lesson 91) Accepting false premises Answering "Why was the Python course discontinued?" when it was not Outdated information Presenting old prices, versions, or policies as current Overconfident specifics Exact numbers or dates where only approximate or no information exists Filled gaps in structured output Inventing a value because a field required one (Lesson 22) Each type has a recognizable cause, and most can be reduced with the techniques below.

Technique 1: Provide the Facts The single most effective way to reduce hallucinations is to give the model the information it needs instead of relying on its memory (Lesson 6). For facts about your organization, current events, prices, or versions, the model's training data either does not contain the answer or contains an outdated one.

Without Context With Context "What is GOCOURSE's refund policy?" → a typical-sounding invented policy The policy text is provided → the answer comes from it "What's the latest version of the library?" → a version from training data Current release notes are provided → the answer is current RAG automates this for large document collections (Lessons 101–106).

Technique 2: Make "I Don't Know" an Acceptable Answer Many hallucinations happen because the prompt implicitly demands an answer. Give the model explicit permission, and a specific format, for missing information (Lesson 10, Tip 3; Lesson 105):

If the information needed to answer is not in the provided context, reply exactly: "I couldn't find this in the provided information." Do not guess or use typical values. Why the Exact Wording Matters An exact fallback sentence is easy to detect in code, which allows you to measure how often the model abstains, as shown later in this lesson.

Apply the Same Idea to Structured Output If every field must have a value, the model will invent values for missing ones. Allow null (Lesson 99):

"stipend_amount": integer in rupees, or null if not stated Technique 3: Restrict the Knowledge Source State clearly which facts must come only from the provided context (Lesson 104):

For any fact about GOCOURSE (policies, prices, schedules, people, or features), use only the provided context. General programming explanations may use your own knowledge. Naming the categories of facts that must be grounded tells the model exactly where its general knowledge is not allowed, while still allowing it to be helpful elsewhere.

Technique 4: Check the Question's Premises A premise is an assumption built into a question. "Why was the Python course discontinued?" assumes it was discontinued. Models tend to accept premises and answer accordingly, inventing reasons for events that never happened.

Before answering, check whether the question assumes something that the context does not support. If it does, say so politely and correct the assumption instead of answering as asked. Example Question: "Why did GOCOURSE stop offering certificates?"

With premise checking:

I couldn't find any information that GOCOURSE stopped offering certificates. According to the certificate policy, certificates are issued after all modules and the final project are completed. Technique 5: Ask for Uncertainty Where It Matters For tasks where some uncertainty is acceptable, such as general explanations or estimates, ask the model to mark uncertain statements:

If you are not sure about a specific detail, such as a version number, date, or exact function name, say that it should be verified instead of stating it as fact. This is especially useful for technical content: "Check the library's documentation for the exact parameter name" is far better than a confidently invented parameter. Note that models' expressions of confidence are not perfectly reliable, so this technique supplements verification; it does not replace it.

Technique 6: Avoid Prompts That Encourage Invention Some prompt wording pushes the model toward making things up:

Wording That Encourages Invention Safer Wording "Give 10 statistics about online learning in India." "Using only the report provided, list the statistics it contains." "Cite three research papers that support this." "List the sources in the provided documents that support this, if any." "Write a detailed answer about our internship stipend." "Answer using the internship policy provided; if the stipend is not mentioned, say so." "Fill in all fields." "Fill in fields that are stated; use null for the rest." Requests for a fixed number of facts, sources, or details are particularly risky: if fewer real ones exist, the model may invent the rest.

Technique 7: Reduce Complexity and Verify Break complex tasks into steps (Lesson 15), so each step has less room for invention. Extract before answering (Lesson 70): quotes or facts first, then the answer from them. Use settings that reduce randomness for factual tasks, where available (Lesson 120). Verify important claims in a separate step or in code (Lessons 104 and 106). Measuring Hallucination and Over-Refusal Reducing hallucinations creates a trade-off. Strict instructions can make the model refuse questions it could answer. You need to measure both sides:

Metric Definition Problem It Detects Hallucination rate Share of unanswerable questions that received an answer instead of an abstention Invented answers Over-refusal rate Share of answerable questions that received an abstention Unhelpful caution ABSTAIN_MARKERS = ("i couldn't find", "not in the provided", "i don't have that information", "cannot confirm")

def abstained(output: str) -> bool: return any(marker in output.lower() for marker in ABSTAIN_MARKERS)

def abstention_metrics(results: list[dict]) -> dict: unanswerable = [r for r in results if not r["answerable"]] answerable = [r for r in results if r["answerable"]] hallucinated = [r["id"] for r in unanswerable if not abstained(r["output"])] over_refused = [r["id"] for r in answerable if abstained(r["output"])] return { "hallucination_rate": round(len(hallucinated) / len(unanswerable), 2), "over_refusal_rate": round(len(over_refused) / len(answerable), 2), "hallucinated": hallucinated, "over_refused": over_refused, }

results = [ {"id": "q1", "answerable": True, "output": "Refunds are available within 7 days of purchase."}, {"id": "q2", "answerable": True, "output": "I couldn't find this in our help documents."}, {"id": "q3", "answerable": True, "output": "Live classes run on weekends from 7 PM to 9 PM IST."}, {"id": "q4", "answerable": False, "output": "Yes, interns receive a stipend of 10,000 rupees per month."}, {"id": "q5", "answerable": False, "output": "I couldn't find information about a Rust course."}, {"id": "q6", "answerable": False, "output": "The certificate shows your final grade as a percentage."}, {"id": "q7", "answerable": False, "output": "That detail is not in the provided documents."}, ] print(abstention_metrics(results)) Output:

{'hallucination_rate': 0.5, 'over_refusal_rate': 0.33, 'hallucinated': ['q4', 'q6'], 'over_refused': ['q2']} Code Explanation abstained() Checks whether an output contains one of the agreed abstention phrases. This works reliably only if the prompt requires a consistent fallback wording (Technique 2); extend the markers if your fallback sentences vary.

abstention_metrics()

Separates questions known to be answerable from those known to be unanswerable, which requires labeled test cases (Lesson 115). Hallucinations: unanswerable questions that got a real answer: q4 invented a stipend, and q6 invented a detail about grades on certificates. Over-refusals: answerable questions that got an abstention: q2. Reading the Results A hallucination rate of 0.5 means half the unanswerable questions received invented answers, a serious problem for a help assistant. The over-refusal rate shows that tightening the prompt must be done carefully: q2 was already refused although the answer existed, perhaps because of weak retrieval (Lesson 105).

When you apply the techniques in this lesson, measure both rates before and after. A good change lowers the hallucination rate without a large increase in over-refusals.

Common Mistakes Mistake 1: Relying on the Model's Memory for Specific Facts What it is: Asking about your organization, current prices, or recent versions without providing context.

Why it happens: The model sounds knowledgeable.

What it causes: Confident, typical-sounding, wrong answers.

How to fix it: Provide the facts as context or through retrieval.

Mistake 2: Demanding an Answer What it is: Prompts and formats that leave no room for "I don't know" or null.

Why it happens: Complete answers look better.

What it causes: Invented answers and values.

How to fix it: Define an exact fallback and allow nulls.

Mistake 3: Asking for a Fixed Number of Facts or Sources What it is: "List 10 statistics" or "Cite 3 papers".

Why it happens: Numbers make requests concrete.

What it causes: Invented items when fewer real ones exist.

How to fix it: Ask for items from provided sources, "if any", up to a maximum.

Mistake 4: Accepting False Premises What it is: Answering questions built on incorrect assumptions.

Why it happens: The model tries to be helpful with the question as asked.

What it causes: Invented explanations for things that never happened.

How to fix it: Instruct the model to check premises against the context.

Mistake 5: Measuring Only One Side What it is: Tracking hallucinations but not over-refusals, or the reverse.

Why it happens: One metric seems sufficient.

What it causes: Prompts that become unhelpfully cautious, or confidently wrong.

How to fix it: Measure both on labeled answerable and unanswerable questions.

Best Practices Provide the facts the task needs. Context and retrieval beat memory for specific facts.

Make abstention explicit and exact. A fixed fallback sentence and nullable fields.

Name the facts that must be grounded. Organization-specific and current information.

Check premises. Correct false assumptions instead of answering them.

Avoid wording that invites invention. No fixed counts of facts or sources.

Break down, extract, and verify. Less room for invention, more checks.

Measure hallucination and over-refusal together. Improve one without sacrificing the other.

Try It Yourself Exercise 1: Unanswerable Questions Write ten questions about your platform: five answerable from your documents and five unanswerable, including two with false premises. Run them with your current prompt, label the results, and calculate both metrics with abstention_metrics().

Exercise 2: Apply Techniques Add Techniques 2, 3, and 4 to your prompt: an exact fallback, a restricted knowledge source, and premise checking. Run the same ten questions and compare both metrics before and after.

Exercise 3: Rewrite Risky Prompts Find three prompts in your work that ask for a fixed number of facts, sources, or examples, or that require every field to be filled. Rewrite each to reduce invention, and test whether outputs change for inputs where the information is incomplete.