Zero-Shot Question Answering

What Is Zero-Shot Question Answering? Question answering (QA) is the task of producing an answer to a question. Zero-shot question answering means doing this with a prompt that contains the question (and possibly reference text) but no examples of other questions with their answers.

There are two very different kinds of zero-shot QA, and recognizing which one you are doing is the most important decision in this lesson.

Type Where the Answer Comes From Example Closed-book QA The model's own training knowledge, with no reference text provided What is the difference between TCP and UDP? Open-book (context-based) QA Reference text you provide in the prompt Using the refund policy below, can I get a refund after 10 days? The names come from exams: in a closed-book exam you answer from memory; in an open-book exam you may consult the material.

Why the Difference Matters In closed-book QA, the model relies on what it learned during training. The answer may be outdated, incomplete, or incorrect, and you cannot easily check where it came from. In open-book QA, the answer should come from a specific source you control. The model's job is to read and apply your text, not to recall facts. This is far more reliable for business, policy, and documentation questions. Closed-Book Question Answering When It Works Well Closed-book QA is suitable for stable, widely documented knowledge: programming concepts, scientific principles, definitions, and historical facts.

Basic Prompt What is the difference between a process and a thread? Improved Prompt Explain the difference between a process and a thread for a computer science student preparing for interviews.

Answer in this structure:

1. One-sentence definition of each
2. A comparison table with rows: Memory, Creation Cost, Communication, Failure Impact
3. One real-world example from a web browser

Keep it under 250 words. Example Explanation Audience and purpose (interview preparation) set the depth: precise, but not academic. Structure ensures the answer includes definitions, a direct comparison, and a concrete example. The comparison rows focus the answer on the differences interviewers usually ask about. Where Closed-Book QA Is Risky Question Type Risk Recent events, prices, versions Training data has a cutoff; information may be outdated Niche or specialized facts Less training data; higher chance of invented details Exact figures, dates, citations The model may produce plausible but incorrect values Your company, product, or policies The model does not know them at all Medical, legal, financial specifics Errors can cause real harm Reducing Risk in Closed-Book QA Which Python version introduced the match statement (structural pattern matching)? If you are not certain, say so instead of guessing. Expected output:

The match statement was introduced in Python 3.10. Giving the model permission to express uncertainty (Lesson 10, Tip 3) reduces confidently wrong answers. For important facts, verify the answer against official documentation, which for this example is the "What's New in Python 3.10" page.

Open-Book (Context-Based) Question Answering The Basic Pattern Answer the question using only the information in . If the context does not contain the answer, reply: "The information is not available."

[reference text]

Question: [question] Example Answer the question using only the information in . If the context does not contain the answer, reply: "The information is not available."

GOCOURSE Internship Program

• Duration: 8 weeks
• Eligibility: final-year students in any engineering or science degree
• Mode: remote
• Applications close on 30 November
• Selected candidates complete a 2-week project before joining

Question: Can a second-year student apply? Expected output:

No. The program is open only to final-year students. Example Explanation using only the information in makes the provided text the source of truth. The model should not add facts from general knowledge. The fallback reply gives the model an acceptable answer when information is missing (see the next example). The answer requires reasoning from the context: the context does not mention second-year students directly, but "final-year students" implies they are not eligible. Good open-book QA includes this kind of direct inference. Example: An Unanswerable Question Question: Is the internship paid? Expected output:

The information is not available. The context says nothing about payment. Without the fallback instruction, the model might write something like "Many internships offer a stipend...", which sounds helpful but is not supported by your information.

Designing Open-Book QA Prompts

1. Define How Strictly to Use the Context Different applications need different levels of strictness.

Strictness Instruction Use When Strict Use only the context. Do not use outside knowledge. Policies, legal text, product facts Context first Base your answer on the context. You may add general explanations, but clearly separate them from what the context says. Learning material, documentation help Open Use the context and your general knowledge. Brainstorming, general discussion For most business applications, strict is the right default.

1. Define What Counts as "Not Available" "Not in the context" can be ambiguous. Clarify:

If the context directly states the answer or it follows clearly from the context, answer. If answering would require guessing or outside information, reply "The information is not available." This allows inference like the second-year student example while blocking speculation.

1. Request the Answer Format Need Instruction Short factual answer Answer in one sentence. Yes/no with reason Start with Yes or No, then give the reason in one sentence. Answer with evidence After the answer, quote the sentence from the context that supports it. Machine-readable Return JSON with keys "answer", "found", and "evidence".
2. Ask for Evidence Asking the model to quote the supporting text has two benefits:

Users can see why the answer is correct. Your code can check that the quoted evidence really exists in the context. Open-Book QA With Evidence Checking in Python The following example answers questions from a provided document, returns the supporting quote, and verifies that the quote actually appears in the document.

Requirements: pip install openai and an OPENAI_API_KEY environment variable.

import json

from openai import OpenAI

client = OpenAI() MODEL = "gpt-5.4" # Replace with a model available in your account

INSTRUCTIONS = """Answer the question using only the information in .

Rules:

• If the answer is stated in the context or follows directly from it, answer it.
• If answering would require guessing or outside information, set "found" to false.
• "evidence" must be copied exactly, word for word, from the context.

Return only a JSON object with these keys: {"found": true or false, "answer": "", "evidence": ""}"""

def answer_from_context(context: str, question: str) -> dict: response = client.responses.create( model=MODEL, instructions=INSTRUCTIONS, input=f"\n{context}\n\n\nQuestion: {question}", ) result = json.loads(response.output_text)

if result.get("found") and result.get("evidence", "") not in context: result["found"] = False result["answer"] = "" result["warning"] = "Evidence not found in context; answer rejected." return result

policy = """Refunds are available within 7 days of purchase. A refund is only possible if less than 20% of the course has been completed. Refunds are processed to the original payment method within 5 to 7 business days."""

for question in [ "How long does it take to receive a refund?", "Can I get a refund if I completed 50% of the course?", "Can I transfer my course to a friend?", ]: print(question) print(answer_from_context(policy, question)) print() Possible output:

How long does it take to receive a refund? {'found': True, 'answer': 'Refunds are processed to your original payment method within 5 to 7 business days.', 'evidence': 'Refunds are processed to the original payment method within 5 to 7 business days.'}

Can I get a refund if I completed 50% of the course? {'found': True, 'answer': 'No. Refunds are only possible if less than 20% of the course has been completed.', 'evidence': 'A refund is only possible if less than 20% of the course has been completed.'}

Can I transfer my course to a friend? {'found': False, 'answer': '', 'evidence': ''} Code Explanation INSTRUCTIONS

Restricts answers to the context and defines when inference is allowed. Requires evidence to be an exact copy from the context, which makes it checkable. Defines a JSON structure with a found flag, so code can easily distinguish answered from unanswerable questions. Input structure The context is wrapped in tags and the question follows on its own labeled line, so the model can clearly distinguish the reference text from the question (Lesson 2: instructions or questions placed after a document).

Evidence verification result.get("evidence", "") not in context checks whether the quoted evidence is actually a substring of the context. If the model paraphrased or invented the evidence, the check fails and the answer is rejected. This is a simple, effective guard against unsupported answers.

Why reject instead of trusting In a support or policy application, an answer without real evidence is risky. Rejecting it lets you show a safe fallback, such as "Please contact support", instead of a possibly wrong policy statement.

Note: Exact substring matching is strict. Differences in whitespace or line breaks between the quote and the context can cause a correct answer to be rejected. In production, normalize whitespace in both strings before comparing, and use structured output features (Lesson 99) to make the JSON format reliable.

Handling Different Question Types Yes/No Questions Start your answer with "Yes" or "No", then explain in one sentence based on the context. Leading with the decision makes the answer easy to read and easy for code to check.

Questions With Multiple Parts Question: When do applications close, and is the internship remote? Instruct the model to answer each part separately:

If the question has several parts, answer each part on its own line. If one part is not covered by the context, say so for that part only. This prevents a partly answerable question from being entirely rejected, or worse, from having its missing part filled with guesses.

Questions That Require Calculation Question: If I bought the course on 3 November, until what date can I request a refund? The model must combine the policy (7 days) with the date. For important calculations, it is safer to extract the rule from the context and compute the date in code, or at least ask the model to show the calculation so it can be checked (Lesson 10, Tip 4).

Questions Outside the Context's Topic A user may ask about something unrelated, such as "What is the capital of France?" in a refund-policy assistant. Decide whether your application should answer from general knowledge or decline, and state this in the prompt:

If the question is not about our courses or policies, reply: "I can only answer questions about GOCOURSE courses and policies." Open-Book QA and RAG In this lesson, the context is provided directly in the prompt. In real applications, documents are often too large to include entirely, so the system first retrieves the most relevant passages and then inserts them into a QA prompt like the ones shown here. This approach is called Retrieval-Augmented Generation (RAG), and it is covered in its own section (Lessons 101–106).

The QA prompt design you learned here, which includes strict use of context, fallback answers, and evidence, is the core of every RAG system.

Common Mistakes Mistake 1: Using Closed-Book QA for Company-Specific Questions What it is: Asking a model about your own policies, prices, or products without providing them.

Why it happens: The model answers fluently, so it seems to know.

What it causes: Invented policies that sound plausible and mislead users.

How to fix it: Provide the relevant information as context and restrict answers to it.

Mistake 2: No Fallback for Missing Information What it is: An open-book prompt without instructions for unanswerable questions.

Why it happens: Test questions were all answerable.

What it causes: The model fills gaps with general knowledge or guesses.

How to fix it: Define an exact reply for questions the context does not answer, and test with unanswerable questions.

Mistake 3: Too Strict to Be Useful What it is: Instructions such as "Only answer if the exact words appear in the context."

Why it happens: Overcorrecting for hallucination.

What it causes: Reasonable inferences are rejected, such as concluding a second-year student is ineligible when the context says "final-year students only".

How to fix it: Allow answers that follow directly from the context, and block only answers that require guessing or outside information.

Mistake 4: Trusting the Evidence Without Checking It What it is: Displaying the model's "quote" as proof without verifying it.

Why it happens: The quote looks authoritative.

What it causes: Paraphrased or invented evidence presented as if it came from the source.

How to fix it: Check in code that the evidence appears in the context.

Mistake 5: Mixing Context and Question What it is: Placing the question inside the reference text block or without a label.

Why it happens: Building the prompt by simple concatenation.

What it causes: The model may treat the question as part of the document, or the document as part of the question.

How to fix it: Delimit the context and label the question separately.

Best Practices Decide whether the answer should come from the model or from your text. Use open-book QA for anything specific, recent, or important.

Make the context the source of truth. State clearly that answers must be based on it.

Allow direct inference, block guessing. Define the boundary explicitly.

Always define the "not available" response. And test it with unanswerable questions.

Ask for evidence and verify it. Quotes make answers transparent and checkable.

Handle multi-part, yes/no, and calculation questions explicitly. Each has its own failure modes.

Use structured output in applications. A found flag and separate fields make results easy to process safely.

Verify closed-book answers for important facts. Check against official or authoritative sources.

Try It Yourself Exercise 1: Closed-Book vs Open-Book Ask a model "What is the refund policy for online courses?" without context. Then ask the same question with a short refund policy of your own in a block and a strict instruction. Compare how specific and how reliable each answer is.

Exercise 2: Test Unanswerable Questions Write a context of 5 to 6 facts about an event (date, venue, fee, eligibility). Write 10 questions: 5 answerable, 3 requiring simple inference, and 2 unanswerable. Run them with the strict QA prompt and check that the unanswerable ones return the fallback and the inference ones are answered.

Exercise 3: Extend the Evidence Checker Improve the answer_from_context function:

Normalize whitespace (for example, with " ".join(text.split())) in both the evidence and the context before comparing. Support multi-part questions by changing the JSON format to a list of answers, each with its own found and evidence. Test it with a two-part question where only one part is answerable.