Build a Research Assistant
The Goal
A research assistant takes a question, breaks it into parts, gathers sources for each part, and writes an answer in which every claim points to the source it came from. The reader can then check any sentence against the page it cites. It is the most demanding project in this section, because it combines planning, retrieval, grounded writing, and verification, and because its usefulness depends on whether the citations are real.
The project draws on the search pattern from Lesson 99, the citation handling from Lessons 155 and 157, and the checking habits from every earlier section. The key discipline is that an answer may cite only sources the assistant actually gathered. Any citation to a source outside that set is an error, and the report says so.
What You Will Build
A research function that:
- splits the question into sub-questions,
- gathers sources for each sub-question, keeping each source once,
- asks the model to answer from the gathered sources, with citations enabled,
- verifies that every citation names a gathered source,
- returns the answer, the sources it used, and any citation problems.
Project Structure
research_assistant/
├── research.py # planning, gathering, answering, and verifying
└── demo.py # a stand-in search index and a stand-in model with one bad citationStep 1: Planning and Gathering
Planning turns one question into parts that can each be searched. The planner here is a fixed function, so the demonstration is predictable, and a production version would ask the model for the parts. Gathering runs each part through the search, and keeps a source once, identified by its address:
"""research.py: plan a question, gather sources, answer from them with citations, and verify the citations."""
from dataclasses import dataclass
MODEL = "claude-sonnet-5-5"
@dataclass(frozen=True)
class Source:
url: str
title: str
text: str
def plan(question: str) -> list[str]:
"""A fixed planner for the demonstration. A production planner asks the model for the parts."""
return [f"{question} - definition", f"{question} - current policy"]
def gather(search, sub_questions: list[str]) -> dict[str, Source]:
sources: dict[str, Source] = {}
for query in sub_questions:
for source in search(query):
sources.setdefault(source.url, source)
return sourcesThe setdefault call keeps the first copy of each source, so a page returned by two searches appears once in the list the model sees, and once in the citations the reader sees.
Step 2: Answering From the Sources
Each gathered source becomes a document block with citations enabled, so the model can cite the passages it relies on. The document blocks are sent with the question, and the answer's text blocks carry the citations:
def answer(client, question: str, sources: dict[str, Source]):
documents = [
{
"type": "document",
"title": source.title,
"source": {"type": "text", "media_type": "text/plain", "data": source.text},
"citations": {"enabled": True},
}
for source in sources.values()
]
response = client.messages.create(
model=MODEL,
max_tokens=800,
messages=[{"role": "user", "content": documents + [
{"type": "text", "text": f"Answer from these sources only, and cite them: {question}"},
]}],
)
return response.contentThe instruction to answer only from the sources is part of the request, and the verification in the next step checks that the instruction was followed in the form that matters: every citation must point to a gathered source.
Step 3: Verifying the Citations
Verification walks the answer's blocks and collects the URL of every citation. A citation to a URL outside the gathered set is recorded as a problem, because it means the answer relies on something the assistant never retrieved, which is the most serious failure a research tool can have:
def verify(content, sources: dict[str, Source]) -> tuple[str, list[str], list[str]]:
parts, cited, problems = [], [], []
for block in content:
if block.type != "text":
continue
parts.append(block.text)
for citation in getattr(block, "citations", None) or []:
if citation.url in sources:
if citation.url not in cited:
cited.append(citation.url)
else:
problems.append(f"citation to a source that was not gathered: {citation.url}")
return "".join(parts), cited, problemsStep 4: A Demonstration
The stand-in search returns two sources for the first sub-question and one of them again for the second. The stand-in model cites the first source correctly and cites a source the search never returned, which exercises the verification:
"""demo.py: a research run with a stand-in search and a stand-in model that cites one unknown page."""
from types import SimpleNamespace
from research import Source, gather, plan, verify, answer
INDEX = {
"refund - definition": [
Source("https://a.example/policy", "Refund policy", "Refunds are available within 14 days."),
Source("https://b.example/faq", "Refund FAQ", "Refund requests go to support."),
],
"refund - current policy": [
Source("https://a.example/policy", "Refund policy", "Refunds are available within 14 days."),
],
}
def search(query: str):
return INDEX.get(query, [])
class ScriptedModel:
def __init__(self):
self.messages = self
def create(self, **request):
citations_ok = [SimpleNamespace(url="https://a.example/policy")]
citations_bad = [SimpleNamespace(url="https://unknown.example/post")]
return SimpleNamespace(content=[
SimpleNamespace(type="text", text="Refunds are available within 14 days. ",
citations=citations_ok),
SimpleNamespace(type="text", text="A blog post also mentions this.",
citations=citations_bad),
])
if __name__ == "__main__":
question = "refund"
sub_questions = plan(question)
sources = gather(search, sub_questions)
content = answer(ScriptedModel(), question, sources)
text, cited, problems = verify(content, sources)
print("sub-questions:", len(sub_questions))
print("sources gathered:", len(sources))
print("answer:", text)
print("cited sources:", cited)
print("problems:", problems)Output:
sub-questions: 2
sources gathered: 2
answer: Refunds are available within 14 days. A blog post also mentions this.
cited sources: ['https://a.example/policy']
problems: ['citation to a source that was not gathered: https://unknown.example/post']How It Works
- Two sub-questions are planned, and the search returns three results between them, of which one page repeats. Gathering keeps two distinct sources.
- The answer contains two text blocks, and the verification reads the citations on each block.
- The first block's citation is to a gathered source, so it is recorded in the cited list, once.
- The second block's citation names a page the search never returned. Verification reports it as a problem, which is exactly the failure a reader would otherwise trust.
- The problem is reported, not hidden. A research answer with a flagged citation should be marked for review, or the unsupported sentence removed, before the report is shared.
Going Live
For live use, the stand-in search is replaced by a real search, which may be the web search tool (Lesson 155) or the application's own index (Lesson 99), and the stand-in model by Anthropic(). The verification is unchanged. With the live model, the most common problem is an answer that states a fact without a citation at all, and a stricter check flags uncited text as well as wrong citations.
Limits
A research assistant can only be as good as its sources. Its planner, its search, and its choice of sources each shape the answer, and the citation check confirms only that the answer points to what was gathered, not that the sources are right. Source quality should be judged separately, by checking the domains and recency of what is gathered, and by asking a person to review answers on questions where the stakes are high.