Generate Typed Data

Generation Versus Extraction

Lesson 106 pulled facts out of a source. Generation creates new content that did not exist before: quiz questions, sample products, test records, practice exercises, or synthetic data for development. The output is still structured, so your application can store and check it, but its correctness comes from the rules you give, not from a source document.

Generated data must be treated with care. A generated quiz answer can be confidently wrong, and generated "customer" records can look real. Generated data should be labeled as generated, and checked before use.

Generating Varied Batches

Asking for many items in one request often produces repetition: similar questions, similar names, similar phrasing. Two techniques reduce this:

1. Split the work by a variable. Ask for a few items per topic, per difficulty, or per persona, instead of one large batch.
2. Remove duplicates in code. Compare normalized text, and discard repeats.

def normalize(text: str) -> str:
    return " ".join(text.lower().split())

seen = set()
unique = []
for item in items:
    key = normalize(item.question)
    if key not in seen:
        seen.add(key)
        unique.append(item)

Duplicates are easy to detect exactly. "Near-duplicates" with different wording need a similarity measure, which is outside this lesson, but the principle of checking in code still applies.

Giving Generation Its Examples

Examples (Lesson 35) matter even more for generation, because they set the style and difficulty. Include one or two real examples in the prompt, marked as examples, and ask for new items that differ from them:

<example>
Question: What does len([1, 2, 3]) return?
</example>

Write 2 new questions in the same style and difficulty, on the topic below.
They must not repeat the example.

Constraints Are Checked in Code

Some rules are better enforced in code than in the schema:

Remember that the SDK removes length constraints from the schema sent to Claude and validates them afterward (Lesson 104). A constraint is therefore enforced by your code, which is why a failed batch raises a validation error rather than returning bad data.

Practical Example: A Quiz Generator

This generator produces quiz questions one topic at a time, validates each question, removes duplicates across the whole set, and keeps only items that pass every check.

"""quiz_generator.py: generate typed quiz questions, then check and deduplicate them."""

from typing import Literal

from anthropic import Anthropic
from pydantic import BaseModel, Field, ValidationError, model_validator

client = Anthropic()
MODEL = "claude-opus-5-5"
TOPICS = ["lists", "dictionaries", "functions", "loops", "strings"]

EXAMPLE = (
    "<example>\n"
    "Question: What does len([1, 2, 3]) return?\n"
    "Options: 2, 3, 4, 6\n"
    "Answer: 3\n"
    "</example>"
)


class Question(BaseModel):
    topic: Literal["lists", "dictionaries", "functions", "loops", "strings"]
    question: str = Field(description="One clear question that ends with a question mark")
    options: list[str] = Field(min_length=4, max_length=4, description="Exactly four distinct options")
    answer_index: int = Field(ge=0, le=3, description="Position of the correct option, from 0 to 3")
    explanation: str = Field(description="One sentence explaining why the answer is correct")

    @model_validator(mode="after")
    def check(self) -> "Question":
        if len(set(self.options)) != len(self.options):
            raise ValueError("options must be distinct")
        if not self.question.strip().endswith("?"):
            raise ValueError("question must end with a question mark")
        return self


class QuizBatch(BaseModel):
    questions: list[Question] = Field(description="Exactly the number of questions requested")


def generate_for_topic(topic: str, count: int) -> list[Question]:
    try:
        response = client.messages.parse(
            model=MODEL,
            max_tokens=4096,
            output_config={"effort": "low"},
            output_format=QuizBatch,
            system="You write multiple-choice quiz questions for beginner Python students. "
                   "Each question has exactly one correct option. Keep questions short and unambiguous.",
            messages=[{"role": "user", "content":
                       f"{EXAMPLE}\n\nWrite {count} new questions about {topic}. "
                       "Vary the question types, and do not repeat the example."}],
        )
    except ValidationError as error:
        print(f"  [{topic}: batch rejected: {error.errors()[0]['msg']}]")
        return []
    if response.stop_reason != "end_turn" or response.parsed_output is None:
        print(f"  [{topic}: incomplete response, stop_reason={response.stop_reason}]")
        return []
    return response.parsed_output.questions


def normalize(text: str) -> str:
    return " ".join(text.lower().split())


def build_quiz(questions_per_topic: int = 2) -> list[Question]:
    candidates = []
    for topic in TOPICS:
        candidates.extend(generate_for_topic(topic, questions_per_topic))

    seen: set[str] = set()
    kept = []
    for question in candidates:
        key = normalize(question.question)
        if key in seen:
            print(f"  [duplicate removed: {question.question!r}]")
            continue
        seen.add(key)
        kept.append(question)
    return kept


if __name__ == "__main__":
    quiz = build_quiz()
    print(f"\nKept {len(quiz)} questions.")
    for number, q in enumerate(quiz[:2], start=1):
        print(f"\n{number}. [{q.topic}] {q.question}")
        for index, option in enumerate(q.options):
            marker = "*" if index == q.answer_index else " "
            print(f"   {marker} {option}")

Expected output (questions and counts vary):

  [duplicate removed: 'What does len("hello") return?']

Kept 9 questions.

1. [lists] What does the append method do to a list?
   * Adds an item to the end of the list
     Removes the first item of the list
     Sorts the list in place
     Returns a copy of the list
...

How It Works

1. Question and QuizBatch describe one question and a batch of questions. The Literal topic restricts values to the topics in TOPICS.
2. model_validator enforces rules the schema cannot: distinct options, and a question that ends with a question mark. Because the SDK validates the response against the original model, a violation raises a validation error, which the generator reports and skips.
3. EXAMPLE shows the style and difficulty. It is marked as an example in tags, so Claude can distinguish it from the request (Lesson 35).
4. One request per topic spreads the batch across subjects, which reduces repetition compared with one large request.
5. effort: "low" keeps the simple generation fast and inexpensive (Lesson 47).
6. Deduplication runs across the whole set, not per topic, so the same question cannot appear twice even when two topics produce overlapping questions.
7. Failures are contained: an incomplete or invalid batch returns nothing for that topic, and the other topics are still kept.

Note: the generated answers are only as correct as the model's knowledge. For a real course, have an expert review generated questions before publishing, and check the correct answers against a reference.

Common Mistakes

Generating One Large Batch

What happens: repetitive content, and one failure loses everything.

Fix: split by topic or variable, and generate in smaller batches.

Trusting Generated Facts

What happens: a plausible but wrong answer is published.

Fix: review, verify against reliable sources, and treat generated content as a draft.

Relying on Length Constraints in the Schema

What happens: lengths or counts are not enforced during generation.

Fix: validate counts and lengths in code, or in the Pydantic model, which validates the response.

Generating Personal or Realistic-Looking Data Carelessly

What happens: fake records look like real people's data, and can be mistaken for it.

Fix: use obviously fictional names and identifiers, and label synthetic datasets.

No Deduplication

What happens: test sets and content catalogs contain the same item several times.

Fix: compare normalized keys across the whole batch.

Best Practices

• Generate in small, varied batches, split by topic or another variable.
• Give one or two examples that set style and difficulty.
• Validate every item with Pydantic models and validators.
• Deduplicate across all batches in code.
• Label generated content, and keep it separate from real data.
• Review before publishing, especially factual content.
• Use lower effort for simple, high-volume generation.

Exercise

1. Add a difficulty: Literal["easy", "medium", "hard"] field, and ask for a mix of difficulties in each batch.
2. Add a validator that rejects questions whose options include "All of the above" or "None of the above".
3. Change build_quiz() to keep generating batches until 10 unique questions are collected, with a maximum of 5 rounds.
4. Explain why the duplicate check compares normalized text instead of the original strings.

Step 3:

def build_quiz(target: int = 10, max_rounds: int = 5) -> list[Question]:
    kept: list[Question] = []
    seen: set[str] = set()
    for _ in range(max_rounds):
        for topic in TOPICS:
            for question in generate_for_topic(topic, 2):
                key = normalize(question.question)
                if key not in seen:
                    seen.add(key)
                    kept.append(question)
        if len(kept) >= target:
            break
    return kept[:target]

The bounded loop guarantees termination even if the model keeps repeating itself, which is why the round limit matters.

Step 4: the same question can differ only in capitalization, spacing, or a trailing space, for example "What does len() return?" and "what does len() return?". Those are the same question to a learner, so comparing normalized text catches them. Comparing the raw strings would keep both.