Handle Invalid JSON

When Output Can Still Be Invalid

Structured outputs remove most JSON problems, but not all. According to the documentation, output might not match your schema when a response is a refusal or is cut off at max_tokens. Your own constraints and validators can also reject well-formed output (Lessons 103 and 104). And code paths without structured outputs, such as older prompt-based JSON, can still produce malformed text.

The key idea: diagnose first, then choose the recovery. Retrying blindly wastes tokens, and some failures can never be fixed by retrying.

Strategy 1: Truncated Output

A truncated response is incomplete JSON. Retrying with more room usually solves it. Retry the same messages, without adding the truncated response to the conversation, and double max_tokens, up to a ceiling:

if response.stop_reason == "max_tokens":
    max_tokens = min(max_tokens * 2, MAX_TOKENS_CEILING)

Remember that thinking also uses max_tokens (Lesson 13). If retries keep truncating, the requested output is probably too large: ask for less, or split the task into smaller requests.

Strategy 2: Retry with Feedback

When the JSON is complete but fails your constraints or validators, Claude can usually fix it, if it knows what was wrong. Continue the conversation with:

1. Claude's previous output, as an assistant turn, and
2. a user turn that lists the specific problems and asks for a corrected version.

messages = messages + [
    {"role": "assistant", "content": previous_text},
    {"role": "user", "content": f"That output has problems: {problems}. Return a corrected version."},
]

This is a normal multi-turn conversation that ends with a user message, so it works on current models, which do not support prefill (Lesson 18). The structured output format still applies to the corrected response.

Getting the Text You Need for Feedback

messages.parse() validates the response for you, but when validation fails, you need the raw text to send it back. A flexible pattern is to request the structured output with messages.create(), sending the schema generated from your model with transform_schema() (Lesson 102), and validate the text yourself with model_validate_json():

response = client.messages.create(
    model=MODEL, max_tokens=max_tokens, messages=messages,
    output_config={"format": {"type": "json_schema", "schema": transform_schema(OutputModel)}},
)
text = next(block.text for block in response.content if block.type == "text")
result = OutputModel.model_validate_json(text)      # Raises ValidationError with details

model_validate_json() parses and validates in one step, and runs all constraints and validators, so you get the same checks as messages.parse(), plus full access to the text.

Strategy 3: Refusals

A refusal is a deliberate safety decision, not a formatting problem. Retrying the same request will usually produce the same result. Return a clear "could not process" outcome, log it, and, if your application allows, offer the user an alternative (Lesson 70).

Strategy 4: Repair for Prompt-Based JSON

Without structured outputs (for example, on a platform or in legacy code that requests JSON through the prompt), malformed responses are more likely: code fences around the JSON, explanatory text, or truncation. Lesson 36's extract_json() handles fences and surrounding text. Beyond that, do not attempt clever repairs of broken syntax; they can silently change data. Switch to structured outputs where possible.

Always Bound the Recovery

Every strategy needs limits:

• a maximum number of attempts (for example, 3),
• a ceiling for max_tokens, and
• a clear failure result when recovery does not succeed, so calling code can handle it.

Practical Example: Structured Output with Recovery

This module wraps structured-output requests with all three recovery strategies: more room for truncated output, feedback for validation failures, and a clean stop for refusals. It generates quiz questions with a validator that checks the answer is one of the options.

"""recover.py: structured output with bounded recovery from invalid results."""

from dataclasses import dataclass
from typing import Generic, TypeVar

from anthropic import Anthropic, transform_schema
from pydantic import BaseModel, Field, ValidationError, model_validator

client = Anthropic()
MODEL = "claude-opus-5-5"
T = TypeVar("T", bound=BaseModel)


@dataclass
class Outcome(Generic[T]):
    value: T | None
    status: str          # "ok", "refused", "truncated", or "invalid"
    attempts: int
    detail: str = ""


def describe_errors(error: ValidationError) -> str:
    return "; ".join(f"{'.'.join(str(p) for p in e['loc']) or 'output'}: {e['msg']}" for e in error.errors())


def parse_with_recovery(output_type: type[T], prompt: str, *, max_tokens: int = 1024,
                        ceiling: int = 8192, max_attempts: int = 3) -> Outcome[T]:
    schema = transform_schema(output_type)
    messages = [{"role": "user", "content": prompt}]
    detail = ""

    for attempt in range(1, max_attempts + 1):
        response = client.messages.create(
            model=MODEL, max_tokens=max_tokens, messages=messages,
            output_config={"format": {"type": "json_schema", "schema": schema}},
        )
        if response.stop_reason == "refusal":
            return Outcome(None, "refused", attempt, "Claude declined this request.")
        if response.stop_reason == "max_tokens":
            if max_tokens >= ceiling:
                return Outcome(None, "truncated", attempt, f"Output did not fit in {ceiling} tokens.")
            max_tokens = min(max_tokens * 2, ceiling)
            detail = f"truncated; retrying with max_tokens={max_tokens}"
            continue

        text = next(block.text for block in response.content if block.type == "text")
        try:
            return Outcome(output_type.model_validate_json(text), "ok", attempt, detail)
        except ValidationError as error:
            detail = describe_errors(error)
            messages = messages + [
                {"role": "assistant", "content": text},
                {"role": "user", "content": f"That output has problems: {detail}. "
                                            "Return a corrected version that fixes all of them."},
            ]

    return Outcome(None, "invalid", max_attempts, detail)


class QuizQuestion(BaseModel):
    question: str
    options: list[str] = Field(min_length=4, max_length=4, description="Exactly 4 answer options")
    answer: str = Field(description="The correct option, copied exactly from options")
    explanation: str = Field(description="One sentence explaining why the answer is correct")

    @model_validator(mode="after")
    def answer_is_an_option(self) -> "QuizQuestion":
        if self.answer not in self.options:
            raise ValueError("answer must be exactly one of the options")
        if len(set(self.options)) != len(self.options):
            raise ValueError("options must be different from each other")
        return self


if __name__ == "__main__":
    outcome = parse_with_recovery(
        QuizQuestion,
        "Write one multiple-choice question about Python list slicing for beginners.",
    )
    print(f"status={outcome.status} attempts={outcome.attempts} detail={outcome.detail!r}")
    if outcome.value:
        q = outcome.value
        print(q.question)
        for option in q.options:
            print(f"  {'*' if option == q.answer else ' '} {option}")
        print(f"  ({q.explanation})")

Example output (questions vary; most runs succeed on the first attempt):

status=ok attempts=1 detail=''
What does my_list[1:3] return if my_list = [10, 20, 30, 40]?
    [10, 20]
  * [20, 30]
    [20, 30, 40]
    [30, 40]
  (Slicing includes the start index 1 and stops before the end index 3.)

How It Works

1. Outcome is a generic result: the validated value (or None), a status, the number of attempts, and details. Generic[T] lets type checkers know that parse_with_recovery(QuizQuestion, ...) returns an Outcome[QuizQuestion].
2. transform_schema(output_type) produces the schema to send, once. The removed constraints, such as min_length=4 on options, appear in the description, and are still checked by model_validate_json().
3. Refusals return immediately with status "refused". Retrying would not help.
4. Truncation doubles max_tokens and retries the same messages, without adding the cut-off output, until the ceiling is reached.
5. Validation failures are described field by field with describe_errors(), and sent back to Claude, together with its previous output, as a new user message. The next attempt produces a corrected version.
6. The attempt limit guarantees that the function ends, with status "invalid" and the last problems if no attempt passed validation.
7. QuizQuestion's validator checks rules that a schema cannot express: the answer must match one of the options exactly, and the options must be distinct. If Claude wrote an answer that differed slightly from its option text, the feedback loop would ask it to copy the option exactly.

Common Mistakes

Retrying Everything the Same Way

What happens: refusals are retried pointlessly, and truncation is "fixed" by asking the same question again with the same limit.

Fix: choose the recovery from the diagnosis.

Retrying Without Feedback

What happens: the same validation error repeats.

Fix: include the previous output and the specific problems.

Adding Truncated Output to the Conversation

What happens: the conversation contains broken JSON, which can confuse later attempts.

Fix: retry truncated requests with the original messages and a higher limit.

Unlimited Retries

What happens: a persistent problem burns tokens indefinitely.

Fix: limit attempts and max_tokens, and return a clear failure.

Retrying a "Schema Too Complex" Error

What happens: every attempt fails with the same 400 error.

Fix: simplify the schema (Lesson 108).

Best Practices

• Check stop_reason before parsing, and handle refusals and truncation separately.
• Retry truncated output with more tokens, up to a ceiling.
• Retry validation failures with specific feedback.
• Use model_validate_json() when you need both validation and the raw text.
• Bound every recovery with attempt and size limits.
• Return a result object with status and details, and log failures.

Exercise

1. Call parse_with_recovery() with max_tokens=64 to force truncation. How many attempts does it take, and what max_tokens value finally works?
2. Add a validator to QuizQuestion that rejects explanations longer than 30 words, and observe whether the feedback loop is needed.
3. Make parse_with_recovery() log every attempt (status and detail) with the logging module.
4. Explain why the retry for validation failures adds Claude's previous output as an assistant message rather than including it in the user message.

Step 1: with max_tokens=64, the first attempt is usually cut off. The function doubles the limit on each retry: 64, then 128, then 256. Because max_attempts=3, the run either succeeds within those three attempts or returns "truncated"/"invalid". Current models think before answering, and thinking counts toward max_tokens, so very small limits can leave no room for the answer at all. Start from realistic values such as 1,024.

Step 4: as an assistant message, the previous output is clearly Claude's own earlier answer, and the user message that follows reads naturally as feedback on it: "That output has problems: …". This is the standard multi-turn structure, and it keeps the roles clear. Pasting the output into the user message would mix the user's instructions with model output, which is less clear and can be confused with data that the user provided.