What a Reasoning Model Actually Does Differently

Every model call so far in this course has followed the same basic shape: the model reads the input and produces output tokens directly, roughly in the order a person would read the response. A reasoning model — gpt-6-astra in this course's lineup, though the pattern applies to any model whose provider labels it a reasoning model — inserts an additional step between reading the input and producing the visible response: it first generates internal reasoning tokens, a private chain of intermediate steps the model uses to work through the problem, and only afterward produces the final answer that's actually returned to you.

This matters practically for three reasons. First, reasoning tokens are real, billed output tokens — they count against your output token cost exactly like the visible response does, even though you don't see them in the returned text by default. Second, more reasoning generally improves performance on tasks that benefit from multi-step deliberation — mathematical proofs, multi-constraint logic puzzles, debugging a subtle error, planning a sequence of tool calls — but does little or nothing for tasks that don't have that structure, like formatting a paragraph or classifying an obviously formal message as formal. Third, reasoning takes real wall-clock time in addition to real money, so applying it indiscriminately to every request in a latency-sensitive application is usually the wrong trade-off.

The reasoning Parameter

Reasoning effort is controlled through a reasoning parameter, and it is important to get its shape right, because it is easy to guess wrong from memory of similar-looking APIs: it is a nested object, not a flat keyword.

# Correct: reasoning is an object with an "effort" key
response = client.responses.create(
    model="gpt-6-astra",
    input="A train leaves station A at 60 mph. Ninety minutes later, a second train leaves the same station on the same track at 90 mph. How far from station A does the second train catch the first?",
    reasoning={"effort": "medium"},
)

# Wrong: this is not a valid keyword argument on responses.create()
response = client.responses.create(
    model="gpt-6-astra",
    input="...",
    reasoning_effort="medium",  # raises a TypeError — not a recognized parameter
)

The reason the parameter is nested rather than flat is that reasoning is designed to carry more than one related setting under a single object — effort level today, and potentially additional reasoning-related controls in future API versions — without requiring a new top-level parameter for each one. This is a common pattern worth recognizing across the SDK generally: related settings that might grow over time tend to be grouped into a single object parameter (you'll see the same shape again with text.format in Unit 6), rather than each becoming its own flat keyword argument.

Effort Levels

The effort key accepts one of several string values, and which values a specific model accepts depends on that model — not every reasoning model supports every level.

Two things about this table matter more than memorizing the exact labels. First, not every model in this course's lineup supports every value — gpt-6-astra, specifically, rejects none with an HTTP 400 error, because as a dedicated reasoning model it is not designed to be run with reasoning fully disabled; if you want zero-reasoning behavior at the lowest cost, the correct move is to switch to a non-reasoning model like gpt-5.6-luna rather than trying to force a reasoning model down to none. Second, the practical difference between adjacent levels (low versus medium, say) is task-dependent and worth testing on your actual workload rather than assumed — a jump from low to high might meaningfully improve accuracy on a genuinely hard logic problem while doing almost nothing for a moderately easy one, at several times the cost.

# This raises an error: gpt-6-astra does not support effort="none"
try:
    response = client.responses.create(
        model="gpt-6-astra",
        input="What is 2 + 2?",
        reasoning={"effort": "none"},
    )
except Exception as e:
    print(f"Failed as expected: {e}")

# Correct approach for a trivial task: use a non-reasoning model instead
response = client.responses.create(
    model="gpt-5.6-luna",
    input="What is 2 + 2?",
)

Reasoning Tokens Are Billed as Output Tokens

This point deserves emphasis because it's the single most common source of surprise cost when developers first work with reasoning models: the internal reasoning tokens a model generates before producing its visible answer are billed at the model's output token rate — the more expensive side of the input/output price asymmetry covered in Unit 1 — even though you never see that reasoning text in the response by default.

You can inspect how many reasoning tokens a given call actually consumed through the response's usage details:

response = client.responses.create(
    model="gpt-6-astra",
    input="Prove that the square root of 2 is irrational.",
    reasoning={"effort": "high"},
)

usage = response.usage
print(f"Output tokens (visible answer): {usage.output_tokens}")
print(f"Reasoning tokens (internal, billed as output): {usage.output_tokens_details.reasoning_tokens}")
print(f"Total output cost basis: {usage.output_tokens}")

It is common, especially at high and above on a genuinely hard problem, for the reasoning token count to substantially exceed the length of the final visible answer — a one-paragraph final proof might be preceded by several thousand tokens of internal exploration, backtracking, and verification that never appear in the output text but are billed exactly as if they had. This is why documentation for reasoning models typically recommends reserving a meaningful token budget — at least 25,000 tokens as a starting point when experimenting with high or above — rather than assuming a short prompt implies a short, cheap response.

def estimate_reasoning_cost(reasoning_tokens: int, visible_output_tokens: int,
                              output_price_per_million: float) -> float:
    """gpt-6-astra output pricing: $50 per million output tokens (reasoning + visible)."""
    total_output_tokens = reasoning_tokens + visible_output_tokens
    return (total_output_tokens * output_price_per_million) / 1_000_000

# A hard proof: modest visible answer, large hidden reasoning cost
cost = estimate_reasoning_cost(reasoning_tokens=4200, visible_output_tokens=180,
                                 output_price_per_million=50)
print(f"Cost for this single request: ${cost:.4f}")

At gpt-6-astra's $50-per-million-output-token rate, 4,200 reasoning tokens alone cost about $0.21 — for a single request whose visible answer might be a short paragraph. Multiply that across even a modest volume of requests, and the reasoning token cost, not the visible response length, dominates the bill. This is the core economic argument for choosing effort level deliberately rather than defaulting to the highest setting out of caution.

When to Use a Reasoning Model, and at What Effort

Use a reasoning model, at medium or above, for: multi-step mathematical or logical problems where an error early in the reasoning chain would invalidate the final answer; debugging tasks that require tracing cause and effect through several layers of a system; planning tasks where the model must sequence several dependent steps correctly (this becomes directly relevant again in Unit 8's function calling and Unit 11's agent-building); and any task where you have observed, through testing, that a non-reasoning model's outputs are inconsistent or wrong on a meaningful fraction of cases that a reasoning model handles reliably.

Use low or minimal reasoning, or skip reasoning models entirely, for: classification tasks with clear category boundaries (Lesson 2 and Lesson 3 of this unit already cover better tools for these — precise instructions and few-shot examples); formatting and extraction tasks where the difficulty is following a pattern, not solving a problem; and any latency-sensitive path in your application where a multi-second reasoning delay is unacceptable to the user experience, such as an interactive chat response a user is actively waiting on.

Do not reach for higher reasoning effort as a first fix for inconsistent output. If a task is producing inconsistent results, the more likely cause, per Lesson 2 of this unit, is an underspecified instruction — increasing reasoning effort on an ambiguous prompt often produces more elaborate internal deliberation about an ambiguity that was never going to be resolved by more thinking, because the model still doesn't know what you actually want. Tighten the instruction first; reach for reasoning effort only once the instruction itself is precise and the remaining difficulty is genuinely about multi-step problem-solving rather than about ambiguity.

Reasoning Effort and Self-Consistency

Lesson 2 of this unit introduced self-consistency — sampling a model multiple times at nonzero temperature and taking the majority answer — as a technique for improving reliability on hard reasoning tasks. It's worth being explicit about how this interacts with the reasoning parameter: self-consistency and reasoning effort address a similar underlying problem (unreliable answers on hard tasks) through different mechanisms, and they can be combined, though doing so multiplies cost by both the effort level and the number of samples.

from collections import Counter

def reasoning_self_consistency(prompt: str, n_samples: int = 5,
                                  effort: str = "medium") -> tuple[str, dict]:
    """Combine reasoning effort with self-consistency sampling for a hard problem."""
    answers = []
    total_reasoning_tokens = 0
    for _ in range(n_samples):
        response = client.responses.create(
            model="gpt-6-astra",
            input=prompt,
            reasoning={"effort": effort},
            temperature=0.7,
        )
        answers.append(response.output_text.strip())
        total_reasoning_tokens += response.usage.output_tokens_details.reasoning_tokens

    tally = Counter(answers)
    majority_answer, count = tally.most_common(1)[0]
    return majority_answer, {
        "agreement": f"{count}/{n_samples}",
        "total_reasoning_tokens": total_reasoning_tokens,
    }

answer, stats = reasoning_self_consistency(
    "Three friends split a restaurant bill unevenly based on what each ordered, "
    "then add an 18% tip on the pre-tip total, split evenly. Given the itemized "
    "orders below, what does each person owe in total?",
    n_samples=5,
    effort="medium",
)
print(f"Majority answer: {answer}")
print(f"Agreement: {stats['agreement']}, total reasoning tokens: {stats['total_reasoning_tokens']}")

In practice, the two techniques are usually not applied at full strength simultaneously, because the combined cost — five samples at high effort, say — grows quickly. A more common pattern is to use a single call at higher reasoning effort for problems where the internal deliberation itself is likely to catch and correct errors (the model reasons its way to a self-consistent answer within one call), and reserve self-consistency sampling across multiple calls at moderate effort for problems where the failure mode is closer to occasional random error than a fixable gap in reasoning depth — the two techniques are complementary tools for related but distinct reliability problems, not a spectrum where you simply pick the strongest combination available.

Reasoning Models Do Not Replace Precise Instructions

It's worth stating directly, because the temptation is real: reasoning effort is not a substitute for the techniques covered earlier in this unit. A reasoning model given a vague instruction will often produce longer, more elaborate internal deliberation, but that deliberation is frequently spent trying to resolve an ambiguity that only the developer, not the model, can actually resolve — no amount of internal reasoning tells the model which output format you wanted if you never specified one. The techniques compose: write precise instructions (Lesson 2), add few-shot examples where format matters (Lesson 3), and apply reasoning effort where the remaining difficulty is genuinely about multi-step problem-solving rather than about instruction ambiguity. Applying reasoning effort to a poorly specified prompt is a common and expensive mistake — it increases cost meaningfully while leaving the actual problem, an underspecified instruction, unaddressed.

Reasoning Models vs. Non-Reasoning Models

It helps to see the two model categories side by side, since the decision between them is one of the first choices you make when starting a new task.

The practical implication of this table is architectural as much as it is a per-request choice: most production applications are better served by treating reasoning models as a specialized tool applied to a specific, identifiable subset of requests, rather than as a universal upgrade applied everywhere gpt-5.6-luna or gpt-5.6-terra was previously used. The cost and latency difference between the two categories is large enough that indiscriminate use of a reasoning model tends to show up quickly in both a monthly bill and a user-facing latency regression.

Truncation Risk: max_output_tokens and Reasoning Budgets

A subtle and easy-to-miss failure mode with reasoning models involves the max_output_tokens parameter, which caps the total number of output tokens a request is allowed to generate. Because reasoning tokens and visible answer tokens draw from the same output token budget, setting max_output_tokens too low on a reasoning-model request can result in the reasoning process consuming the entire budget, leaving no tokens left for the visible answer — the response comes back with an empty or severely truncated answer, which looks like a bug in your code but is actually a budget exhaustion problem.

# Risky: a tight max_output_tokens budget on a hard problem at high effort
response = client.responses.create(
    model="gpt-6-astra",
    input="Solve this multi-step optimization problem: ...",
    reasoning={"effort": "high"},
    max_output_tokens=500,  # may be entirely consumed by reasoning, leaving no room for the answer
)

if response.output_text.strip() == "":
    reasoning_used = response.usage.output_tokens_details.reasoning_tokens
    print(f"Empty answer — likely truncated. Reasoning alone used {reasoning_used} tokens "
          f"against a {500}-token total budget.")

The fix is straightforward once the mechanism is understood: size max_output_tokens generously enough to cover both the expected reasoning token count at your chosen effort level and the expected visible answer length, particularly at high effort and above where reasoning token counts of several thousand are common on genuinely hard problems. This is the concrete reasoning behind the earlier recommendation to reserve at least 25,000 tokens when experimenting with high effort levels — not because the visible answer is expected to be that long, but because the reasoning process needs enough room to complete before the budget runs out.

def size_output_budget(expected_reasoning_tokens: int, expected_answer_tokens: int,
                         safety_margin: float = 1.3) -> int:
    """A generous max_output_tokens that leaves room for both reasoning and the visible answer."""
    return int((expected_reasoning_tokens + expected_answer_tokens) * safety_margin)

budget = size_output_budget(expected_reasoning_tokens=8000, expected_answer_tokens=400)
print(f"Recommended max_output_tokens: {budget}")

response = client.responses.create(
    model="gpt-6-astra",
    input="Solve this multi-step optimization problem: ...",
    reasoning={"effort": "high"},
    max_output_tokens=budget,
)

A Real-World Planning Example

Reasoning effort earns its cost most clearly on tasks with genuine multi-step structure, where an error early in the process invalidates everything downstream. Consider a travel-planning assistant that needs to sequence a multi-city itinerary subject to several interacting constraints — flight availability windows, a fixed total budget, a minimum number of nights in each city, and one city that must be visited before another due to a scheduled conference.

itinerary_prompt = """Plan a 12-day, 3-city itinerary (Lisbon, Porto, Madrid) starting and
ending in Lisbon, subject to these constraints:
- A conference in Madrid requires arrival by day 7 at the latest.
- Minimum 3 nights in each city.
- Total inter-city travel (train or flight) must not exceed 4 segments.
- Total estimated transport cost must stay under $600.

Return a day-by-day itinerary with city, dates, and transport method between cities."""

response = client.responses.create(
    model="gpt-6-astra",
    input=itinerary_prompt,
    reasoning={"effort": "medium"},
    max_output_tokens=6000,
)

print(response.output_text)
print(f"Reasoning tokens used: {response.usage.output_tokens_details.reasoning_tokens}")

This is a good candidate for reasoning effort specifically because the constraints interact — satisfying the Madrid arrival deadline affects how many nights remain available in Lisbon and Porto, which affects whether the 4-segment travel cap is achievable, which affects the total cost — and a wrong early decision (say, scheduling Porto after Madrid) can make the rest of the constraints impossible to satisfy simultaneously, requiring the kind of backtracking and constraint-checking that internal reasoning tokens are well suited to working through before committing to a final answer. Contrast this with a single-constraint task like "list flights under $200 from this data" — no genuine multi-step interaction exists, and a non-reasoning model handles it just as well at a fraction of the cost. This same constraint-satisfaction pattern reappears in Unit 11, where an agent built on the Agents SDK plans a sequence of tool calls to satisfy a multi-step user goal — reasoning effort is frequently the right choice for the planning step in that kind of system, even when the individual tool calls themselves are handled by simpler, non-reasoning logic.

Reasoning Tokens Are Recomputed on Every Turn of a Chained Conversation

Lesson 1 of this unit established that instructions must be re-supplied on every call in a previous_response_id chain, since they are not carried forward automatically. Reasoning has a related, distinct implication worth knowing: the reasoning process for each call in a chain is performed fresh, based on the accumulated conversation available to that call — a reasoning model does not "remember" or reuse the internal reasoning tokens it generated on a previous turn when responding to a new one. Every turn's usage.output_tokens_details.reasoning_tokens therefore reflects the reasoning done for that turn alone, and a long multi-turn conversation using a reasoning model will incur a separate reasoning cost on every single turn, compounding the quadratic cost growth of long conversations already introduced in Unit 1.

first = client.responses.create(
    model="gpt-6-astra",
    input="Here is a dataset description and a modeling question: ...",
    reasoning={"effort": "medium"},
)
print(f"Turn 1 reasoning tokens: {first.usage.output_tokens_details.reasoning_tokens}")

second = client.responses.create(
    model="gpt-6-astra",
    previous_response_id=first.id,
    input="Given that, what would change if the dataset were 10x larger?",
    reasoning={"effort": "medium"},
)
print(f"Turn 2 reasoning tokens: {second.usage.output_tokens_details.reasoning_tokens}")

Both turns incur their own reasoning cost, and because the second turn's input to the model implicitly includes the growing conversation history, its reasoning process has more context to work through, which can mean more reasoning tokens, not fewer, as a conversation progresses — the opposite of what intuition about "the model already figured this out once" might suggest. For a multi-turn application built on a reasoning model, this is a strong argument for keeping conversations focused and for periodically summarizing or resetting context (a technique covered in Unit 4) rather than letting a reasoning-heavy conversation grow indefinitely.

Common Mistakes

Using a flat reasoning_effort keyword instead of the nested reasoning={"effort": ...} object. This raises a TypeError immediately, since reasoning_effort is not a recognized parameter on responses.create(). If you've seen this flat form in older documentation or a different provider's API, it's worth double-checking the current parameter shape rather than assuming it carries over.

Requesting effort="none" on a dedicated reasoning model. Some reasoning models, including gpt-6-astra, reject this outright with an HTTP 400 error, since they are not designed to run with reasoning fully disabled. Use a non-reasoning model instead when you want the lowest possible cost and no reasoning step.

Assuming a short visible answer means a cheap request. Reasoning tokens are invisible in the default response text but fully billed, so a request with a one-line visible answer can still be moderately expensive if it involved several thousand reasoning tokens internally. Always check usage.output_tokens_details.reasoning_tokens before assuming a request was cheap.

Reaching for higher reasoning effort to fix inconsistent output caused by an ambiguous instruction. This increases cost without addressing the root cause; tighten the instruction (Lesson 2) first.

Applying high reasoning effort uniformly across an entire application regardless of task difficulty. This is the reasoning-model equivalent of always using the most expensive model for every request (Unit 2, Lesson 5) — match effort level to actual task difficulty, and route only the genuinely hard requests to higher effort.

Best Practices

Test effort levels empirically on your actual task, rather than assuming high is always better — the practical accuracy gain from one effort level to the next is task-dependent and sometimes small relative to its added cost.

Monitor reasoning_tokens in production, not just visible output length, since it is the dominant cost driver for reasoning-heavy workloads and can vary considerably across seemingly similar requests.

Route by task type, not globally. In an application handling a mix of request types, apply reasoning effort selectively — a routing layer that sends genuinely hard requests to a reasoning model at appropriate effort, and simpler requests to a cheaper non-reasoning model, usually outperforms a single uniform choice applied to every request (this pattern recurs in Unit 12's production-readiness material).

Combine reasoning effort with the ambiguity-removal techniques from earlier in this unit, not as a replacement for them. Precise instructions and few-shot examples reduce the kind of failure that more reasoning cannot fix; reasoning effort addresses genuine multi-step difficulty once the instruction itself is already precise.

Reserve a generous token budget when experimenting with high or xhigh effort, especially during development — a request that appears to fail or truncate unexpectedly at high effort may simply have exhausted an output token limit that didn't account for a large hidden reasoning token count.

Keep reasoning-heavy conversations short and focused, since reasoning cost is recomputed on every turn and grows with accumulated context — for a long-running interaction, consider summarizing or resetting the conversation (Unit 4) rather than letting reasoning cost compound turn after turn.

Note: Effort-level names and their exact behavior are controlled entirely by the model provider and can change between model versions. Treat the effort table in this lesson as a description of this course's model lineup at the time of writing, and confirm current behavior against the provider's model documentation before relying on a specific effort level in a production system, particularly if you're working with a model released after this course.

Additional Mistake and Practice Worth Calling Out

One more mistake deserves its own mention because it's easy to fall into once you've internalized the "reasoning is powerful" lesson: treating reasoning effort as a free accuracy dial with no downside beyond cost. Beyond a certain effort level for a given task, additional reasoning tokens can produce diminishing or even slightly negative returns — a model that reasons for far longer than a problem warrants can occasionally talk itself into an overcomplicated or second-guessed answer on a problem that was actually straightforward, in the same way a person overthinking a simple decision can sometimes arrive at a worse outcome than a quick, confident one. This is not a reason to avoid higher effort levels on genuinely hard problems, but it is a reason to test effort levels against real task difficulty (as this lesson has recommended throughout) rather than defaulting to the maximum on the assumption that more reasoning is strictly better.

The corresponding best practice is to build effort selection into your application logic as a first-class decision, not an afterthought: a simple triage step — even a lightweight, non-reasoning classification call using gpt-5.6-luna to estimate whether an incoming request is "simple" or "complex" — can route each request to an appropriate effort level automatically, capturing most of the accuracy benefit of reasoning models on the requests that actually need it while keeping the bulk of routine traffic on cheaper, faster, non-reasoning paths. This routing pattern is a recurring theme across the remainder of this course, particularly in Unit 12's discussion of production cost and latency management.

def triage_and_route(user_request: str) -> dict:
    """Cheap triage call decides whether a request needs reasoning effort at all."""
    triage = client.responses.create(
        model="gpt-5.6-luna",
        instructions="Classify this request as 'simple' or 'complex'. "
                     "'complex' means it requires multi-step math, logic, or planning. "
                     "Respond with exactly one word.",
        input=user_request,
    )
    complexity = triage.output_text.strip().lower()

    if complexity == "complex":
        final = client.responses.create(
            model="gpt-6-astra",
            input=user_request,
            reasoning={"effort": "medium"},
            max_output_tokens=6000,
        )
    else:
        final = client.responses.create(model="gpt-5.6-luna", input=user_request)

    return {"complexity": complexity, "response": final.output_text}

The triage call itself is cheap — a single short classification on a low-cost model — and its cost is negligible next to the savings from not routing every request through gpt-6-astra by default. This kind of two-tier routing is a practical, easy-to-implement first step toward the more systematic evaluation and cost-management techniques covered later in this course.