The Default Behavior: Wait, Then Receive Everything at Once

Every call to client.responses.create() you've made so far in this course has followed the same pattern: send a request, wait, and receive the complete response only once the model has finished generating it in full. For a short response — a one-sentence classification, a brief answer — this wait is barely noticeable. For a longer response — a multi-paragraph explanation, a long-form document, a detailed code review — the wait can stretch to many seconds, during which your application has nothing to show the person waiting on the other end except, at best, a generic loading indicator.

import time

start = time.time()
response = client.responses.create(
    model="gpt-5.6-luna",
    input="Write a detailed 800-word explanation of how photosynthesis works.",
)
elapsed = time.time() - start
print(f"Waited {elapsed:.1f} seconds before seeing any output at all.")
print(response.output_text)

Depending on the model and response length, this might take anywhere from a few seconds to significantly longer — and for the entire duration, the calling application has received precisely nothing usable. This is the behavior this lesson motivates changing: not because the non-streaming approach is wrong, but because it creates a specific, well-understood user experience problem that streaming exists to solve.

The Perception Gap: Total Time vs. Perceived Time

The core insight streaming is built around is that total completion time and perceived waiting time are not the same thing, and the gap between them is where a large amount of user experience quality lives. A response that takes eight seconds to fully generate feels dramatically different to a waiting person depending on whether they see nothing at all for eight seconds, or whether they start seeing words appear on screen within the first half-second and watch the rest arrive progressively.

This is a well-established finding in human-computer interaction generally, not something specific to language models: people tolerate a task taking a while far better when they receive continuous evidence that progress is happening, compared to an equivalent or even shorter wait with no visible progress at all. A blank screen for two seconds can feel worse than a progressively filling screen over five, because the blank screen carries an implicit, anxiety-inducing question — is anything happening, or has this silently failed? — that continuously arriving text answers immediately and continuously.

# Non-streaming: nothing visible until the entire response is ready
response = client.responses.create(model="gpt-5.6-luna", input=long_prompt)
display_to_user(response.output_text)  # user has seen nothing until this exact line runs

# Streaming (mechanics covered in Lesson 2): text appears incrementally
for event in client.responses.create(model="gpt-5.6-luna", input=long_prompt, stream=True):
    if event.type == "response.output_text.delta":
        display_incrementally(event.delta)  # user sees each piece as it arrives

The second version's user experiences the same total generation time as the first, but the experience of waiting through it is qualitatively different, because visible progress is arriving the entire time rather than only at the very end.

Where Streaming Matters Most

Streaming is not equally valuable across every use case, and it's worth being precise about where the benefit is largest, since applying it indiscriminately everywhere adds implementation complexity (Lessons 2 and 3 cover the mechanics) without a proportional benefit in every context.

Interactive, conversational interfaces benefit the most. A person actively waiting for a chat response, watching a screen, has their perception of responsiveness dominated by how quickly something appears, not primarily by the total time to completion. This is the single strongest case for streaming, and it's why virtually every consumer-facing chat product streams its responses.

Long-form generation — an essay, a detailed explanation, a long document — benefits nearly as much, for the same reason: the longer a response takes to generate in total, the larger the perception gap between streamed and non-streamed delivery becomes, since there's proportionally more time during which a non-streaming approach would be showing nothing.

Background or batch processing benefits little to not at all. If a request is being processed as part of an automated pipeline with no person actively watching a screen waiting for the specific response — nightly data classification, a scheduled report generation — there is no perceived-responsiveness benefit to streaming, since there's no one perceiving the wait in real time at all. Unit 5, Lesson 4 covers a complementary mechanism, background mode, specifically suited to this kind of workload, which optimizes for a different property (not tying up a connection for a long-running job) rather than for perceived responsiveness.

Short, quick responses benefit least among interactive use cases. A classification task that reliably completes in under a second gains little from streaming, since there's barely any wait to perceive differently in the first place — the added implementation complexity of handling a stream of events, covered in Lessons 2 and 3, is harder to justify for a response so short that non-streaming already feels instantaneous.

Streaming Does Not Change Total Cost or Total Time

It's worth being explicit about what streaming does not do, since it's easy to conflate "feels faster" with "is faster." Streaming does not reduce the total number of tokens generated, does not reduce the total cost of a request (tokens are billed the same way whether delivered all at once or incrementally), and in most cases does not meaningfully reduce the total wall-clock time until the last token is produced. What streaming changes is when your application receives access to each piece of the output relative to when the model produces it — moving that access earlier, piece by piece, rather than making the model itself generate faster or cheaper.

def compare_total_time(prompt: str) -> None:
    """Illustrating that total completion time is comparable whether streamed or not —
    streaming changes when content becomes available, not how long generation takes."""
    import time

    start = time.time()
    response = client.responses.create(model="gpt-5.6-luna", input=prompt)
    non_streaming_total = time.time() - start

    start = time.time()
    full_text = ""
    for event in client.responses.create(model="gpt-5.6-luna", input=prompt, stream=True):
        if event.type == "response.output_text.delta":
            full_text += event.delta
    streaming_total = time.time() - start

    print(f"Non-streaming total time: {non_streaming_total:.2f}s")
    print(f"Streaming total time: {streaming_total:.2f}s")
    print("(Comparable — streaming's benefit is perceived responsiveness, not raw speed.)")

This distinction matters for setting the right expectations when introducing streaming to a team or a stakeholder: streaming is a user-experience improvement, not a performance optimization in the sense of reducing cost or total processing time. Conflating the two can lead to disappointment when a stakeholder expects streaming to make responses "faster" in a literal sense and instead finds total completion time essentially unchanged, just delivered differently.

The Trade-off: Implementation Complexity

Streaming's user-experience benefit comes with a real cost on the implementation side, which is worth naming honestly before the next two lessons dive into the mechanics. A non-streaming call is simple: send a request, receive a complete, well-formed response object, use it. A streaming call requires handling a sequence of incremental events over time, reassembling partial content, handling the specific event types that matter for your use case (Lesson 3), and gracefully handling a connection that's interrupted partway through — none of which is conceptually difficult, but all of which is more code than the equivalent non-streaming call, and more that can go wrong.

This trade-off is exactly why the "where streaming matters most" section above is worth taking seriously as a decision framework, rather than defaulting to streaming everywhere out of a general sense that it's "more modern" or "better." A background classification pipeline processing thousands of independent requests with no one watching a screen gains nothing from streaming and only adds unnecessary implementation complexity; an interactive chat interface gains enormously and the added complexity is well justified by the user experience improvement it buys.

A Concrete Before/After Comparison

To make the perceptual argument tangible rather than abstract, consider a simple measurement: the time to first visible content, as distinct from time to total completion, comparing the same request handled both ways.

import time

def measure_time_to_first_content(prompt: str) -> None:
    # Non-streaming: time to first content equals time to total completion
    start = time.time()
    response = client.responses.create(model="gpt-5.6-luna", input=prompt)
    non_streaming_first_content = time.time() - start
    print(f"Non-streaming: first visible content at {non_streaming_first_content:.2f}s "
          f"(same as total completion time)")

    # Streaming: time to first content can be a small fraction of total completion time
    start = time.time()
    first_content_time = None
    for event in client.responses.create(model="gpt-5.6-luna", input=prompt, stream=True):
        if event.type == "response.output_text.delta" and first_content_time is None:
            first_content_time = time.time() - start
    print(f"Streaming: first visible content at {first_content_time:.2f}s")

For a response that takes several seconds to generate in full, the streaming version's first_content_time is typically a small fraction of its non-streaming counterpart's wait — often under a second, even when total completion takes considerably longer. This gap — between "when does something become visible" and "when is everything finished" — is precisely the metric streaming is designed to improve, and it's a genuinely useful metric to track directly in a production application's monitoring, separately from total request latency, since the two numbers tell you different things about how a user actually experiences a feature.

Setting Up for the Rest of This Unit

This lesson has deliberately stayed at the level of why streaming matters, without yet covering how to actually implement it — that begins in Lesson 2, with the stream=True parameter and the event-based interface it produces. Before moving on, it's worth internalizing the two-part argument this lesson has made: perceived responsiveness and total completion time are different properties, and closing the gap between them for interactive, actively-watched use cases is what streaming buys you, at the cost of a real but manageable increase in implementation complexity that the remaining lessons in this unit will walk through directly.

Streaming Enables Early Cancellation

Beyond perceived responsiveness, streaming provides a second, distinct practical benefit worth naming on its own: because content arrives incrementally, an application (or a user) can decide to stop a generation partway through, before it completes, without needing to wait for the full response to finish first. This is meaningfully different from a non-streaming call, where the only way to "stop" a request already in flight is to abandon it entirely at the network level and discard whatever the model eventually produces, having gained nothing from the partial work already done.

def stream_with_early_stop(prompt: str, stop_condition) -> str:
    """Stop consuming a stream as soon as a condition is met, without waiting
    for the model to finish generating the rest of a response we no longer need."""
    collected = ""
    stream = client.responses.create(model="gpt-5.6-luna", input=prompt, stream=True)
    for event in stream:
        if event.type == "response.output_text.delta":
            collected += event.delta
            if stop_condition(collected):
                stream.close()  # stop consuming further events
                break
    return collected

# Example: stop as soon as the model has produced a complete first sentence
result = stream_with_early_stop(
    "Explain quantum entanglement in detail.",
    stop_condition=lambda text: text.strip().endswith(".") and len(text) > 20,
)
print(result)

This is directly useful for a "stop generating" button in a chat interface — a common, expected feature in consumer chat products — letting a user interrupt a response they've already seen enough of, or one that has gone in a direction they no longer want continued, without waiting for it to run to completion first. It is a capability streaming makes natural and non-streaming makes essentially impossible to offer in a comparably responsive way.

Streaming and the Feeling of "Thinking"

There is a subtler perceptual effect worth naming, beyond the raw responsiveness argument made earlier: the pacing of streamed text — words or phrases appearing progressively, at a pace roughly comparable to reading speed — tends to feel, to a person watching it, closer to watching someone compose a thoughtful answer in real time than to receiving a pre-written document all at once. This is a genuinely subjective, experiential quality rather than a measurable performance metric, but it is a large part of why chat products across the industry have converged on streaming as the default presentation for conversational responses specifically — the incremental arrival of text reinforces the conversational framing of the interaction in a way a single, instantaneous block of text does not, even when the two contain identical content.

This effect is worth being aware of specifically because it can inform interface design choices beyond the raw technical decision to stream or not: some applications deliberately pace the display of streamed content to roughly match natural reading speed, even when the underlying tokens are technically available to display faster, specifically to preserve this perceptual quality. This is a presentation-layer decision separate from the API-level streaming mechanics Lessons 2 and 3 cover, but it's worth knowing the two are related, since a streaming implementation that dumps all buffered tokens onto the screen the instant a network buffer flushes can inadvertently produce a jerky, uneven display rather than the smooth, natural-feeling progression that makes streaming's perceptual benefit strongest.

Streaming in Multi-Step and Agentic Systems

A brief preview worth flagging before Unit 8 (function calling) and Unit 11 (the Agents SDK) cover the relevant mechanics directly: streaming's value compounds in systems that involve multiple sequential steps — a model deciding to call a tool, waiting for that tool's result, then producing a final answer — because without streaming, a user watching such a system would see nothing at all not just during one model call, but across the entire multi-step process, which can take considerably longer in total than any single call within it.

# A preview of the kind of multi-step visibility streaming enables —
# full mechanics covered in Units 8 and 11
def stream_agentic_step(step_name: str, prompt: str) -> str:
    print(f"\n[{step_name}] ", end="", flush=True)
    collected = ""
    for event in client.responses.create(model="gpt-5.6-luna", input=prompt, stream=True):
        if event.type == "response.output_text.delta":
            print(event.delta, end="", flush=True)
            collected += event.delta
    return collected

Streaming each individual step of a multi-step process — showing "searching for flights..." progress, then "comparing options..." progress, then the final recommendation, each streamed incrementally — gives a user continuous evidence that the system is actively working through a genuinely more complex task, rather than leaving them staring at a single loading indicator for the combined duration of several sequential model calls and tool executions. This is a natural extension of this lesson's core argument, applied to a more elaborate system than a single conversational turn, and it's part of why streaming becomes close to essential, rather than merely nice to have, once an application's response involves any meaningful multi-step latency.

A Practical Comparison Table

This table crystallizes the decision this lesson has built toward: streaming is the right choice specifically when a person is actively watching and waiting, and the perceptual and interactive benefits (faster-feeling responses, cancellability, visibility into multi-step work) are worth the added implementation cost the next two lessons will walk through in detail.

Common Mistakes

Assuming streaming makes a request complete faster or cheaper. As covered above, streaming changes when content becomes visible, not the total time or cost of generating it — a common and understandable misconception worth correcting explicitly before building any expectations around streaming as a performance optimization.

Applying streaming uniformly across an entire application, including background and batch workloads with no one actively watching. This adds implementation complexity for no perceptual benefit in contexts where there's no real-time viewer to perceive the improvement, and unnecessarily complicates code that could otherwise use the simpler non-streaming interface.

Underestimating the implementation complexity streaming introduces, and adopting it without planning for the additional considerations Lessons 2 and 3 cover — handling partial content, specific event types, and connection interruptions gracefully all require real engineering attention, not just flipping a stream=True flag and assuming everything else works the same way.

Not measuring time-to-first-content separately from total completion time in a production application, missing the specific metric that actually reflects the user-experience improvement streaming is meant to deliver, and instead only tracking total latency, which streaming does not meaningfully change.

Best Practices

Reserve streaming for interactive, actively-watched use cases — chat interfaces, long-form generation a person is waiting on — where the perceived-responsiveness benefit is largest, and use the simpler non-streaming interface for background and batch workloads where no one is watching a screen in real time.

Set expectations correctly when introducing streaming to a team: it is a user-experience improvement to perceived waiting time, not a reduction in total cost or total generation time, and communicating this distinction clearly avoids later confusion or disappointment.

Track time-to-first-content as a distinct metric from total request latency in any production application using streaming, since it's the number that actually reflects the improvement streaming is meant to deliver, and total latency alone will not show it.

Weigh the added implementation complexity honestly against the actual use case, rather than adopting streaming reflexively for every feature — the next two lessons make the mechanics manageable, but "manageable" is not the same as "free," and the decision to stream should be a deliberate one made for use cases where the trade-off is clearly worthwhile.

Two Realistic Application Scenarios, Contrasted

To ground this lesson's argument in something closer to real product decisions, consider two features a single company might build on top of the same underlying model, where the streaming decision comes out differently for each.

Scenario one: a customer-facing chat assistant embedded in a product's help center. A person types a question and watches the screen, actively waiting for a response, often while simultaneously trying to solve their own problem. Every argument from this lesson applies directly here: the perception gap between streamed and non-streamed delivery is large, the ability to stop a response early is a genuinely expected feature of a modern chat interface, and the total added implementation cost is a reasonable, one-time investment for a feature the company will maintain and iterate on over a long product lifetime. This is close to the canonical case for streaming.

Scenario two: a nightly batch job that generates a short internal summary of each of the day's support tickets, for an analytics dashboard nobody watches in real time. No person is present, actively waiting, while any individual summary is generated — the job runs unattended, overnight, and the results are simply available the next morning when someone opens the dashboard. Here, none of the perceptual arguments for streaming apply: there is no one to perceive faster time-to-first-content, no one who might want to cancel a specific summary partway through, and the simpler non-streaming interface is both easier to implement correctly and easier to reason about for a batch pipeline that likely already has its own retry and error-handling logic (Unit 12) built around simple request/response calls.

The point of this contrast is not that one of these scenarios is more "advanced" or that streaming is simply the better default — it's that the same company, building on the same API, correctly makes two different decisions for two different features, based on whether a real person is actively watching and waiting for that specific response. This is the practical test worth applying to any new feature going forward: is there a person, right now, watching a screen for this response? If yes, streaming is very likely worth its implementation cost. If no, the simpler non-streaming call is very likely the better engineering choice, and reaching for streaming anyway would be complexity added without a genuine beneficiary.

A third, intermediate scenario is worth naming too, since real applications don't always sort neatly into "clearly interactive" or "clearly batch": a feature that generates a longer report on request, where a user clicks a button and then waits — not typing back and forth conversationally, but still actively present and watching for the result to appear. This sits between the two scenarios above, and the right call often comes down to how long the wait typically is. A report that reliably completes in under two seconds gains little from streaming's added complexity, since the wait is already short enough that a simple loading indicator suffices. A report that can take twenty or thirty seconds to fully generate is a much stronger candidate for streaming, even though it isn't a back-and-forth conversation, precisely because the absolute length of the wait — not the conversational shape of the interaction — is what determines how much a person watching that specific screen benefits from seeing progressive content rather than a single, long, unbroken pause.