A Third Option, Alongside Synchronous and Streaming Calls

Lessons 1 through 3 of this unit covered streaming as the solution for keeping an actively-watched, interactive request feeling responsive. There is a distinct, complementary problem this lesson addresses: what happens when a request is expected to take a genuinely long time — well beyond what a typical HTTP connection is designed to stay open for — regardless of whether anyone is watching it in real time at all. Background mode is the API's answer to this: rather than holding a connection open for the full duration of a long-running generation, you submit the request, receive an identifier immediately, and poll (or receive a webhook notification) for the result once it's ready, without needing an open connection the entire time.

job = client.responses.create(
    model="gpt-6-astra",
    input="Write an extremely detailed, 20,000-word technical analysis of...",
    reasoning={"effort": "high"},
    background=True,
)
print(f"Job submitted, ID: {job.id}, status: {job.status}")

Setting background=True changes the call's behavior fundamentally: instead of blocking until the response is ready (as a normal synchronous call does) or streaming events as they arrive (as Lessons 1 through 3 covered), the call returns almost immediately with a job reference in a pending state, and your application is responsible for checking back later to retrieve the actual result once generation has finished.

Why Long-Running Requests Need Their Own Mechanism

A standard HTTP request/response cycle — which underlies both the synchronous and streaming interfaces covered so far — is not well suited to a connection that might need to stay open for many minutes. Network infrastructure between your application and the API (load balancers, proxies, client libraries with their own timeout defaults) commonly imposes connection timeouts well under what a genuinely long generation — a very large document, an extensive multi-step reasoning task at high effort, a bulk analysis — might require to complete. A request that exceeds these intermediate timeouts fails not because the model itself failed, but because some piece of infrastructure between your application and the model gave up waiting before generation actually finished.

Background mode sidesteps this entirely by decoupling job submission from result retrieval: the initial responses.create(..., background=True) call itself completes quickly (it only needs to accept and queue the job, not wait for it to finish), and your application separately, and at its own pace, checks whether the job has completed — an operation that itself is quick regardless of how long the underlying generation is taking.

Checking Job Status and Retrieving Results

Once a job has been submitted in background mode, your application polls its status using the returned job ID, until the job reaches a terminal state.

import time

def wait_for_background_job(job_id: str, poll_interval: float = 5.0, timeout: float = 600.0) -> "Response":
    """Poll a background job until it completes, fails, or the timeout is reached."""
    start = time.time()
    while time.time() - start < timeout:
        job = client.responses.retrieve(job_id)
        if job.status == "completed":
            return job
        elif job.status == "failed":
            raise RuntimeError(f"Background job {job_id} failed: {job.error}")
        elif job.status in ("queued", "in_progress"):
            time.sleep(poll_interval)
        else:
            raise RuntimeError(f"Unexpected job status: {job.status}")
    raise TimeoutError(f"Background job {job_id} did not complete within {timeout} seconds")

job = client.responses.create(
    model="gpt-6-astra",
    input="Produce a comprehensive analysis of...",
    reasoning={"effort": "high"},
    background=True,
)
print(f"Submitted job {job.id}, polling for completion...")
result = wait_for_background_job(job.id)
print(result.output_text)

This polling loop is the essential pattern for background mode: submit, then periodically check back, sleeping between checks to avoid hammering the API with unnecessary status requests. The poll_interval should be chosen based on the expected duration of the job — polling every five seconds is reasonable for a job expected to take a few minutes, but wastefully frequent for a job expected to take hours, where a much longer interval (or, better, a webhook-based notification if the platform supports one) would be more appropriate.

Combining Background Mode with Streaming

A detail worth understanding clearly: background mode and streaming address different problems and are not mutually exclusive in concept, but combining them requires attaching to a background job's stream after the fact, rather than streaming being the mechanism by which you submit the job in the first place.

# Submit in the background, without holding a connection open
job = client.responses.create(
    model="gpt-6-astra",
    input="A very long analysis task...",
    reasoning={"effort": "high"},
    background=True,
)

# Later, once the job is likely progressing, attach to its stream to watch progress
# (the exact mechanism for this — e.g., a dedicated streaming-retrieval call — is
# worth confirming against your SDK version's current support for this combination)

Note: Support for attaching a live stream to an already-submitted background job is a more advanced and platform-version-dependent capability than the basic submit-and-poll pattern this lesson focuses on — confirm current support and the exact method for it against your SDK version's documentation before depending on it, and default to the simpler poll-until-complete pattern above when in doubt, since it is the more universally supported and more predictable approach.

Handling Failures in Background Jobs

A background job can fail for the same reasons any request can fail — an invalid parameter, a content policy violation, a transient server error — but because the failure surfaces asynchronously, your application's error handling needs to check for it explicitly during polling, rather than relying on an exception being raised at the point of submission.

def submit_and_handle(prompt: str, **kwargs) -> str | None:
    try:
        job = client.responses.create(input=prompt, background=True, **kwargs)
    except Exception as e:
        print(f"Job submission itself failed: {e}")
        return None

    try:
        result = wait_for_background_job(job.id)
        return result.output_text
    except RuntimeError as e:
        print(f"Job ran but failed during processing: {e}")
        return None
    except TimeoutError as e:
        print(f"Job did not complete in time: {e}")
        # The job may still complete later — consider checking back rather than
        # assuming it's permanently lost, depending on your application's needs
        return None

Distinguishing submission failure (something wrong with the request itself, caught immediately) from processing failure (the job was accepted but failed while running, discovered only during polling) from a timeout (the job may still be running, just taking longer than your application chose to wait) is important because each calls for a different response: a submission failure usually means the request itself needs correcting before retrying; a processing failure may or may not be worth retrying depending on the cause; and a timeout specifically does not mean the job failed — it means your application stopped waiting, and the job might complete successfully at some point after your polling loop gave up, which matters for whether it's safe to simply resubmit an identical job or whether you risk running the same expensive, long job twice unnecessarily.

Cancelling a Background Job

Because background jobs run independently of any actively-held connection, an application may need to explicitly cancel one that's no longer needed — a user closing a feature before its long-running job completes, or an application-level decision that a queued job is no longer relevant.

def cancel_if_running(job_id: str) -> None:
    job = client.responses.retrieve(job_id)
    if job.status in ("queued", "in_progress"):
        client.responses.cancel(job_id)
        print(f"Cancelled job {job_id}")
    else:
        print(f"Job {job_id} already in a terminal state ({job.status}); nothing to cancel")

Explicit cancellation matters for cost control specifically: a background job that's no longer needed but isn't cancelled will still run to completion and be billed accordingly, even though nothing in your application is waiting for or will use its result — an easy way for cost to leak out of an application that submits background jobs speculatively or on behalf of a user who might navigate away before a job finishes.

When Background Mode Is the Right Choice

Choose background mode when a request is expected to take long enough that a held-open connection risks hitting infrastructure timeouts well before generation naturally completes — very high reasoning effort (Unit 3, Lesson 4) on a genuinely hard problem, an unusually long generation task, or a job whose duration is inherently unpredictable and could occasionally run much longer than typical.

Prefer streaming (Lessons 1 through 3) when a person is actively watching and waiting, and the request's expected duration comfortably fits within a normal held-open connection's lifetime — background mode's poll-based model actively works against the perceived-responsiveness goal streaming is designed to serve, since there's no equivalent of incremental content display while a job sits in a queued or in_progress state between polls.

Prefer a standard synchronous call when a request reliably completes quickly and there's no meaningful risk of hitting a connection timeout — the added complexity of background mode's submit-then-poll pattern isn't justified for a request that would complete well within a normal request's lifetime anyway.

A Worked Example: A Batch Report Generator Using Background Mode

Consider a feature that generates a detailed, lengthy analytical report on demand — the intermediate scenario from Lesson 1, where a user clicks a button and waits, but the expected duration (several minutes, given a large, complex prompt and high reasoning effort) makes background mode, rather than streaming, the more robust choice.

def generate_report_async(topic: str) -> str:
    job = client.responses.create(
        model="gpt-6-astra",
        instructions="Produce a thorough, well-organized analytical report.",
        input=f"Write a detailed report on: {topic}",
        reasoning={"effort": "high"},
        max_output_tokens=20000,
        background=True,
    )
    return job.id

def check_report_status(job_id: str) -> dict:
    job = client.responses.retrieve(job_id)
    return {"status": job.status, "text": job.output_text if job.status == "completed" else None}

A web application built around this pattern would return job.id to the user's browser immediately after generate_report_async() is called, then have the browser poll check_report_status() periodically (via its own lightweight endpoint wrapping this function) to update a progress indicator, without the browser needing to hold a single long-lived connection open for the entire report-generation duration — a more robust architecture than either a plain synchronous call (which risks the browser's own request timing out) or streaming (which would require a persistent connection for a duration that may extend well beyond what's practical to hold open reliably).

Background Jobs Do Not Change Cost, Only Timing

As with streaming (Lesson 1), it's worth being explicit that background mode does not change the cost of a request — the same tokens are generated and billed at the same rate whether a request runs synchronously, streams, or runs in the background. What background mode changes is purely the timing and connection model around when and how your application learns the result is ready, not what the model actually computes or what it costs to compute it.

def compare_cost_across_modes(usage_tokens: int, output_price_per_million: float) -> None:
    """Illustrating that background mode doesn't change the underlying cost calculation."""
    cost = (usage_tokens * output_price_per_million) / 1_000_000
    print(f"Cost is identical regardless of mode: ${cost:.4f}")
    print("Synchronous, streaming, and background all bill the same generated tokens the same way.")

This matters for setting correct expectations when choosing background mode for a cost-sensitive application: the motivation for background mode is entirely about connection reliability and architecture, not about saving money — an application shouldn't expect a discount for choosing background processing, and any cost-optimization strategy (Unit 12 covers this in depth) needs to come from elsewhere, such as prompt caching (Unit 1, Lesson 5) or choosing an appropriately-sized reasoning effort level (Unit 3, Lesson 4), not from the choice of synchronous versus background execution.

Storing Job State for a Multi-User Application

A real application submitting background jobs on behalf of many different users needs to track which job ID belongs to which user's request, exactly analogous to the per-user state tracking Unit 4 covered for conversation memory. A simple table mapping application-level identifiers to job IDs and their last-known status is the typical foundation.

import sqlite3

def init_jobs_table(path: str = "background_jobs.db"):
    conn = sqlite3.connect(path)
    conn.execute("""
        CREATE TABLE IF NOT EXISTS jobs (
            request_id TEXT PRIMARY KEY,
            user_id TEXT NOT NULL,
            job_id TEXT NOT NULL,
            status TEXT NOT NULL DEFAULT 'queued',
            result TEXT
        )
    """)
    return conn

def submit_user_job(conn, request_id: str, user_id: str, prompt: str, **kwargs) -> str:
    job = client.responses.create(input=prompt, background=True, **kwargs)
    conn.execute(
        "INSERT INTO jobs (request_id, user_id, job_id, status) VALUES (?, ?, ?, 'queued')",
        (request_id, user_id, job.id),
    )
    conn.commit()
    return job.id

def poll_and_update(conn, request_id: str) -> dict:
    row = conn.execute("SELECT job_id, status FROM jobs WHERE request_id = ?", (request_id,)).fetchone()
    if not row:
        raise ValueError(f"No job found for request {request_id}")
    job_id, current_status = row

    if current_status in ("completed", "failed"):
        return {"status": current_status}

    job = client.responses.retrieve(job_id)
    if job.status == "completed":
        conn.execute(
            "UPDATE jobs SET status = 'completed', result = ? WHERE request_id = ?",
            (job.output_text, request_id),
        )
        conn.commit()
        return {"status": "completed", "result": job.output_text}
    elif job.status == "failed":
        conn.execute("UPDATE jobs SET status = 'failed' WHERE request_id = ?", (request_id,))
        conn.commit()
        return {"status": "failed"}
    else:
        return {"status": job.status}

This design lets a web application's frontend poll a lightweight endpoint (poll_and_update(conn, request_id), wrapped in whatever web framework the application uses) repeatedly, without re-querying the underlying model API on every single check once a job has already reached a terminal state — the local database record short-circuits redundant API calls for jobs that are already known to be finished, which matters at scale when many users might be polling simultaneously for jobs that completed some time ago.

A Note on Webhook-Based Notification

Polling, as shown throughout this lesson, is the simplest and most universally supported way to learn when a background job completes, but it isn't the most efficient at scale — an application with many outstanding background jobs polling on a fixed interval generates a steady stream of "not done yet" status checks that carry no new information most of the time. Some platforms offer an alternative: registering a webhook URL that the platform calls automatically once a job completes, eliminating the need for your application to poll at all.

# Illustrative only — exact webhook registration and payload verification
# details are platform-specific and should be confirmed against current documentation.
def register_job_webhook(job_id: str, callback_url: str) -> None:
    """Conceptual sketch: some platforms let you register a callback URL
    to be notified automatically once a background job completes, rather
    than requiring your application to poll for status."""
    pass  # actual mechanism depends on the platform's current webhook support

Note: Webhook support, its registration mechanism, and its payload verification requirements (typically involving a signature check to confirm a webhook request genuinely originated from the platform rather than a malicious third party) are platform-version-specific details worth confirming directly against current documentation before building a production feature around them. Where webhook support is available and your application's infrastructure can reliably receive inbound HTTP callbacks, it is generally the more efficient choice for a high-volume background-job workload than polling; where it isn't available or practical for your deployment environment, the polling pattern this lesson covers in depth remains a fully workable, universally supported fallback.

Testing Background Job Logic with a Fake Client

Following the dependency-injection testing pattern used throughout this course, the polling and state-management logic around background jobs can be tested without waiting for real, potentially slow background jobs to actually complete, by substituting a fake client that reports a controlled sequence of statuses.

class FakeJob:
    def __init__(self, id_: str, status: str, output_text: str = "", error=None):
        self.id = id_
        self.status = status
        self.output_text = output_text
        self.error = error

class FakeClientForPolling:
    """Returns a scripted sequence of statuses across successive retrieve() calls,
    simulating a job that takes a few polls to complete."""
    def __init__(self, status_sequence: list[str]):
        self.status_sequence = status_sequence
        self.call_count = 0

    def retrieve(self, job_id: str):
        status = self.status_sequence[min(self.call_count, len(self.status_sequence) - 1)]
        self.call_count += 1
        return FakeJob(job_id, status, output_text="done!" if status == "completed" else "")


def test_wait_for_background_job_eventually_completes():
    fake = FakeClientForPolling(["queued", "in_progress", "in_progress", "completed"])

    def fake_wait(job_id, poll_interval=0.0, timeout=10.0):
        import time
        start = time.time()
        while time.time() - start < timeout:
            job = fake.retrieve(job_id)
            if job.status == "completed":
                return job
            elif job.status == "failed":
                raise RuntimeError("job failed")
        raise TimeoutError()

    result = fake_wait("job_fake_1", poll_interval=0)
    assert result.output_text == "done!"
    assert fake.call_count == 4
    print("PASS: polling logic correctly waits through queued/in_progress states to completion")

test_wait_for_background_job_eventually_completes()

Setting poll_interval=0 in this test avoids any real waiting during the test run, while the scripted status_sequence exercises the exact state-transition logic (queued → in_progress → completed) the real polling loop needs to handle correctly, without needing an actual long-running job to test against.

Comparing All Three Delivery Modes

This table completes the picture this unit has built across its four technical lessons: none of these three modes is universally correct, and a mature application typically uses more than one of them across its different features, choosing per feature based on expected duration, whether a person is actively watching, and how much risk of hitting an infrastructure timeout a given request realistically carries.

Common Mistakes

Using background mode for requests that reliably complete quickly, adding unnecessary polling complexity to a request that would have worked perfectly well as a simple synchronous call.

Polling too aggressively, checking job status far more frequently than the job's expected duration warrants, adding unnecessary load and API calls for information that isn't going to change meaningfully between such closely-spaced checks.

Not distinguishing a timeout in your own polling loop from an actual job failure, and either giving up on a job that may still complete successfully, or resubmitting an expensive job unnecessarily when the original may still be running.

Forgetting to cancel background jobs that are no longer needed, leaving them to run to completion and be billed for, even though no part of the application will ever use their result.

Best Practices

Reserve background mode specifically for requests genuinely at risk of exceeding a normal connection's practical lifetime — very high reasoning effort on hard problems, unusually long generations, or jobs with unpredictable duration — rather than adopting it as a default for anything that merely takes a few seconds longer than typical.

Choose a polling interval proportional to the job's expected duration, and consider an increasing backoff (checking more frequently early, less frequently as a job continues to run) rather than a fixed interval that's either too aggressive for a long job or too sparse for a shorter one.

Handle submission failure, processing failure, and polling timeout as three distinct outcomes, each with its own appropriate response, rather than collapsing them into identical error handling that obscures which situation actually occurred.

Build explicit cancellation into any feature that lets a user or an application decide a background job is no longer needed, to avoid paying for generation whose result will never be used.