Building on Lesson 1's Distinction

Lesson 1 established that retryable errors (429, 500, 503) and connection-level failures (timeouts, network errors) are worth retrying, while client errors (400, 401, 403, 404) aren't — retrying them just reproduces the same failure. This lesson builds the actual retry logic that acts on that distinction: how long to wait between attempts, how many attempts to allow, and how the SDK's own built-in retry behavior relates to logic you might still want to write yourself.

The SDK Retries Some Failures Automatically

The base SDK client already retries a subset of failures — typically connection errors and certain retryable status codes — without any code on your part, following a built-in backoff strategy.

from openai import OpenAI

client = OpenAI(max_retries=3)

response = client.responses.create(model="gpt-5.6-terra", input="Hello")

Note: The exact default number of retries, which specific status codes and error types are retried automatically, and the backoff timing used can vary by SDK version. Confirm the current default retry behavior against your installed SDK version's documentation before assuming a specific number of automatic retries.

This means a meaningful amount of transient-failure handling already happens without you writing anything — a request that fails with a 503 and would have succeeded on a second attempt a moment later is often retried automatically, invisibly, before an exception ever reaches your code at all. Understanding this matters because it changes what additional retry logic is actually worth building on top: mostly, cases the default behavior doesn't cover, or cases where you want more control over the specific policy than the default provides.

Why Backoff Needs to Be Exponential, Not Fixed

A retry loop that waits the same fixed amount of time between every attempt tends to make a transient overload situation worse rather than better — if a service is struggling under load and every failed client retries again after exactly one second, the retries themselves add to the load in a synchronized burst. Exponential backoff — waiting progressively longer between each successive retry — spreads retry attempts out over time instead, giving the underlying condition more room to recover.

import time
import random

def call_with_backoff(client, max_attempts: int = 5, base_delay: float = 1.0):
    for attempt in range(max_attempts):
        try:
            return client.responses.create(model="gpt-5.6-terra", input="Hello")
        except Exception as e:
            if attempt == max_attempts - 1:
                raise
            delay = base_delay * (2 ** attempt) + random.uniform(0, 0.5)
            print(f"Attempt {attempt + 1} failed, retrying in {delay:.1f}s")
            time.sleep(delay)

base_delay * (2 ** attempt) is what produces the exponential growth — 1 second, then 2, then 4, then 8 — and the small random amount added on top (random.uniform(0, 0.5)), known as jitter, prevents many separate clients from retrying at exactly synchronized intervals, which would otherwise recreate the same synchronized-burst problem exponential backoff alone doesn't fully solve.

Only Retrying What's Actually Retryable

This loop, as written so far, retries every exception indiscriminately — exactly the mistake Lesson 1 warned against. A correct version checks Lesson 1's retryable-versus-client-error distinction before deciding to retry at all.

from openai import APIStatusError, APIConnectionError, APITimeoutError

def is_retryable_error(exception) -> bool:
    if isinstance(exception, (APIConnectionError, APITimeoutError)):
        return True
    if isinstance(exception, APIStatusError):
        return exception.status_code in (429, 500, 503)
    return False

def call_with_smart_backoff(client, max_attempts: int = 5, base_delay: float = 1.0):
    for attempt in range(max_attempts):
        try:
            return client.responses.create(model="gpt-5.6-terra", input="Hello")
        except Exception as e:
            if not is_retryable_error(e) or attempt == max_attempts - 1:
                raise
            delay = base_delay * (2 ** attempt) + random.uniform(0, 0.5)
            time.sleep(delay)

This is the meaningful improvement over the naive version: a 400 error from a malformed request now fails immediately, on the first attempt, rather than being retried four more times with no chance of succeeding — exactly the wasted cost and time Lesson 1 flagged as the consequence of not making this distinction.

Setting Explicit Timeouts

Beyond retrying a failed request, it's worth explicitly bounding how long any single attempt is allowed to wait for a response, since a request that hangs indefinitely is a different failure mode than one that fails quickly and can be retried promptly.

client = OpenAI(timeout=30.0)

response = client.responses.create(
    model="gpt-5.6-terra",
    input="Hello",
    timeout=10.0,  # overrides the client-level default for this specific call
)

Note: The exact default timeout, and whether it's a single overall timeout or separate connect/read timeouts, can vary by SDK version. Confirm the current default and configuration options against your installed SDK version's documentation.

Setting an explicit timeout matters especially for a request with a longer expected processing time — Unit 3's higher reasoning-effort settings, or Unit 9's built-in tools performing real work (Code Interpreter execution, a multi-step MCP interaction) — where the appropriate timeout is meaningfully longer than for a simple, fast request, and a single fixed timeout across every kind of request in an application risks being either too short for the slow cases or unnecessarily long for the fast ones.

Retrying an Entire Multi-Step Loop, Not Just One Call

Unit 8, Lesson 3's tool-calling loop and Unit 11's agent runs involve multiple internal calls to the model, not just one — retry logic applied naively at the level of the whole loop can end up repeating steps that already succeeded, which is wasteful and, for a tool with a real side effect, potentially harmful (retrying an entire loop that already successfully issued a refund, for instance, risking issuing it twice).

def run_single_step_with_retry(client, input_messages, tools):
    return call_with_smart_backoff_for_response(client, input_messages, tools)

def call_with_smart_backoff_for_response(client, input_messages, tools, max_attempts=3, base_delay=1.0):
    for attempt in range(max_attempts):
        try:
            return client.responses.create(model="gpt-5.6-terra", input=input_messages, tools=tools)
        except Exception as e:
            if not is_retryable_error(e) or attempt == max_attempts - 1:
                raise
            time.sleep(base_delay * (2 ** attempt))

Applying retry logic at the level of a single model call within the loop, rather than around the entire multi-step loop, means a transient failure on step three of a five-step interaction only retries step three, not steps one and two that already completed successfully — a meaningfully safer and less wasteful granularity for retry logic in any multi-step system.

Common Mistakes

Retrying every exception indiscriminately, rather than checking Lesson 1's retryable-versus-client-error distinction first, wasting time and cost on requests that will never succeed no matter how many times they're retried.

Using a fixed delay between retries instead of exponential backoff, risking a synchronized retry burst that makes an already-overloaded service worse rather than better.

Omitting jitter from an exponential backoff implementation, allowing many clients to retry at exactly synchronized intervals even with growing delays.

Wrapping retry logic around an entire multi-step loop or agent run, risking repeated execution of steps — including consequential tool calls — that already succeeded on an earlier attempt.

Relying entirely on the SDK's default automatic retry behavior without setting an explicit timeout, leaving a request that hangs indefinitely with no bound on how long a single attempt is allowed to take.

Best Practices

Check whether a failure is actually retryable before retrying it, using Lesson 1's status-code distinction to avoid wasting attempts on client errors.

Use exponential backoff with jitter for any custom retry logic, spreading retry attempts out over time rather than risking a synchronized burst.

Set explicit timeouts appropriate to the expected duration of a specific kind of request, rather than relying on one fixed timeout across requests with very different expected processing times.

Apply retry logic at the granularity of a single model call within a multi-step loop, rather than around the entire loop, to avoid re-executing steps — especially consequential tool calls — that already succeeded.