Set Request Timeouts

How Long Is Too Long to Wait?

Every request needs an answer to a simple question: how long should the caller wait before giving up? Without an explicit limit, a request can hang for a long time when the network stalls, the server is slow, or a connection is silently broken. A timeout turns that open-ended wait into a bounded one, and the bounded wait can then be handled like any other error (Lessons 141 and 143).

Timeouts are easy to set and easy to set badly. A value that is too short cuts off valid answers that take longer than expected, especially long generations. A value that is too long lets stuck calls hold resources for minutes. This lesson explains what each timeout measures, what the SDK's defaults are, how to change them for one call or for a whole client, and how to budget time across retries.

What a Timeout Measures

An HTTP request passes through several phases, and each one can be limited separately. The httpx library, which the SDK uses underneath, exposes four settings:

The read timeout is the one that matters most for generation. It measures the gap between received data, not the total time for the whole response. A model that streams steadily never triggers it, even if the whole answer takes minutes. A model that stops sending for longer than the timeout does trigger it.

This distinction matters for long outputs. A generation that takes four minutes but sends data continuously does not time out under a read timeout of one minute, because data keeps arriving. For non-streamed requests, the server sends the response at the end, so the read timeout effectively measures the whole generation time. That is why the SDK default is generous.

The SDK's Defaults

The documentation and the SDK source set these defaults, which apply unless you change them:

Timeouts are retried by default, as Lesson 143 explained. A timeout is an APITimeoutError, which is a kind of connection error, so the SDK's retry policy covers it. With the default of two retries, a request that always times out can keep the caller waiting for three full timeouts plus the backoff between them.

Setting Timeouts

The timeout can be a single number, which sets every phase to that value, or an httpx.Timeout object, which sets each phase separately. Both can be given to the client, so they apply to every call, or to one call with with_options():

import httpx
from anthropic import Anthropic

client = Anthropic(timeout=httpx.Timeout(read=120.0, connect=5.0, write=30.0, pool=10.0))

quick = client.with_options(timeout=20.0)                 # A short limit for one kind of call
long_job = client.with_options(timeout=900.0, max_retries=1)  # More time, fewer retries

with_options() returns a copy of the client with the changed settings. The original client is not modified, so a shared client can keep its defaults while one call path uses different ones.

The example below shows how the settings are stored, so the values can be checked in a test before any request is made:

import httpx


def describe(timeout: httpx.Timeout) -> str:
    return f"read={timeout.read} connect={timeout.connect} write={timeout.write} pool={timeout.pool}"


print(describe(httpx.Timeout(timeout=600.0, connect=5.0)))
print(describe(httpx.Timeout(read=30.0, connect=2.0, write=10.0, pool=5.0)))

Output:

read=600.0 connect=5.0 write=600.0 pool=600.0
read=30.0 connect=2.0 write=10.0 pool=5.0

When a single number is given, the read, write, and pool timeouts all take that value, and the connect timeout is handled as the first example shows. The first line matches the SDK's default: a ten-minute overall limit with a five-second connect limit.

Choosing a Value

No single timeout suits every call. The right value depends on how long the work should take, and on how much waiting the user or the job can tolerate. Two reasonable approaches set the value:

A chat reply that a person is waiting for might have a timeout of about thirty seconds, because a longer wait is worse than a retry. A nightly summary can accept several minutes. The longer timeout is not a sign of a problem, as long as the call is genuinely doing work.

A simple local policy can tie the timeout to the size of the output requested. The relationship below is an application's own assumption, not a figure from the documentation, and it should be checked against measured durations:

def timeout_for(max_tokens: int, seconds_per_thousand_tokens: float = 30.0,
                floor: float = 20.0) -> float:
    """A local policy: allow more time when the answer may be longer."""
    return max(floor, max_tokens / 1000 * seconds_per_thousand_tokens)


print(timeout_for(256))
print(timeout_for(8000))

Output:

20.0
240.0

Timeouts and Retries Together

Timeouts and retries multiply. A request with a timeout of T seconds, retried R times with total backoff B, can take up to (R + 1) × T + B seconds before the caller finally receives an error. That worst case is what the caller must be prepared for, and it is often much larger than the single timeout value suggests.

def worst_case_seconds(timeout: float, retries: int, backoff: float) -> float:
    return (retries + 1) * timeout + backoff


print(worst_case_seconds(timeout=600.0, retries=2, backoff=0.5 + 1.0))
print(worst_case_seconds(timeout=60.0, retries=2, backoff=0.5 + 1.0))

Output:

1801.5
181.5

The first line is the SDK default: almost half an hour in the worst case, which is acceptable for a background job and far too long for a user waiting on a screen. The second line shows a sixty-second timeout, which bounds the same failure to about three minutes.

A Total Time Budget

Per-attempt timeouts do not limit the total time spent on a unit of work, because retries and backoff add to it. Some tasks have a deadline measured from the start, and their timeouts must shrink as the deadline approaches. A deadline object tracks the time left and gives each attempt no more than the remaining budget:

"""deadline.py: give each attempt no more time than the work has left."""


class Deadline:
    def __init__(self, seconds: float, clock):
        self.clock = clock
        self.end = clock() + seconds

    def remaining(self) -> float:
        return max(0.0, self.end - self.clock())

    def attempt_timeout(self, cap: float) -> float:
        return min(cap, self.remaining())


if __name__ == "__main__":
    now = [0.0]
    deadline = Deadline(100.0, clock=lambda: now[0])
    print("first attempt:", deadline.attempt_timeout(cap=60.0))
    now[0] = 70.0
    print("after 70 seconds:", deadline.attempt_timeout(cap=60.0))
    now[0] = 120.0
    print("after the deadline:", deadline.attempt_timeout(cap=60.0))

Output:

first attempt: 60.0
after 70 seconds: 30.0
after the deadline: 0.0

How It Works

1. The deadline is set once, when the work starts, as a fixed end time on a monotonic clock. The clock is injected, so tests control time exactly.
2. Each attempt's timeout is the smaller of its cap and the remaining budget. Early in the work, the cap applies. Later, the budget takes over.
3. After the deadline, the timeout is zero. A caller that sees zero should stop, rather than start an attempt that cannot finish.
4. The budget is shared across retries. If a first attempt uses most of the budget, a retry receives only what is left, which bounds the total time, not only the time per attempt.

Timeouts and Streaming

A streamed response is read in pieces, so the read timeout applies between pieces rather than to the whole answer. A stream can run for many minutes and still be healthy, provided it keeps sending. The timeout matters in the other case, where the stream stalls and nothing arrives. Lesson 57 covered handling stream errors, and the same read timeout is what detects a stalled stream. A stream that times out has already delivered part of its answer, so the partial text is worth keeping, as Lesson 145 explains.

Summary

A timeout bounds how long a request can stall, and it is measured in phases, with the read timeout doing most of the work. The SDK's default is generous, which suits long generations, and it is retried by default, which multiplies the worst case. Timeouts can be set per client or per call, and a deadline shrinks each attempt's budget as the work runs out of time. Choosing the value is a decision about how long a caller can afford to wait, and the worst case with retries is the number to check against that answer.