Paginate API Results

Lists That Do Not Fit in One Response

Lesson 135 listed batches with a single call, and the iterator quietly fetched more pages as needed. That convenience hides a real mechanism. Many endpoints that return lists cannot send every item at once, because a workspace may hold thousands of batches, files, or records. Instead, the service returns one page at a time, together with the information needed to ask for the next one. This technique is called pagination, and understanding it matters for three reasons: it determines how many requests a listing costs, it decides how a program resumes an interrupted scan, and it is the source of subtle bugs when the logic is written by hand.

This lesson explains the cursor-based pattern used by the API's list endpoints, shows how the SDK's automatic iteration works underneath, and builds a paginator that can stop early, count its requests, and resume from a saved position.

Cursors, Not Offsets

There are two common ways to ask for the next page. An offset says "skip the first two hundred items". A cursor says "give me the items that come after this one". The API's list endpoints use cursors, and the difference is significant:

A cursor is an identifier taken from the last item of the previous page. The service uses it to find the next items, however the list has changed since. This stability is why the documentation's list endpoints return identifiers that are then passed back as cursors.

The batch list from Lesson 135 uses the same pattern. Its requests accept limit, which sets the page size, and after_id or before_id, which set the cursor. Each page reports whether more items exist, and the identifiers needed to ask for them.

What a Page Contains

Each page is a response with a list of items and some metadata. The fields to rely on are:

The rule that follows from these fields is simple. Continue while has_more is true, and take the next cursor from last_id. Stop when has_more is false. A program should not infer the end of a list from a short page, or from a page size, because the service decides what each page holds.

The Automatic Iterator

The SDK's list methods return an iterator that performs this loop internally. Iterating over the result fetches each page only when the previous page is exhausted:

for batch in client.messages.batches.list(limit=20):
    print(batch.id)

limit sets how many items each page asks for, and it does not limit the total. The iterator keeps requesting pages until has_more is false. The iterator is lazy, so it requests a page only when its items are needed, which is what makes early stopping cheap.

To see the mechanism clearly, and to add control, a program can implement the same loop by hand. The paginator below does exactly what the SDK does, with a function that fetches one page as its only dependency, so it can be tested without the network:

"""pagination.py: iterate over cursor-based pages, one page at a time."""


def iter_pages(fetch_page, limit: int = 3, start_after: str | None = None):
    """Yield items from every page, fetching the next page only when needed."""
    after = start_after
    while True:
        page = fetch_page(after_id=after, limit=limit)
        for item in page["data"]:
            yield item
        if not page["has_more"] or not page["data"]:
            return
        after = page["last_id"]

The loop yields each item as it is consumed. The next page is requested only after the current one is exhausted, and the request uses the last identifier of the current page as its cursor.

Stopping Early

Often a program needs only the first few items, such as the latest five batches. A lazy paginator makes that cheap, because pages after the first are never requested unless the items are actually consumed. The standard library's islice takes a prefix of an iterator, and the paginator fetches only the pages required to produce that prefix:

from itertools import islice

first_four = list(islice(iter_pages(fetch_page), 4))

Stopping early is more than a convenience. A listing of a very large collection, stopped after a few items, costs one or two requests instead of hundreds, which matters for both time and rate limits (Lesson 142).

Resuming from a Checkpoint

A long scan can be interrupted, by a crash, a deployment, or a timeout. Without a checkpoint, the scan must start again from the beginning, and every item is processed twice. A checkpoint solves this: the program saves the identifier of the last item it finished, and a later run starts after that identifier. The paginator accepts a starting cursor for exactly this purpose.

checkpoint = "batch_05"       # The last item a previous run finished
remaining = list(iter_pages(fetch_page, start_after=checkpoint))

The checkpoint must be saved after each item is processed, not after the whole scan, or an interruption between the two loses the progress. The cost of saving after each item is small, and the benefit is that a restart resumes exactly where the work stopped.

Practical Example: A Paginated Scan with Checkpoints

The example below simulates a service with eight items served in pages of three. It walks the whole list, counts the requests made, stops after four items to show the cost of early stopping, and resumes from a saved checkpoint. The fake service records each request, so the counts in the output are exact.

"""paginated_scan.py: walk, stop, and resume a cursor-based list."""

from itertools import islice

from pagination import iter_pages

ITEMS = [f"batch_{n:02d}" for n in range(1, 9)]
requests_made: list[str | None] = []


def fetch_page(after_id: str | None = None, limit: int = 3) -> dict:
    requests_made.append(after_id)
    start = 0 if after_id is None else ITEMS.index(after_id) + 1
    data = ITEMS[start:start + limit]
    return {
        "data": data,
        "has_more": start + limit < len(ITEMS),
        "last_id": data[-1] if data else None,
    }


if __name__ == "__main__":
    everything = list(iter_pages(fetch_page))
    print(f"full scan: {len(everything)} items, {len(requests_made)} requests")

    requests_made.clear()
    first_four = list(islice(iter_pages(fetch_page), 4))
    print(f"first four: {first_four}, {len(requests_made)} requests")

    requests_made.clear()
    resumed = list(iter_pages(fetch_page, start_after="batch_05"))
    print(f"resumed after batch_05: {resumed}, {len(requests_made)} requests")

Output:

full scan: 8 items, 3 requests
first four: ['batch_01', 'batch_02', 'batch_03', 'batch_04'], 2 requests
resumed after batch_05: ['batch_06', 'batch_07', 'batch_08'], 1 requests

How It Works

1. The full scan makes three requests for eight items in pages of three: three items, three items, and two items. The last page has has_more set to false, so the loop stops without asking for a fourth page.
2. The early stop makes two requests. The first page supplies three items, and the fourth requires the second page. The third page is never requested, because nothing consumed its items.
3. The resumed scan starts after batch_05. The fake service locates that identifier and returns the items after it, in one page, which is why only one request is made.
4. The request log records the cursors. The first request has no cursor, and later requests carry the last identifier of the previous page, which is exactly the documented pattern.
5. The counts are exact, because the fake service records every call. The same counting is how a program can report what a listing cost.

Pitfalls of Hand-Written Pagination

A hand-written paginator is easy to get wrong. A loop that stops when a page comes back short will miss items when the service returns fewer items than requested for its own reasons. A loop that never checks has_more will request an empty page forever. A loop that uses offsets instead of cursors will skip or repeat items when the list changes during the scan. The paginator above avoids all three by relying on has_more, using the last identifier as the cursor, and stopping on an empty page as a safeguard.

Summary

List endpoints return items in pages, and each page carries a cursor for the next. Cursors are stable when the list changes, and offsets are not. The SDK's iterator implements the loop lazily, which makes early stopping cheap, and a hand-written version makes the mechanism explicit and checkpointable. Saving the last processed identifier turns an interrupted scan into a resumable one. The last lesson in this section brings the pieces together into a single reliable call.