Create a Message Batch

Building a Batch You Can Trust

Lesson 131 submitted a two-request batch. Real batches contain thousands of requests built from data, and a mistake in any of them is expensive: a rejected batch wastes hours, and a request with a bad custom_id may be discovered only when the batch ends. The batch API validates request contents asynchronously, so the only fast checks are the ones you run yourself.

This lesson builds a reusable batch builder. It:

1. creates requests from your data, in one consistent shape,
2. validates them locally against the documented rules,
3. splits them into batches that stay within the limits, and
4. submits each batch and stores its ID.

The Documented Rules

These limits are the API's. Leave some margin below them in your own code, so that a small estimate error does not produce a rejected batch.

Requests Are Plain Dictionaries

The SDK's Request and MessageCreateParamsNonStreaming types are typed dictionaries, so at runtime a request is an ordinary dictionary with the same shape as the JSON in the documentation. Building them as dictionaries makes them easy to validate, serialize, and store:

{
    "custom_id": "review-001",
    "params": {
        "model": "claude-opus-5-5",
        "max_tokens": 256,
        "messages": [{"role": "user", "content": "Classify this review: ..."}],
    },
}

Practical Example: A Batch Builder with Local Checks

The module below validates requests, estimates their size, and splits them into safe batches. The demonstration runs offline, so you can see the checks in action before spending anything.

"""batch_builder.py: build, validate, and split message batches."""

import json
import re

MODEL = "claude-opus-5-5"
CUSTOM_ID = re.compile(r"^[a-zA-Z0-9_-]{1,64}$")
API_MAX_REQUESTS = 100_000
API_MAX_BYTES = 256 * 1024 * 1024
SAFE_MAX_REQUESTS = 50_000             # Margin below the API limits
SAFE_MAX_BYTES = 200 * 1024 * 1024


def make_request(custom_id: str, prompt: str, max_tokens: int = 256) -> dict:
    return {
        "custom_id": custom_id,
        "params": {"model": MODEL, "max_tokens": max_tokens,
                   "messages": [{"role": "user", "content": prompt}]},
    }


def validate(requests: list[dict]) -> list[str]:
    problems: list[str] = []
    seen: set[str] = set()
    for request in requests:
        custom_id = request.get("custom_id", "")
        if not CUSTOM_ID.match(custom_id):
            problems.append(f"invalid custom_id {custom_id!r}")
        if custom_id in seen:
            problems.append(f"duplicate custom_id {custom_id!r}")
        seen.add(custom_id)
        params = request.get("params", {})
        if params.get("max_tokens", 0) < 1:
            problems.append(f"{custom_id}: max_tokens must be at least 1")
        if params.get("stream"):
            problems.append(f"{custom_id}: streaming is not supported in batches")
        if not params.get("messages"):
            problems.append(f"{custom_id}: no messages")
    return problems


def size_in_bytes(request: dict) -> int:
    return len(json.dumps(request, ensure_ascii=False).encode("utf-8"))


def split_into_batches(requests: list[dict], max_requests: int = SAFE_MAX_REQUESTS,
                       max_bytes: int = SAFE_MAX_BYTES) -> list[list[dict]]:
    batches: list[list[dict]] = []
    current: list[dict] = []
    current_bytes = 0
    for request in requests:
        size = size_in_bytes(request)
        if current and (len(current) >= max_requests or current_bytes + size > max_bytes):
            batches.append(current)
            current, current_bytes = [], 0
        current.append(request)
        current_bytes += size
    if current:
        batches.append(current)
    return batches


def create_batch(client, requests: list[dict]) -> str:
    """Submit one batch and return its ID. Validation must pass first."""
    problems = validate(requests)
    if problems:
        raise ValueError("; ".join(problems))
    batch = client.messages.batches.create(requests=requests)
    return batch.id


if __name__ == "__main__":
    # Offline demonstration: no API calls
    reviews = [
        make_request("review-001", "Classify: 'Great course!'"),
        make_request("review 002", "Classify: 'Too fast.'"),           # Space is not allowed
        make_request("review-001", "Classify: 'Buggy quizzes.'"),      # Duplicate ID
        make_request("review-004", "Classify: 'Loved it.'", max_tokens=0),
    ]
    for problem in validate(reviews):
        print("problem:", problem)

    valid = [make_request(f"item-{n:03d}", f"Describe product {n}.") for n in range(1, 6)]
    batches = split_into_batches(valid, max_requests=2)
    print("batches:", len(batches), "sizes:", [len(b) for b in batches])

Output:

problem: invalid custom_id 'review 002'
problem: duplicate custom_id 'review-001'
problem: review-004: max_tokens must be at least 1
batches: 3 sizes: [2, 2, 1]

How It Works

1. make_request() produces the request shape from the documentation. Every request in the project goes through this one function, so the shape never drifts.
2. validate() applies the documented rules locally. Each problem names the specific request, so it can be fixed without searching.
3. The duplicate check uses a set. The second review-001 is reported, so results could not be matched without this check.
4. size_in_bytes() measures each request's JSON size. Because the API limits the whole batch to 256 MB, a running total is needed.
5. split_into_batches() starts a new batch when either the request count or the byte total would exceed the safe limit. The demonstration uses a small count, so the split is visible.
6. create_batch() refuses to submit a batch that fails validation. Problems surface immediately, not after hours of processing.

Sizing a Real Batch

The byte estimate is only as good as the requests. Long documents dominate the size, so a batch of short classification requests can hold far more items than a batch of full-length documents. Check the estimate on a sample, and leave margin:

sample_size = sum(size_in_bytes(r) for r in valid) / len(valid)
print(f"average request: {sample_size:.0f} bytes")

Long documents are better split into chunks (Lesson 118) before they become batch requests, so each request stays a predictable size.

Common Mistakes

Building IDs from Positions That Change

What happens: inserting a new item shifts every later ID, and results from an earlier batch can no longer be matched.

Fix: derive IDs from stable identities, as Lesson 133 explains.

Submitting Without Local Validation

What happens: a malformed request is discovered only when the batch ends.

Fix: validate before every submission.

Filling Batches to the Exact Limit

What happens: an estimate that is slightly low produces a rejected batch.

Fix: keep a margin below the documented limits.

Using max_tokens: 0

What happens: the batch rejects the request, because pre-warming does not apply to batches.

Fix: use a normal positive limit.

Keeping Only the Batch ID

What happens: if the batch fails, the original requests are lost.

Fix: store the request list as well, so a failed batch can be resubmitted (Lesson 138).

Best Practices

• Build every request through one function.
• Validate locally before each submission.
• Split by both count and bytes, with a margin below the limits.
• Store the requests alongside the batch ID.
• Log the batch ID, its size, and the number of requests for each submission.
• Test the builder offline, as the demonstration does.

Exercise

1. Add a check that rejects a request whose prompt is an empty string.
2. Change split_into_batches() to also return the index of the first request in each batch, so results can be mapped back to their position.
3. Generate 60,000 short requests and compute how many batches split_into_batches() produces with the default safe limits.
4. Explain why validate() checks duplicates across the whole list, not within each batch.

Step 3: with the default safe limit of 50,000 requests per batch, 60,000 requests split into two batches: 50,000 and 10,000. The byte limit may split them earlier if the requests are long, so the answer depends on the size of each request. For short requests of a few hundred bytes, the count limit is the one that applies.

Step 4: a duplicate custom_id in two different batches is still a problem. Results from both batches would share the same identifier, and matching a result to its input would be ambiguous, even though each batch on its own looks valid. Checking across the whole set catches the problem where it starts. If the requests come from several sources, a central registry of IDs (Lesson 133) is the safer approach.