Set Batch IDs

The Custom ID Is the Only Link Back

When a batch finishes, its results come back as a file. Lesson 131 noted that the order is not guaranteed, so the custom_id is the only reliable way to connect each result to the input that produced it. Choosing these identifiers well is one of the most important design decisions in batch work. A poor choice makes results impossible to match after a retry, and a good one makes the entire pipeline easy to rerun and audit.

The rules are short (Lesson 132): 1 to 64 characters, letters, digits, hyphens, and underscores, unique within the batch. The design question is what to put in them.

Position Is a Poor Identifier

The simplest choice is the row number: item-1, item-2, and so on. It breaks in predictable ways:

An identifier should describe what the item is, not where it sits.

Derive IDs from Stable Identities

If each source item has a stable key, such as a database primary key, a product SKU, or a document path, use it:

def custom_id_for(record_key: str, task: str) -> str:
    return f"{task}-{record_key}"        # e.g. "classify-review-8841"

The key may contain characters the rule does not allow, such as spaces, dots, or slashes. A hash of the key always produces safe characters, and it is stable, because the same key always gives the same hash:

import hashlib


def custom_id_for(record_key: str, task: str) -> str:
    digest = hashlib.sha256(record_key.encode("utf-8")).hexdigest()[:24]
    return f"{task}-{digest}"            # e.g. "classify-3f9a0c7e41d84b6f9c1e0a5b"

Twenty-four hex characters give 96 bits of identity, which is far more than enough for any realistic job. The task prefix keeps IDs from different jobs apart, and makes them readable in logs.

The trade-off: hashed IDs are not human-readable. Keep a mapping from the ID to the record key in your own database, so people can still find the original item. That mapping is also the place to answer "which items were never processed?".

Make the ID Part of the Work's Identity

A batch often includes the same record more than once, once for each task. The task name in the ID keeps those requests apart:

classify-3f9a0c7e41d84b6f9c1e0a5b     ← classification of record 8841
summarize-3f9a0c7e41d84b6f9c1e0a5b    ← summary of the same record

The record key is shared, but the task makes each ID unique. If the same task is later run on the same record with a different prompt, you can include a prompt version in the task name, such as classify-v3, so the new results are not confused with the old ones.

Keeping a Registry

An ID is only useful if you can look up what it refers to. A small registry table stores the mapping, and records the batch in which each request was submitted and what happened to it:

import sqlite3

REGISTRY_DDL = """
CREATE TABLE IF NOT EXISTS batch_items (
    custom_id   TEXT PRIMARY KEY,
    record_key  TEXT NOT NULL,
    task        TEXT NOT NULL,
    batch_id    TEXT,
    status      TEXT NOT NULL DEFAULT 'pending'
)
"""

The status column records outcomes (pending, succeeded, errored, expired, canceled) as results arrive. Lesson 138 uses it to decide what to resubmit.

Practical Example: Stable IDs and a Registry

This example builds IDs from record keys, stores them in a registry, and shows that inserting a new record at the top does not change the ID of an existing one. The demonstration runs offline.

"""batch_ids.py: stable custom IDs derived from record keys, with a registry."""

import hashlib
import sqlite3

from batch_builder import make_request, validate


def custom_id_for(record_key: str, task: str) -> str:
    digest = hashlib.sha256(record_key.encode("utf-8")).hexdigest()[:24]
    return f"{task}-{digest}"


def open_registry(path: str = ":memory:") -> sqlite3.Connection:
    db = sqlite3.connect(path)
    db.execute("""CREATE TABLE IF NOT EXISTS batch_items (
        custom_id TEXT PRIMARY KEY, record_key TEXT NOT NULL, task TEXT NOT NULL,
        batch_id TEXT, status TEXT NOT NULL DEFAULT 'pending')""")
    return db


def build_batch(db: sqlite3.Connection, records: dict[str, str], task: str,
                prompt_template: str) -> list[dict]:
    requests = []
    for record_key, text in records.items():
        cid = custom_id_for(record_key, task)
        requests.append(make_request(cid, prompt_template.format(text=text)))
        db.execute("INSERT OR IGNORE INTO batch_items (custom_id, record_key, task) VALUES (?, ?, ?)",
                   (cid, record_key, task))
    db.commit()
    return requests


if __name__ == "__main__":
    db = open_registry()
    template = "Classify this review as positive, negative, or mixed: {text}"

    records = {"review-8841": "Great course!", "review-8842": "Too fast for beginners."}
    first = build_batch(db, records, "classify", template)

    # A new record arrives at the top of the source, shifting every position
    records_shifted = {"review-9000": "New review.", **records}
    second = build_batch(db, records_shifted, "classify", template)

    ids_first = {r["custom_id"] for r in first}
    ids_second = {r["custom_id"] for r in second}
    print("existing IDs unchanged:", ids_first <= ids_second)
    print("new IDs added:", len(ids_second - ids_first))
    print("problems:", validate(second) or "none")
    print("registry rows:", db.execute("SELECT COUNT(*) FROM batch_items").fetchone()[0])

Output:

existing IDs unchanged: True
new IDs added: 1
problems: none
registry rows: 3

How It Works

1. custom_id_for() hashes the record key, so the ID depends only on the record and the task, never on its position in the list.
2. The new record shifts nothing. The two existing records keep their IDs, which is why the first check is True.
3. INSERT OR IGNORE adds each request to the registry once. Records already there are left alone, so the registry does not grow with duplicates when a batch is rebuilt.
4. The registry has three rows after the second build: two from the first batch, and one new record. This is the mapping that answers questions about which records have results.
5. validate() from Lesson 132 confirms the hashed IDs meet the documented format.

Common Mistakes

Using Row Numbers

What happens: inserting or sorting records silently mismatches results with inputs.

Fix: derive IDs from stable keys.

Using Raw Keys That Contain Forbidden Characters

What happens: spaces, dots, or slashes in keys fail the format rule, and the batch is rejected.

Fix: hash the key, and keep the mapping in your registry.

Reusing an ID for a Different Prompt

What happens: new results are confused with old ones, because the identifiers match.

Fix: include a prompt version in the task name.

Forgetting Uniqueness Across Batches

What happens: a result from one batch overwrites the record of another, in your own storage.

Fix: use the registry as the source of truth for what has been submitted.

Making IDs Unreadable Without a Mapping

What happens: nobody can tell which item a failed ID refers to.

Fix: keep the registry, and include the task name in every ID.

Best Practices

• Derive IDs from stable record keys, not from positions.
• Hash keys that contain unsafe characters, and keep the mapping.
• Include the task and prompt version in the ID.
• Keep a registry of every submitted request and its outcome.
• Use the registry, not the batch, as the source of truth for what has been done.
• Test ID stability whenever the record source changes.

Exercise

1. Change custom_id_for() to use the record key directly, and submit a key containing a space. Which check catches it, and where?
2. Build a batch with two tasks, classify and summarize, for the same record. Confirm that the two IDs differ, and explain why.
3. Add a query that lists every record whose status is still pending for a given task.
4. Explain why INSERT OR IGNORE is used rather than INSERT, and what would happen to the registry if INSERT were used on a rerun.

Step 2: the two IDs begin with different task names (classify- and summarize-), so they differ even though the hashed record key is the same. The task prefix is what separates the two jobs, so their results can coexist in one registry and never be confused.

Step 4: INSERT OR IGNORE skips a row whose custom_id already exists, so rebuilding a batch leaves the registry unchanged for records that are already present. With a plain INSERT, the second build would raise an integrity error on the primary key, halting the rebuild partway through. If the table had no primary key, the rerun would instead create duplicate rows, and every count from the registry would be inflated.