Putting the Unit Together: A Receipt Processing Tool

This lesson combines everything from the unit into one coherent application: a command-line tool that takes a folder of receipt images, extracts structured data from each one, flags anything that needs manual review, and writes the results to a summary. It uses image encoding (Lesson 3), structured output (Lesson 6), quality validation (Lesson 8), and deliberate prompt design (Lesson 9).

Step 1: Define the Data Contract

from typing import List, Optional
from pydantic import BaseModel


class LineItem(BaseModel):
    description: str
    price: float


class ReceiptExtraction(BaseModel):
    merchant_name: Optional[str] = None
    purchase_date: Optional[str] = None
    total_amount: Optional[float] = None
    line_items: List[LineItem] = []
    confidently_extracted: bool
    notes: Optional[str] = None

ReceiptExtraction is the schema the model's answer must conform to. Every field that might not be visible on a given receipt is Optional, following the principle from Lesson 6 and Lesson 8 that a missing value should be represented explicitly rather than guessed. confidently_extracted is the self-reported confidence flag from Lesson 9, and notes gives the model a place to briefly explain any uncertainty (for example, "total is partially obscured by a fold in the paper") without polluting the other fields with hedging text.

Step 2: Image Validation and Encoding

import base64
import os

SUPPORTED_EXTENSIONS = {".jpg", ".jpeg", ".png", ".webp"}
MAX_IMAGE_BYTES = 15 * 1024 * 1024


def validate_image_file(path: str) -> None:
    extension = os.path.splitext(path)[1].lower()
    if extension not in SUPPORTED_EXTENSIONS:
        raise ValueError(f"Unsupported image format: {extension}")
    size_bytes = os.path.getsize(path)
    if size_bytes > MAX_IMAGE_BYTES:
        raise ValueError(f"Image too large: {size_bytes} bytes")


def guess_mime_type(path: str) -> str:
    extension = os.path.splitext(path)[1].lower()
    return {
        ".jpg": "image/jpeg",
        ".jpeg": "image/jpeg",
        ".png": "image/png",
        ".webp": "image/webp",
    }[extension]


def encode_image(path: str) -> str:
    with open(path, "rb") as f:
        return base64.b64encode(f.read()).decode("utf-8")

These three functions are the same validation and encoding building blocks introduced in Lesson 3, Lesson 5, and Lesson 8, kept deliberately small and focused so each one can be tested and reused independently. validate_image_file is called before any encoding work happens, so an unsupported or oversized file is rejected immediately with a clear message rather than after the cost of base64-encoding it has already been paid.

Step 3: The Extraction Function

from openai import OpenAI

client = OpenAI()

EXTRACTION_PROMPT = (
    "This is a photo of a purchase receipt. Extract the merchant name, "
    "purchase date (in YYYY-MM-DD format if determinable), the total amount, "
    "and every line item with its price. "
    "Set confidently_extracted to false if any part of the receipt is blurry, "
    "cropped, or otherwise hard to read, and briefly explain why in 'notes'. "
    "Leave any field you cannot determine as null rather than guessing."
)


def extract_receipt(image_path: str) -> ReceiptExtraction:
    validate_image_file(image_path)
    mime_type = guess_mime_type(image_path)
    encoded = encode_image(image_path)

    response = client.responses.parse(
        model="gpt-5.6-terra",
        input=[
            {
                "role": "user",
                "content": [
                    {"type": "input_text", "text": EXTRACTION_PROMPT},
                    {
                        "type": "input_image",
                        "image_url": f"data:{mime_type};base64,{encoded}",
                        "detail": "high",
                    },
                ],
            }
        ],
        text_format=ReceiptExtraction,
    )
    return response.output_parsed

EXTRACTION_PROMPT is defined as a module-level constant rather than an inline string, which follows the "library of tested prompt templates" practice from Lesson 9 — it's written once, can be reviewed and improved independently of the function that uses it, and stays consistent across every call. The prompt explicitly requests the confidence flag and explicitly instructs the model to use null instead of guessing, exactly the pattern established in Lesson 4, Lesson 6, and Lesson 9. detail="high" is used because reading receipt text accurately genuinely depends on resolution, as discussed in Lesson 2 and Lesson 4.

Step 4: Processing a Batch and Routing Uncertain Results

from dataclasses import dataclass


@dataclass
class ProcessingResult:
    image_path: str
    extraction: Optional[ReceiptExtraction]
    needs_review: bool
    error: Optional[str] = None


def process_receipt_folder(folder_path: str) -> List[ProcessingResult]:
    results = []
    for filename in sorted(os.listdir(folder_path)):
        full_path = os.path.join(folder_path, filename)
        if not os.path.isfile(full_path):
            continue

        try:
            extraction = extract_receipt(full_path)
            needs_review = (
                not extraction.confidently_extracted
                or extraction.total_amount is None
            )
            results.append(
                ProcessingResult(
                    image_path=full_path,
                    extraction=extraction,
                    needs_review=needs_review,
                )
            )
        except ValueError as validation_error:
            results.append(
                ProcessingResult(
                    image_path=full_path,
                    extraction=None,
                    needs_review=True,
                    error=str(validation_error),
                )
            )
    return results

process_receipt_folder iterates every file in the folder, skipping anything that isn't a regular file (such as a subdirectory). For each image, it calls extract_receipt inside a try block that specifically catches ValueError — the exception type both validate_image_file raises. This is a deliberately narrow exception type, not a bare except Exception, so that a genuinely unexpected error (a bug elsewhere in the code, for instance) is not silently swallowed and misreported as a simple validation failure. A result is flagged needs_review either when the model itself reported low confidence, or when a required field like total_amount came back None despite the model's confidence flag — a belt-and-suspenders check, since either signal alone could miss a genuine problem.

Step 5: Producing a Summary Report

def summarize_results(results: List[ProcessingResult]) -> str:
    lines = []
    total_processed = len(results)
    needs_review_count = sum(1 for r in results if r.needs_review)

    lines.append(f"Processed {total_processed} receipts.")
    lines.append(f"{needs_review_count} flagged for manual review.\n")

    for result in results:
        name = os.path.basename(result.image_path)
        if result.error:
            lines.append(f"[ERROR] {name}: {result.error}")
        elif result.extraction:
            status = "REVIEW" if result.needs_review else "OK"
            merchant = result.extraction.merchant_name or "unknown merchant"
            total = result.extraction.total_amount
            total_display = f"${total:.2f}" if total is not None else "unknown total"
            lines.append(f"[{status}] {name}: {merchant} - {total_display}")

    return "\n".join(lines)

This function builds a plain-text summary, one line per receipt, prefixed with [ERROR], [REVIEW], or [OK] so a human scanning the report can immediately see which files need attention. Using result.extraction.merchant_name or "unknown merchant" handles the Optional field gracefully — if the model returned None for the merchant name, the report substitutes a readable placeholder instead of printing the literal word "None."

Step 6: Testing the Logic Without Any Real API Calls

Every function above that doesn't itself call the API — summarize_results and the review-flagging logic — can be tested directly with hand-built fake data, following the same dependency-injection pattern used throughout this unit:

def test_summarize_results_flags_low_confidence():
    fake_results = [
        ProcessingResult(
            image_path="receipts/clear.jpg",
            extraction=ReceiptExtraction(
                merchant_name="Corner Cafe",
                purchase_date="2026-02-01",
                total_amount=12.50,
                line_items=[LineItem(description="Coffee", price=12.50)],
                confidently_extracted=True,
            ),
            needs_review=False,
        ),
        ProcessingResult(
            image_path="receipts/blurry.jpg",
            extraction=ReceiptExtraction(
                merchant_name=None,
                purchase_date=None,
                total_amount=None,
                line_items=[],
                confidently_extracted=False,
                notes="Image too blurry to read clearly.",
            ),
            needs_review=True,
        ),
    ]

    summary = summarize_results(fake_results)
    assert "Processed 2 receipts." in summary
    assert "1 flagged for manual review." in summary
    assert "[OK] clear.jpg" in summary
    assert "[REVIEW] blurry.jpg" in summary
    print("PASS: summarize_results correctly reports counts and per-file status")


def test_summarize_results_handles_errors():
    fake_results = [
        ProcessingResult(
            image_path="receipts/bad_format.tiff",
            extraction=None,
            needs_review=True,
            error="Unsupported image format: .tiff",
        )
    ]
    summary = summarize_results(fake_results)
    assert "[ERROR] bad_format.tiff" in summary
    print("PASS: summarize_results correctly reports validation errors")


if __name__ == "__main__":
    test_summarize_results_flags_low_confidence()
    test_summarize_results_handles_errors()

Both tests build ProcessingResult and ReceiptExtraction instances directly, with no network call anywhere in the test — exactly the same approach used for DetailedReceipt in Lesson 6 and ExtractionResult in Lesson 8. The first test checks that a mix of one confident and one low-confidence result produces the correct aggregate counts and the correct per-line status markers. The second test checks that a validation error (as would come from validate_image_file rejecting an unsupported format) is reported clearly in the summary rather than crashing the report generation. Because summarize_results only depends on plain data objects, not on the API client, these tests run instantly and require no network access or API key.

Assembling the Command-Line Entry Point

def main(folder_path: str) -> None:
    results = process_receipt_folder(folder_path)
    report = summarize_results(results)
    print(report)


if __name__ == "__main__":
    import sys
    if len(sys.argv) != 2:
        print("Usage: python receipt_processor.py <folder_path>")
        sys.exit(1)
    main(sys.argv[1])

This final entry point ties the pipeline together: read a folder path from the command line, process every receipt in it, and print the summary. Keeping main this thin — just orchestration, no business logic of its own — means every meaningful decision in the pipeline (validation rules, extraction prompt, review criteria, report formatting) lives in a separately testable function, which is the same architectural discipline followed throughout this unit: small, focused functions with a single clear responsibility, validated locally wherever possible, and tested independently of the network calls they depend on.