Analyzing Multiple Images in One Request

Ma Mahalakshmi V Updated 16 Sep 2026
6 min read ·Lesson 102 of 224

Why You'd Send More Than One Image

Some questions cannot be answered from a single image. "Which of these two product photos looks more professional?" "Do these three screenshots show the same bug at different steps?" "Has anything changed between this before-and-after pair?" All of these require the model to hold multiple images in view simultaneously and reason about the relationship between them — not analyze each one independently and have you compare the results yourself afterward.

The Responses API supports this directly: a single content list can include several input_image parts alongside your text, and the model treats them as part of one shared context rather than as separate, disconnected requests.

Sending Two Images for Comparison

import base64
from openai import OpenAI

client = OpenAI()


def encode_image(path: str) -> str:
    with open(path, "rb") as f:
        return base64.b64encode(f.read()).decode("utf-8")


before_b64 = encode_image("photos/kitchen_before.jpg")
after_b64 = encode_image("photos/kitchen_after.jpg")

response = client.responses.create(
    model="gpt-5.6-terra",
    input=[
        {
            "role": "user",
            "content": [
                {"type": "input_text", "text": "The first image is 'before' and the second is 'after'."},
                {
                    "type": "input_image",
                    "image_url": f"data:image/jpeg;base64,{before_b64}",
                    "detail": "high",
                },
                {
                    "type": "input_image",
                    "image_url": f"data:image/jpeg;base64,{after_b64}",
                    "detail": "high",
                },
                {"type": "input_text", "text": "List every visible change between the two images."},
            ],
        }
    ],
)

print(response.output_text)

Two structural details matter here:

  • The images are labeled by their position, and that labeling is stated explicitly in text ("The first image is 'before' and the second is 'after'"). The model receives the content parts in order, but it has no inherent way to know your intended labels for each one beyond the order they appear in and whatever you tell it. Never assume the model will infer which image is which just because you know internally that the first one is "before" — say so directly.
  • The question comes after both images, following the same principle from Lesson 2: placing the actual task immediately before the model generates its answer keeps that task front-of-mind, especially when there's a meaningful amount of visual content in between.

Analyzing a Larger Set of Images

The same pattern extends to more than two images. This is useful for tasks like reviewing a batch of product photos or checking a multi-page scanned document:

def build_multi_image_content(image_paths, question):
    content = [{"type": "input_text", "text": question}]
    for index, path in enumerate(image_paths, start=1):
        encoded = encode_image(path)
        content.append({"type": "input_text", "text": f"Image {index}:"})
        content.append(
            {
                "type": "input_image",
                "image_url": f"data:image/jpeg;base64,{encoded}",
                "detail": "high",
            }
        )
    return content


image_paths = [
    "scans/invoice_page_1.jpg",
    "scans/invoice_page_2.jpg",
    "scans/invoice_page_3.jpg",
]

response = client.responses.create(
    model="gpt-5.6-terra",
    input=[
        {
            "role": "user",
            "content": build_multi_image_content(
                image_paths,
                "These are three pages of the same invoice, in order. Extract the grand total, which should appear on the final page.",
            ),
        }
    ],
)

print(response.output_text)

build_multi_image_content generalizes the labeling pattern from the two-image example: it interleaves an "Image N:" text label immediately before each image's input_image part, using enumerate(image_paths, start=1) so the numbering matches how a person would naturally refer to "page 1, page 2, page 3" rather than starting from zero. This labeling becomes increasingly important as the number of images grows — with two images "before" and "after" is unambiguous without numbering, but with five or six images, unlabeled ordinal references quickly become unclear both to you and to the model.

Cost and Context Scaling

Every image you add contributes its own token cost, on top of whatever detail level you request for it (see Lesson 1 and Lesson 8). Sending ten high-detail images in one request is not a lightweight operation — it can consume a very large number of tokens before the model has processed a single word of your actual question. This has two practical implications:

  • Batch only the images that genuinely need to be compared together. If your task is "summarize each of these fifty screenshots independently," that's fifty independent single-image requests, not one fifty-image request — there's no cross-image reasoning to justify paying for shared context, and independent requests can also run in parallel for better throughput.
  • Reserve detail="high" for images where fine detail actually matters to the comparison. If you're comparing five photos for overall composition or color palette, lower detail may be entirely sufficient and meaningfully cheaper across five images than defaulting every one of them to high detail out of habit.

When Multiple Images Should Be Separate Requests Instead

Not every "several images" scenario belongs in one request. Use multiple images in a single request only when the task requires joint reasoning across them — comparison, sequence, consistency-checking. If each image needs an independent, unrelated answer (for example, tagging each photo in a gallery with its own set of labels, unrelated to the other photos), it is both cheaper and more parallelizable to issue one request per image:

def tag_each_image_independently(image_paths):
    results = {}
    for path in image_paths:
        encoded = encode_image(path)
        response = client.responses.create(
            model="gpt-5.6-terra",
            input=[
                {
                    "role": "user",
                    "content": [
                        {"type": "input_text", "text": "List three descriptive tags for this image."},
                        {
                            "type": "input_image",
                            "image_url": f"data:image/jpeg;base64,{encoded}",
                        },
                    ],
                }
            ],
        )
        results[path] = response.output_text
    return results

This function processes each image with its own independent request, because tagging one photo has nothing to do with tagging another — there is no relationship between them for the model to reason about jointly, so bundling them into one request would only add unnecessary shared context cost without improving the result.

Common Mistakes

Sending several unrelated images in one request out of convenience, rather than because the task actually requires comparing them. This wastes tokens on shared context the model doesn't need and often produces a worse answer, since the model may try to find relationships between images that were never meant to be related.

Failing to label which image is which, especially past two images, and then getting an answer that mixes up or misattributes details between them. Always state explicitly, in text, which image corresponds to which role or position in your question.

Defaulting every image in a multi-image request to detail="high", which multiplies token cost across every image in the request. Set detail per image based on what that specific image actually needs, not uniformly out of caution.

Best Practices

Interleave a short text label immediately before each image when sending more than one, so both you and the model have an unambiguous way to refer to "image 2" or "the second screenshot."

Reserve multi-image requests for genuinely comparative or sequential tasks, and fall back to independent single-image requests — which can also be run concurrently — whenever each image's analysis doesn't depend on the others.

Keep a hard, explicit limit on how many images you send in one request in your own application code, both to control cost predictably and because extremely large image counts increase the risk that the model conflates or loses track of individual images within the set.

0 Comments

Reviewed before they appear

No comments yet.

OpenAI SDK
Introduction to the OpenAI SDK Setting Up Python Creating an API Key Your First Call — client.responses.create() and response.output_text Understanding Billing, Credits, and What a Request Costs Why Responses Replaced Chat Completions Anatomy of a Request: model, input, and instructions Anatomy of a Response: The Typed output Array, Not Just Text Roles: User, Assistant, and Developer/System Choosing a Model, and Reading the Models Page Instead of Memorizing Names Instructions vs. Input Writing Prompts That Get Consistent Results Few-Shot Examples Reasoning Models and the reasoning Parameter Debugging a Prompt That Misbehaves Why Streaming Matters for User Experience stream=True and Iterating Over Events Handling the Event Types You Actually Care About Background Mode for Long-Running Jobs Project — Add Live Streaming to Your Chatbot The Problem With Parsing Free Text JSON Schema and Strict Mode Pydantic Models With the SDK's Parse Helpers Handling Refusals and Validation Failures Project — A Resume-to-JSON Extractor Working With input_image input_file, PDFs, and the Files API Image Generation Speech-to-Text and Text-to-Speech Project: A PDF Question-Answering Script What Function Calling Is Defining a Tool Schema The Full Loop Multiple Tools Errors, Timeouts, and Untrusted Arguments Project: A Weather Assistant Web Search File Search and Vector Stores Code Interpreter Remote MCP Servers and Connectors Project: A Research Assistant What an Embedding Is, Without the Maths Generating and Storing Embeddings Similarity Search From Scratch Hosted Vector Stores vs. Rolling Your Own A Small RAG App Over a Folder of Notes Agents vs. a Single API Call — When You Need One pip install openai Giving Agents Tools Handoffs and Multi-Agent Triage Guardrails and Approvals Tracing and Observing What Your Agent Did A Multi-Agent Support Desk Error Codes and What Each One Means Retries, Timeouts, and Backoff Rate Limits and Spend Limits Prompt Caching and Cost Optimisation The Batch API for Bulk Work Async Clients and Concurrency Moderation and Safety Best Practices Designing the App Backend With FastAPI Streaming to a Simple Frontend Deploying and a Cost/Safety Checklist Why Web Search Is Useful for Current Information Using the Web Search Tool with the Responses API Configuring Search Behavior for Application Use Cases Understanding Citations and Source Attribution Where to Go Next Building a Research Assistant with Web Search Combining Web Search with Structured Outputs Handling Conflicting or Low-Quality Web Sources Reducing Unsupported Claims with Grounded Generation Testing Freshness-Sensitive AI Answers Production Considerations for Web-Grounded Applications Understanding File Search and Retrieval-Augmented Generation Creating and Organizing Vector Stores Uploading Documents for Retrieval Connecting Vector Stores to Responses API Requests Designing Document Metadata and Filtering Strategies Building a PDF Question-Answering Application Improving Retrieval Quality With Better Document Preparation Handling Missing Evidence and Retrieval Failures Combining File Search With Web Search Building a Production Knowledge-Base Assistant What the Code Interpreter Tool Is Designed For Running Python-Based Analysis Through the OpenAI SDK Uploading Datasets for Analysis Analyzing CSV and Spreadsheet Data Generating Charts and Data Summaries Handling Generated Files and Downloadable Artifacts Building a Data-Analysis Assistant Combining Code Execution with Structured Outputs Validating Generated Calculations and Results Security and Sandbox Considerations for Code Execution Understanding Multimodal Input with the OpenAI SDK Sending Images to a Model Image Analysis from URLs and Uploaded Files Extracting Text and Information from Screenshots Building an Image-Question-Answering Application Combining Image Input with Structured Output Analyzing Multiple Images in One Request Handling Image Quality and Input Limitations Designing Multimodal Prompts for Reliable Results Building a Practical Vision-Powered Python Application Understanding Speech-to-Text and Text-to-Speech Workflows Transcribing Audio with the OpenAI SDK Working with Uploaded Audio Files Handling Timestamps and Transcription Metadata Building a Meeting Transcription Workflow Generating Spoken Responses from Text Handling Long Audio and Processing Failures Combining Audio with Text and Tool Calling Building an End-to-End Python Voice Application What Embeddings Are and When to Use Them Generating Embeddings With the OpenAI API Preparing Text for Embedding Comparing Vectors With Cosine Similarity Building a Simple Semantic Search Engine in Python Storing Embeddings in a Database Metadata Filtering for Semantic Search Chunking Strategies for Better Retrieval Evaluating Semantic Search Quality Building a Document Similarity Application When Batch Processing Makes Sense Designing Large-Volume AI Processing Pipelines Using Asynchronous Python with the OpenAI SDK Running Concurrent Requests Safely Controlling Concurrency and Avoiding Rate Limits Tracking Batch Job Progress Handling Partial Failures in Bulk Workloads Retrying Failed Items Without Duplicating Successful Work Designing Resumable AI Processing Jobs Building a Production Batch-Processing Pipeline Batch Processing Makes Sense Large-Scale AI Processing Pipelines Async Python with OpenAI SDK Safe Concurrent Requests Concurrency & Rate Limits Batch Progress Tracking Partial Failure Handling Safe Retry Handling Resumable AI Jobs Production Batch Pipeline System–User Data Separation Reusable App Instructions Prompt Templates & Variables Extraction & Classification Prompts Summarization & Transformation Prompts Explicit Output Requirements Prompt Version Management Prompt Testing & Evaluation Reusable Python Prompt Library API Key Security Secure API Key Storage Secure Secret Management Prompt Injection Prevention Trusted vs. Untrusted Content Tool Argument Validation Sensitive Data Handling Secure Logging AI Action Authorization Production AI Security Checklist Why AI Applications Need Evaluation Beyond Unit Tests Unit Testing OpenAI SDK Integration Code Mocking API Responses in Python Tests Testing Structured Outputs Against Schemas Testing Tool-Calling Workflows Building a Small Evaluation Dataset Measuring Accuracy, Consistency, and Failure Rates Regression Testing Prompts and Model Changes Human Evaluation Versus Automated Evaluation Creating a Repeatable Evaluation Pipeline AI Request Monitoring Token Cost Management Usage Metrics Design Reducing Model Calls Prompt & Context Optimization Model Selection & Optimization AI Caching Strategies Interactive Latency Optimization Usage Dashboards & Budget Alerts Performance & Cost Checklist Every API Call Starts Fresh Fixing API Statelessness Server-Side Conversation Memory Limits of Response Chaining What We're Building Conversation Memory Challenges Preparing an OpenAI SDK Application for Deployment Environment-Specific Configuration for Development and Production Deploying a Python AI Service with Docker Container Health Checks and Startup Configuration Managing Secrets in Cloud Deployments Background Workers for Long-Running AI Tasks Queues and Asynchronous Job Architectures Scaling AI Workloads Horizontally Monitoring Production Incidents and Failures Production Deployment Checklist for OpenAI SDK Applications Reusable OpenAI Service Classes AI Client Dependency Injection Typed AI Responses Python Configuration Management AI Request Decorators Centralized AI Error Handling Clean SDK Abstractions Reusable OpenAI Utilities Internal AI Python Libraries SDK Integration Maintenance Production AI Chatbot Document Q&A System Web Research Assistant Customer Support Agent AI Data Analysis Assistant Image Analysis App Meeting Transcription & Summary Semantic Document Search Multi-Tool AI Agent Production OpenAI SDK App Why "It Looked Fine When I Tested It" Isn't Enough Timing Note Status Note Pre-Decision Status Note Current Availability Note
Ask about this post
AI Ask about this post

Ask questions about Analyzing Multiple Images in One Request and get answers drawn from it.

Signed-in readers only.