Send Image Inputs

Claude Can See Images

Claude is a multimodal model: in the same request, it can read text and look at images. You can ask it to describe a photo, read a handwritten note, compare two screenshots, or check whether a chart matches its caption. The image is not converted into text first. Claude reads it directly, which is why images are useful for tasks such as document understanding, quality checks, and accessibility descriptions.

Images are sent as content blocks in a user message (Lesson 16), alongside text blocks:

{
    "role": "user",
    "content": [
        {"type": "image", "source": {...}},           # The image
        {"type": "text", "text": "What is in this?"}  # The question
    ],
}

Lesson 111 introduces the basic pattern. Lessons 112 and 113 cover the two most common ways to supply an image: by URL and as base64 data. Lesson 116 covers the Files API, which lets you upload an image once and reuse it.

Supported Formats and Sizes

According to the current vision documentation:

Two details matter in practice:

• The 10 MB limit applies to the base64 text, which is about one third larger than the original file. An image of about 7.5 MB becomes roughly 10 MB once encoded.
• Very large images are resized anyway. Sending a 6000-pixel photo gains nothing over a 2576-pixel version for most tasks, and costs more to transfer. Resize images before sending them when the task does not need full detail.

Image input also uses tokens. Larger images generally cost more tokens, so the same resizing that saves bandwidth also reduces cost. Measure with count_tokens() (Lesson 46) when you need exact numbers.

A Helper Module

The examples in Lessons 111–120 share a small helper module, media_utils.py. It uses only the standard library, so it works without extra packages. It provides:

• make_png(), which creates a small PNG in memory, so examples can run without image files;
• detect_media_type(), which identifies the format from the file's first bytes; and
• encode_base64(), which converts bytes into the text form the API expects.

"""media_utils.py: small helpers for images and PDFs, using only the standard library."""

import base64
import struct
import zlib

MAX_IMAGE_BASE64_BYTES = 10 * 1024 * 1024
IMAGE_SIGNATURES = [
    (b"\x89PNG\r\n\x1a\n", "image/png"),
    (b"\xff\xd8\xff", "image/jpeg"),
    (b"GIF87a", "image/gif"),
    (b"GIF89a", "image/gif"),
]


def _chunk(kind: bytes, data: bytes) -> bytes:
    checksum = zlib.crc32(kind + data) & 0xFFFFFFFF
    return struct.pack(">I", len(data)) + kind + data + struct.pack(">I", checksum)


def make_png(width: int, height: int, rgb: tuple[int, int, int]) -> bytes:
    """A solid-colour PNG. Each scanline begins with filter byte 0 (no filtering)."""
    row = b"\x00" + bytes(rgb) * width
    header = struct.pack(">IIBBBBB", width, height, 8, 2, 0, 0, 0)   # 8-bit RGB
    return (b"\x89PNG\r\n\x1a\n"
            + _chunk(b"IHDR", header)
            + _chunk(b"IDAT", zlib.compress(row * height))
            + _chunk(b"IEND", b""))


def detect_media_type(data: bytes) -> str | None:
    """Identify an image from its first bytes, not from its file name."""
    for signature, media_type in IMAGE_SIGNATURES:
        if data.startswith(signature):
            return media_type
    if data[:4] == b"RIFF" and data[8:12] == b"WEBP":
        return "image/webp"
    return None


def encode_base64(data: bytes) -> str:
    return base64.b64encode(data).decode("ascii")

Why check the bytes, not the file name: a file called photo.jpg can contain anything. The first bytes, called the file's signature, identify the real format. Trusting the extension can send the API a mislabeled file and produce a confusing error.

Practical Example: Your First Image Question

This example creates a solid red image in memory and asks Claude what color it is.

"""first_image.py: send an image in a message and ask about it."""

from anthropic import Anthropic

from media_utils import encode_base64, make_png

client = Anthropic()

image = make_png(64, 64, (220, 40, 40))        # A 64x64 solid red square

message = client.messages.create(
    model="claude-opus-5-5",
    max_tokens=512,
    messages=[{
        "role": "user",
        "content": [
            {"type": "image", "source": {
                "type": "base64",
                "media_type": "image/png",
                "data": encode_base64(image),
            }},
            {"type": "text", "text": "What color is this image? Answer in one word."},
        ],
    }],
)

print("".join(block.text for block in message.content if block.type == "text").strip())

Expected output:

Red

How It Works

1. make_png(64, 64, (220, 40, 40)) builds a PNG file in memory. It is a real, valid image, so the example needs no files on disk.
2. encode_base64() converts the bytes into text, and the image block tells the API the format with media_type.
3. The image block comes first, then the question. Placing the image before the text is the common pattern, and it reads naturally: "here is an image, now answer about it."
4. The answer is read by block type, the same way as any other text response (Lessons 41 and 42).
5. The question constrains the answer ("one word"), which makes the output easy to check.

Common Mistakes

Sending the Wrong Media Type

What happens: the API rejects the image, or behaves unexpectedly.

Fix: detect the format from the bytes (detect_media_type()), and send that value.

Forgetting the Base64 Size Overhead

What happens: images that seem under 10 MB are rejected, because their encoded form is larger.

Fix: check the length of the encoded string against the limit.

Sending Huge Images Unchanged

What happens: slow requests, higher token costs, and no gain in detail beyond the model's resolution.

Fix: resize images to the task's needs before encoding (Lesson 113).

Asking Before the Image Block

What happens: the question may be answered without reference to the image, because the text arrives first.

Fix: place image blocks before the question that refers to them.

Best Practices

• Detect the format from the bytes, and reject anything unsupported before sending it.
• Resize large images to the detail the task needs.
• Put images before the questions about them.
• Constrain answers with explicit formats such as "one word" or a JSON schema.
• Measure token cost for image-heavy workloads.
• Keep images out of logs unless you have a clear need and appropriate protections, since images may contain personal data.

Exercise

1. Change the color to (40, 90, 220) and ask for the color's name. Does the answer change as you expect?
2. Ask the same question with the text block first, then the image block. Compare the answers over several runs.
3. Use detect_media_type() on the bytes of make_png(...), and on bytes that start with b"GIF89a". What does each return?
4. Explain why the base64 size limit is checked against the encoded text rather than the original bytes.

Step 3: the PNG returns image/png, because its signature is \x89PNG\r\n\x1a\n. The GIF89a bytes return image/gif. Bytes that match no signature return None, which your code should treat as an unsupported file.

Step 4: the API receives the encoded text, and the documented limit is measured on that base64 size. Base64 represents every three bytes as four characters, so the encoded form is about one third larger. Checking the original size would allow files that the API then rejects.