Test SDK Functions

Why an Application That Calls a Model Needs Tests

An application that calls a language model has two kinds of behavior. Most of it is ordinary software: parsing input, building requests, validating output, storing results, and handling errors. A small part is the model's answer, which varies from run to run and cannot be checked with an exact comparison. Testing works best when these two kinds are separated. The ordinary software is tested thoroughly, quickly, and without any network access. The model's answers are evaluated separately, with the methods from Lesson 110.

This lesson covers the first kind: the functions in an application that build requests, read responses, and apply rules. It uses pytest, the most common Python test runner, and it shows how to structure code so that most of it can be tested without calling the API at all. Lessons 182 through 184 extend this to fake responses, error paths, and streams.

What a Test Is

A test is a small function that calls one piece of code with known input, and checks that the output matches what is expected. A test passes or fails, and a suite of tests runs with a single command. The value of tests is that they catch changes that break existing behavior, which matters when code is edited, refactored, or upgraded to a new SDK version.

Tests are written in files whose names start with test_, and each test is a function whose name starts with test_. The assert statement states the expectation. When an assertion is false, pytest reports the values it compared, which makes the failure easy to understand.

Structure Code for Testing

The most important decision is how to organize the code. Functions that do not call the API are easy to test, and functions that do must be kept thin. A common structure separates the work into three parts: a function that builds the request from the application's data, a function that reads the response into the application's types, and a function that makes the call and connects them. Only the last part needs the network, and the first two can be tested with ordinary data.

def build_request(question: str, course: str) -> dict:
    return {
        "model": "claude-sonnet-5-5",
        "max_tokens": 512,
        "system": f"You are the tutor for {course}.",
        "messages": [{"role": "user", "content": question}],
    }


def read_answer(message_content: list[dict]) -> str:
    return "".join(block["text"] for block in message_content if block["type"] == "text")

Both functions are pure: the same input always gives the same output, and nothing outside the function is touched. That property makes them the easiest code to test and the most valuable to test, because every request and every answer the application handles passes through them.

A Test File

The test file below checks the request builder and the answer reader. It uses plain values and needs no API key, no network, and no SDK import:

"""test_requests.py: tests for the request builder and the answer reader."""

from requests_module import build_request, read_answer


def test_request_includes_course_in_system_prompt():
    request = build_request("What is a loop?", "Python Basics")
    assert "Python Basics" in request["system"]


def test_request_puts_question_in_user_message():
    request = build_request("What is a loop?", "Python Basics")
    assert request["messages"] == [{"role": "user", "content": "What is a loop?"}]


def test_answer_joins_text_blocks_and_ignores_thinking():
    content = [
        {"type": "thinking", "thinking": "reasoning", "signature": "sig"},
        {"type": "text", "text": "A loop "},
        {"type": "text", "text": "repeats code."},
    ]
    assert read_answer(content) == "A loop repeats code."

The three tests cover the rules that matter most: the course reaches the system prompt, the question reaches the user message, and the answer is assembled from text blocks only. The third test uses a thinking block on purpose, because reading the first block of a response is a bug that appears only when thinking is enabled (Lesson 153).

Running the Tests

The tests are run from the project directory with a single command:

python -m pip install pytest
python -m pytest -q test_requests.py

...                                                                      [100%]
3 passed in 0.02s

The -q option gives a short report, with one dot for each passed test. A failure would show the failing test's name, the expected and actual values, and the line that broke, which is enough to find most problems without rerunning anything.

The module name in the import, requests_module, is the file that contains the two functions. In a project, the test file imports the code under test from the package, and the tests run from the project root so that the import resolves.

Testing Edge Cases

The happy path is only part of the job. The tests that catch the most real problems check edges: an empty answer, a response with no text blocks, a question containing quotation marks or line breaks, and a course name with unusual characters. Each edge is a small test with a clear name, and each one documents a behavior the application relies on. When a new edge is discovered in production, a test for it should be added before the fix, so that the fix can be shown to work and the problem cannot return unnoticed.

Fixtures and Shared Data

When several tests need the same data, a fixture supplies it. A fixture is a function marked with @pytest.fixture that returns the data, and any test that names the fixture as a parameter receives it. Fixtures keep test data in one place, so a change to the shape of a response is made once. A sample response captured from a real call, with the sensitive values removed, is a good fixture, and it should be refreshed when the SDK or the platform changes its response format.

What These Tests Do Not Show

A test that passes shows that the code behaves as the test describes. It does not show that the model answers well, that the prompt is effective, or that the platform will respond as expected. Those questions need evaluation with real calls, and they are covered by the golden-set methods in Lesson 110. The tests in this lesson protect the code around the model, and the evaluations protect the model's behavior. A project needs both, and it should not expect one to substitute for the other.

Summary

Separate the code that builds requests and reads responses from the code that makes the network call, so that most of the application can be tested with ordinary data. pytest runs test functions that assert expected outputs, and a passing run shows that the rules hold. Edge cases deserve tests as much as the happy path, and fixtures keep sample data in one place. The next lesson shows how to test code that calls the client, by replacing the client with a stand-in that returns chosen responses.