Separating What the Model Decides From What Your Code Does

A tool-calling (function-calling) workflow has two distinct halves. The model decides which tool to call and with what arguments — that decision is a model-behavior question, and whether the model tends to make good decisions belongs to the evaluation layer (Unit 13). Your code then takes whatever the model decided, looks up the matching local function, executes it, and feeds the result back — that dispatch-and-execution logic is ordinary, deterministic Python code, and it is exactly the kind of thing this unit's testing techniques apply to directly.

This lesson focuses entirely on the second half: given a tool call the model already produced (simulated with a fake response, never a real one), does your routing code behave correctly? This distinction matters enough to repeat, because it is the single most common confusion when testing agentic code: a test that asserts "the model should have called get_weather" is an eval; a test that asserts "given a tool call for get_weather, my dispatcher invokes the get_weather Python function with the right arguments" is a unit test.

A Minimal Tool-Calling Dispatcher

import json


def get_weather(city: str) -> str:
    fake_data = {"Paris": "18C, cloudy", "Tokyo": "24C, clear"}
    return fake_data.get(city, "unknown city")


def get_exchange_rate(base: str, quote: str) -> str:
    fake_rates = {("USD", "EUR"): 0.92, ("USD", "JPY"): 149.3}
    rate = fake_rates.get((base, quote))
    return f"{rate}" if rate is not None else "rate unavailable"


TOOL_REGISTRY = {
    "get_weather": get_weather,
    "get_exchange_rate": get_exchange_rate,
}


def dispatch_tool_call(tool_name: str, arguments_json: str) -> str:
    if tool_name not in TOOL_REGISTRY:
        raise ValueError(f"Unknown tool requested: {tool_name!r}")

    try:
        arguments = json.loads(arguments_json)
    except json.JSONDecodeError as exc:
        raise ValueError(f"Malformed arguments for {tool_name}: {exc}") from exc

    function = TOOL_REGISTRY[tool_name]
    return function(**arguments)

dispatch_tool_call is the exact seam this lesson tests: it accepts a tool name and a raw JSON argument string (matching the shape the SDK gives you for a function-call output item), looks the tool up in a registry, parses the arguments, and invokes the corresponding Python function. Every branch here is deterministic and fully within your control, which is precisely why it deserves direct, thorough unit test coverage rather than being folded into a broader evaluation.

Testing the Routing Logic With Simulated Tool Calls

def test_dispatch_tool_call_routes_to_get_weather():
    result = dispatch_tool_call("get_weather", '{"city": "Paris"}')
    assert result == "18C, cloudy"
    print("PASS: dispatch_tool_call routes get_weather correctly")


def test_dispatch_tool_call_routes_to_get_exchange_rate():
    result = dispatch_tool_call("get_exchange_rate", '{"base": "USD", "quote": "JPY"}')
    assert result == "149.3"
    print("PASS: dispatch_tool_call routes get_exchange_rate correctly")


def test_dispatch_tool_call_rejects_unknown_tool():
    try:
        dispatch_tool_call("get_stock_price", '{"ticker": "ACME"}')
        raised = False
    except ValueError:
        raised = True
    assert raised
    print("PASS: dispatch_tool_call rejects an unregistered tool name")


def test_dispatch_tool_call_rejects_malformed_json():
    try:
        dispatch_tool_call("get_weather", '{"city": "Paris"')  # missing closing brace
        raised = False
    except ValueError:
        raised = True
    assert raised
    print("PASS: dispatch_tool_call rejects malformed argument JSON")

Each test simulates exactly one scenario the model's output could produce — a known tool with valid arguments, an unrecognized tool name, and malformed JSON — without ever calling the model to produce it. This is the crucial technique: you are constructing the model's hypothetical output by hand, as a plain string, the same way you constructed fake API responses in earlier lessons. The malformed-JSON case matters especially in real systems, because while well-behaved models rarely emit invalid JSON for a well-specified tool schema, "rarely" is not "never," and code that crashes with an unhandled JSONDecodeError in production is a code-quality bug, not a model-quality one.

Testing the Full Tool-Calling Loop With a Fake Client

A more complete workflow involves a loop: call the model, check whether it requested a tool call, execute the tool, send the result back, and repeat until the model produces a final answer. Testing this loop requires a fake client capable of returning different responses on successive calls — a step up in sophistication from the single-response fakes used earlier.

class ScriptedFakeClient:
    """Returns a pre-scripted sequence of responses, one per call."""

    def __init__(self, scripted_responses: list):
        self._responses = list(scripted_responses)
        self.call_count = 0

        class _Responses:
            def create(inner_self, **kwargs):
                response = self._responses[self.call_count]
                self.call_count += 1
                return response

        self.responses = _Responses()


class FakeToolCallResponse:
    def __init__(self, tool_name: str, arguments_json: str):
        self.tool_calls = [{"name": tool_name, "arguments": arguments_json}]
        self.output_text = None


class FakeFinalResponse:
    def __init__(self, text: str):
        self.tool_calls = []
        self.output_text = text


def run_tool_calling_loop(client, user_input: str, max_turns: int = 5) -> str:
    for _ in range(max_turns):
        response = client.responses.create(model="gpt-5.6-terra", input=user_input)
        if not response.tool_calls:
            return response.output_text

        call = response.tool_calls[0]
        tool_result = dispatch_tool_call(call["name"], call["arguments"])
        user_input = f"Tool result: {tool_result}"

    raise RuntimeError("Exceeded max_turns without a final answer")


def test_run_tool_calling_loop_executes_one_tool_then_finishes():
    fake_client = ScriptedFakeClient([
        FakeToolCallResponse("get_weather", '{"city": "Tokyo"}'),
        FakeFinalResponse("It is 24C and clear in Tokyo."),
    ])

    result = run_tool_calling_loop(fake_client, "What's the weather in Tokyo?")

    assert result == "It is 24C and clear in Tokyo."
    assert fake_client.call_count == 2
    print("PASS: run_tool_calling_loop executes a tool call then returns the final answer")


def test_run_tool_calling_loop_raises_after_max_turns():
    endless_tool_call = FakeToolCallResponse("get_weather", '{"city": "Paris"}')
    fake_client = ScriptedFakeClient([endless_tool_call] * 5)

    try:
        run_tool_calling_loop(fake_client, "Weather?", max_turns=5)
        raised = False
    except RuntimeError:
        raised = True

    assert raised
    print("PASS: run_tool_calling_loop raises RuntimeError instead of looping forever")

ScriptedFakeClient holds a list of pre-built responses and returns the next one in sequence on each call, tracked with call_count — this simulates a multi-turn conversation without any real model involved, and it does so completely deterministically, so the test produces the exact same result every time it runs. The first test confirms the ordinary path: a tool call followed by a final answer, in exactly two round trips. The second test confirms an important safety property — that a model which keeps requesting tools indefinitely does not turn into an infinite loop in your application, but instead fails loudly with RuntimeError. This kind of guard, and the test that proves it works, is easy to omit and expensive to omit: an infinite loop against a real API in production silently burns cost with every iteration.

Testing That the Tool Result Is Correctly Fed Back

A subtle bug in tool-calling loops is constructing the follow-up message incorrectly — for example, forgetting to include the tool's result, or attaching it to the wrong tool-call ID in a multi-tool-call turn. This is worth testing explicitly by inspecting what was actually sent on the second call.

def test_run_tool_calling_loop_feeds_tool_result_back_into_next_call():
    fake_client = ScriptedFakeClient([
        FakeToolCallResponse("get_weather", '{"city": "Paris"}'),
        FakeFinalResponse("Done."),
    ])

    run_tool_calling_loop(fake_client, "Weather in Paris?")

    # Reconstruct what the second call actually received by re-invoking manually,
    # since ScriptedFakeClient does not store per-call inputs by default here.
    assert fake_client.call_count == 2
    print("PASS: the loop made exactly two calls, implying the tool result was used")

In a real test suite you would typically extend ScriptedFakeClient to also record each call's kwargs (the same last_kwargs pattern from Lesson 2), then assert that the second call's input contains the tool's actual result string — this confirms the feedback step is wired correctly rather than, for instance, silently discarding the tool's output and re-sending the original question unchanged.

Common Mistakes

• Testing whether the model chose the right tool, in a unit test. That is an evaluation concern requiring realistic inputs and a grading rubric (Unit 13); a unit test should assume a specific tool call happened and verify only that your code handles it correctly.
• Not testing the "unknown tool" and "malformed arguments" branches. These are the exact cases most likely to appear the first time the model behaves unexpectedly in production, and they are also the cheapest to test, since they require no realistic model behavior at all — only a hand-built string.
• Building a tool-calling loop test that never sets a turn limit and calling it "safe" without testing the limit. Without a specific test for the max-turns path, an infinite-loop bug can go undetected until it appears as a runaway API bill.

Best Practices

• Draw a hard line between "does the model pick the right tool" (eval) and "does my dispatcher route correctly given a tool call" (unit test), and keep each concern in its own test suite.
• Simulate multi-step conversations with a scripted fake client that returns a fixed sequence of responses, so multi-turn tool-calling loops are testable without any real model calls.
• Explicitly test failure and boundary paths — unknown tool names, malformed arguments, and the maximum-turns safeguard — since these are the paths most likely to cause real production incidents and the easiest to verify with fakes.