Project 4: Build a Customer-Support Tool-Calling Agent

This project builds a single-domain support agent that answers account and order questions by calling real backend functions, using the function-calling mechanics from Unit 8 as the primary approach. Function calling is chosen over the full Agents SDK (Unit 11) here deliberately: the domain is narrow and well-defined enough that explicit function schemas and a manual tool loop give more predictable, auditable behavior than handing control to an autonomous agent loop, which is better suited to open-ended, multi-step tasks like Project 9.

Scope and Design Decisions

The agent handles three kinds of requests: looking up order status, checking account details, and issuing refunds within policy limits. It talks to a set of backend functions rather than a database directly, which mirrors how a real support tool would sit in front of existing internal APIs.

Three decisions shape the design:

1. Function calling, not the Agents SDK, for this domain. With a fixed, small set of well-understood operations, a manual loop over client.responses.create with explicit tools gives full control over exactly which functions can run and in what order, which matters for an agent that can issue refunds.
2. Refunds require a policy check before execution, not just a function call. The model deciding to call issue_refund is not the same as the refund being approved — a policy layer sits between the model's tool call and the actual side-effecting operation.
3. Every tool call and result is logged. Support interactions that touch money need an audit trail independent of the model's own narration of what it did.

Defining the Tools

from openai import OpenAI
import json

client = OpenAI()

TOOLS = [
    {
        "type": "function",
        "name": "get_order_status",
        "description": "Look up the current status of an order by order ID.",
        "parameters": {
            "type": "object",
            "properties": {"order_id": {"type": "string"}},
            "required": ["order_id"],
        },
    },
    {
        "type": "function",
        "name": "get_account_summary",
        "description": "Retrieve account details for a customer by customer ID.",
        "parameters": {
            "type": "object",
            "properties": {"customer_id": {"type": "string"}},
            "required": ["customer_id"],
        },
    },
    {
        "type": "function",
        "name": "issue_refund",
        "description": "Issue a refund for an order, subject to policy limits.",
        "parameters": {
            "type": "object",
            "properties": {
                "order_id": {"type": "string"},
                "amount_cents": {"type": "integer"},
                "reason": {"type": "string"},
            },
            "required": ["order_id", "amount_cents", "reason"],
        },
    },
]

Each tool's description and parameters schema is the entire interface the model has to the backend — it never sees the underlying implementation. Keeping issue_refund's parameters explicit (amount_cents as an integer, a required reason) means the model must commit to a specific, reviewable refund amount and justification rather than issuing a vague or open-ended action, which is exactly the kind of interface discipline Unit 8 emphasizes for any tool with real-world side effects.

Backend Implementations and the Refund Policy Gate

FAKE_ORDERS = {
    "ORD-1001": {"status": "shipped", "total_cents": 4999, "customer_id": "CUST-1"},
}
FAKE_CUSTOMERS = {
    "CUST-1": {"name": "Jordan Lee", "tier": "standard"},
}
REFUND_POLICY_LIMIT_CENTS = 5000

def get_order_status(order_id: str) -> dict:
    order = FAKE_ORDERS.get(order_id)
    if not order:
        return {"error": f"No order found with ID {order_id}"}
    return {"order_id": order_id, "status": order["status"]}

def get_account_summary(customer_id: str) -> dict:
    customer = FAKE_CUSTOMERS.get(customer_id)
    if not customer:
        return {"error": f"No customer found with ID {customer_id}"}
    return {"customer_id": customer_id, **customer}

class RefundDeniedError(Exception):
    pass

def issue_refund(order_id: str, amount_cents: int, reason: str) -> dict:
    order = FAKE_ORDERS.get(order_id)
    if not order:
        return {"error": f"No order found with ID {order_id}"}
    if amount_cents > REFUND_POLICY_LIMIT_CENTS:
        raise RefundDeniedError(
            f"Refund of {amount_cents} cents exceeds the auto-approval limit "
            f"of {REFUND_POLICY_LIMIT_CENTS} cents and requires human review."
        )
    if amount_cents > order["total_cents"]:
        raise RefundDeniedError("Refund amount exceeds the original order total.")

    # In a real system this would call a payments API.
    return {"order_id": order_id, "refunded_cents": amount_cents, "status": "refund_issued"}

TOOL_IMPLEMENTATIONS = {
    "get_order_status": get_order_status,
    "get_account_summary": get_account_summary,
    "issue_refund": issue_refund,
}

issue_refund raises RefundDeniedError rather than silently capping the amount or returning a generic failure. This distinction matters for the tool loop below: a policy violation is a distinct, expected outcome that should be reported back to the model as a structured denial reason (so it can explain the situation to the customer), not swallowed as an unhandled exception or treated the same as "order not found." Separating "denied by policy" from "not found" from "success" gives the model enough information to respond appropriately in each case.

The Tool-Calling Loop

def run_support_agent(user_message: str, conversation: list[dict] | None = None) -> str:
    conversation = conversation or []
    conversation.append({"role": "user", "content": user_message})

    for _ in range(5):  # bounded to prevent runaway tool-call loops
        response = client.responses.create(
            model="gpt-5.6-terra",
            input=conversation,
            tools=TOOLS,
        )

        tool_calls = [item for item in response.output if item.type == "function_call"]
        if not tool_calls:
            final_text = response.output_text
            conversation.append({"role": "assistant", "content": final_text})
            return final_text

        conversation.extend(response.output)

        for call in tool_calls:
            args = json.loads(call.arguments)
            impl = TOOL_IMPLEMENTATIONS[call.name]
            try:
                result = impl(**args)
            except RefundDeniedError as exc:
                result = {"denied": True, "reason": str(exc)}

            print(f"AUDIT: tool={call.name} args={args} result={result}")

            conversation.append({
                "type": "function_call_output",
                "call_id": call.call_id,
                "output": json.dumps(result),
            })

    return "I was unable to complete this request after several tool calls. A human agent will follow up."

The loop bounds itself to five iterations — an important safeguard from Unit 12's production-readiness patterns applied here: without a bound, a model that keeps calling tools without converging on a final answer would run indefinitely, consuming API calls and, worse in this domain, potentially issuing repeated refund attempts. The RefundDeniedError is caught specifically and turned into a structured {"denied": True, "reason": ...} result rather than an unhandled exception, so the model receives a clear, actionable reason it can relay to the customer instead of the agent crashing outright.

The print(f"AUDIT: ...") line stands in for a real audit log (a database row or structured log line shipped to a logging pipeline in production); it is placed immediately after every tool execution, capturing the exact arguments and result independent of whatever the model later says happened — which is precisely the property an audit trail for money-moving actions needs.

Testing the Policy Gate Without the Model

def test_refund_within_limit_succeeds():
    result = issue_refund("ORD-1001", 2000, "damaged item")
    assert result["status"] == "refund_issued"
    print("PASS: refund within policy limit succeeds")

def test_refund_over_limit_is_denied():
    try:
        issue_refund("ORD-1001", 10000, "customer request")
        assert False, "expected RefundDeniedError"
    except RefundDeniedError as exc:
        assert "exceeds the auto-approval limit" in str(exc)
        print("PASS: refund over policy limit is denied")

def test_refund_over_order_total_is_denied():
    try:
        issue_refund("ORD-1001", 4999 + 1, "over-refund attempt")
        assert False, "expected RefundDeniedError"
    except RefundDeniedError:
        print("PASS: refund exceeding order total is denied")

test_refund_within_limit_succeeds()
test_refund_over_limit_is_denied()
test_refund_over_order_total_is_denied()

These tests exercise issue_refund directly, entirely independent of the model, the tool-calling loop, or any API call. This is deliberate: the policy logic is the part of this agent with real financial consequences, and it needs to be correct and independently verifiable regardless of what the model decides to do. Testing run_support_agent end to end would additionally require a fake client.responses.create that returns scripted tool-call sequences — worth adding in a fuller test suite, but the policy gate is the higher-priority unit to cover first.

Extending This Project

Add a human-in-the-loop step for denied refunds so a support supervisor can approve an over-limit request, and add per-tool rate limiting so a single conversation cannot trigger an unbounded number of backend calls even within the five-iteration bound.

Common Mistakes

• Letting the model's tool call directly trigger a side effect with no policy layer. The model choosing to call issue_refund is a request, not an authorization; always validate against business rules before the effect happens.
• Returning the same generic error for "not found," "denied by policy," and "invalid input." The model cannot respond helpfully to the customer if every failure looks identical. Distinguish failure types in the returned structure.
• Omitting a bound on the tool-calling loop. An unbounded loop risks runaway API usage and, in domains with side effects, repeated unintended actions if the model does not converge.

Best Practices

• Keep an audit log independent of the model's own narration. Log tool name, arguments, and result at the moment of execution, not based on what the assistant later claims it did.
• Design tool parameter schemas to force commitment. Requiring an explicit amount and reason for a refund, rather than accepting free-form intent, produces reviewable, unambiguous actions.
• Choose function calling over an autonomous agent framework for narrow, well-defined domains. The explicit loop in this project is easier to audit and bound than a general-purpose agent loop, and that predictability is worth more than the Agents SDK's added convenience when the tool surface is small and fixed.