Log Tool Activity

Why Log Tool Activity

With tools, Claude's work is no longer only text: it takes actions in your systems. When something goes wrong, such as a wrong answer, a failed booking, a slow response, or an unexpected cost, you need to know what happened:

Logging tool activity is part of running tools responsibly (Lessons 176–178), not just a debugging aid.

What the SDK Logs for You

According to the tool runner documentation:

• When a tool raises an unhandled exception, the Python SDK logs the full exception, including its stack trace, through Python's standard logging module, while Claude receives only the exception's type and message.
• The ANTHROPIC_LOG environment variable turns on the SDK's own logging of requests and responses:

export ANTHROPIC_LOG=info     # or: debug, for more verbose output

ANTHROPIC_LOG=debug is useful during development, but too verbose, and potentially too revealing, for production. For production, add structured logs of your own at two levels: tools and model requests.

Two Levels of Logs

A shared run ID, a unique identifier for one user request, connects all entries from the same run, so you can reconstruct exactly what happened.

Logging Tool Calls with a Wrapper

As in Lesson 96, a wrapper with functools.wraps adds behavior to a tool without changing it, and keeps the generated definition intact. @beta_tool must be outermost:

@beta_tool
@audit.wrap
def search_courses(topic: str) -> str:
    ...

The wrapper measures the duration with time.perf_counter(), logs one entry per call, and re-raises exceptions, so the tool runner still turns them into error results for Claude.

Redact Sensitive Inputs

Tool inputs can contain personal data: emails, phone numbers, free-text messages. Logs are often widely accessible and kept for a long time. Redact sensitive fields before logging, and log sizes of results rather than full content:

SENSITIVE = {"email", "phone", "password", "token", "message"}
safe_input = {key: ("[redacted]" if key in SENSITIVE else value) for key, value in kwargs.items()}

Structured (JSON) Logs

As in Lesson 49, write each log entry as one line of JSON. Log platforms can then filter and aggregate by field, for example "all tool_call entries with outcome error in the last hour", or "average duration_ms per tool".

Practical Example: An Audited Tool Run

This example logs every tool call and every model request of a run as JSON lines, linked by a run ID. One tool fails, to show how errors appear in the log.

"""audited_run.py: structured logs for tool calls and model requests."""

import functools
import json
import logging
import time
import uuid

from anthropic import Anthropic, beta_tool

logging.basicConfig(level=logging.INFO, format="%(message)s")
log = logging.getLogger("tool_audit")
SENSITIVE = {"email", "phone", "password", "token", "message"}


class Audit:
    """Logs tool activity for one run."""

    def __init__(self, user_id: str):
        self.run_id = uuid.uuid4().hex[:12]
        self.user_id = user_id

    def event(self, **fields) -> None:
        log.info(json.dumps({"run_id": self.run_id, "user": self.user_id, **fields}, ensure_ascii=False))

    def wrap(self, function):
        @functools.wraps(function)
        def wrapper(*args, **kwargs):
            safe_input = {k: ("[redacted]" if k in SENSITIVE else v) for k, v in kwargs.items()}
            start = time.perf_counter()
            try:
                result = function(*args, **kwargs)
            except Exception as error:
                self.event(event="tool_call", tool=function.__name__, input=safe_input, outcome="error",
                           error=type(error).__name__,
                           duration_ms=round((time.perf_counter() - start) * 1000, 1))
                raise
            self.event(event="tool_call", tool=function.__name__, input=safe_input, outcome="ok",
                       result_chars=len(result),
                       duration_ms=round((time.perf_counter() - start) * 1000, 1))
            return result
        return wrapper


def make_tools(audit: Audit) -> list:
    @beta_tool
    @audit.wrap
    def search_courses(topic: str) -> str:
        """Search the GOCOURSE catalog for courses on one topic.

        Use it when a student asks which courses exist for a subject.
        Returns course titles as JSON.

        Args:
            topic: One subject, e.g. 'Django'
        """
        catalog = {"django": ["Django for Beginners", "Advanced Django APIs"], "sql": ["SQL Basics"]}
        return json.dumps(catalog.get(topic.lower(), []))

    @beta_tool
    @audit.wrap
    def get_course_fee(course_title: str) -> str:
        """Return the fee for one course in Indian rupees.

        Use it when a student asks what a course costs.

        Args:
            course_title: Exact course title from search_courses
        """
        raise TimeoutError("payments service timeout")     # Simulated outage

    return [search_courses, get_course_fee]


def run(user_id: str, question: str) -> str:
    audit = Audit(user_id)
    runner = Anthropic().beta.messages.tool_runner(
        model="claude-opus-5-5",
        max_tokens=4096,
        max_iterations=6,
        system="You are the GOCOURSE assistant. If a tool fails, say what you could not check.",
        tools=make_tools(audit),
        messages=[{"role": "user", "content": question}],
    )
    last = None
    for step, message in enumerate(runner, start=1):
        audit.event(event="model_response", step=step, message_id=message.id,
                    stop_reason=message.stop_reason,
                    tools_requested=[b.name for b in message.content if b.type == "tool_use"],
                    input_tokens=message.usage.input_tokens, output_tokens=message.usage.output_tokens)
        last = message
    return "".join(b.text for b in last.content if b.type == "text")


if __name__ == "__main__":
    print(run("meena", "Which beginner Django course do you have, and what does it cost?"))

Example output (log lines, then the answer; run IDs, message IDs, token counts, and durations vary):

{"run_id": "4f1c2a9be07d", "user": "meena", "event": "model_response", "step": 1, "message_id": "msg_01...", "stop_reason": "tool_use", "tools_requested": ["search_courses"], "input_tokens": 912, "output_tokens": 88}
{"run_id": "4f1c2a9be07d", "user": "meena", "event": "tool_call", "tool": "search_courses", "input": {"topic": "Django"}, "outcome": "ok", "result_chars": 48, "duration_ms": 0.1}
{"run_id": "4f1c2a9be07d", "user": "meena", "event": "model_response", "step": 2, "message_id": "msg_01...", "stop_reason": "tool_use", "tools_requested": ["get_course_fee"], "input_tokens": 1034, "output_tokens": 71}
{"run_id": "4f1c2a9be07d", "user": "meena", "event": "tool_call", "tool": "get_course_fee", "input": {"course_title": "Django for Beginners"}, "outcome": "error", "error": "TimeoutError", "duration_ms": 0.0}
{"run_id": "4f1c2a9be07d", "user": "meena", "event": "model_response", "step": 3, "message_id": "msg_01...", "stop_reason": "end_turn", "tools_requested": [], "input_tokens": 1150, "output_tokens": 64}
Our beginner Django course is Django for Beginners. I couldn't check its fee right now because the payments service isn't responding. Please try again later.

How It Works

1. Audit is created per run, with a short unique run_id from uuid.uuid4(). Every entry includes the run ID and the user, so a single filter on run_id reconstructs the whole run.
2. audit.wrap wraps each tool. It redacts sensitive keys, times the call with time.perf_counter(), logs either an ok entry with the result size, or an error entry with the exception type, and re-raises errors so the tool runner can still send them to Claude. The search result ["Django for Beginners", "Advanced Django APIs"] is 48 characters long as JSON text, which is what result_chars records.
3. @beta_tool is outermost, and functools.wraps keeps each tool's name, docstring, and annotations, so the generated definitions are unchanged (Lesson 96).
4. The runner loop logs one model_response entry per step: message ID, stop reason, requested tools, and token usage. These entries show the order of decisions and the cost of each step.
5. Ordering: the runner yields a message first, so its model_response entry is logged, and then runs the requested tools when the loop continues, so the tool_call entries follow. Reading the log top to bottom shows the run exactly as it happened.
6. The failing tool appears as outcome: "error" with error: "TimeoutError". The Python SDK additionally logs the full traceback, as documented, so developers can investigate, while Claude received only a short error result and answered honestly.

Note: Logs are data about your users. Apply retention limits, restrict access, and never log secrets. Redact personal fields, and prefer result sizes over full results unless you have a clear need and appropriate protections.

Common Mistakes

Logging Without a Correlation ID

What happens: tool calls from different users and requests are mixed, and runs cannot be reconstructed.

Fix: include a run ID (and user ID) in every entry.

Swallowing Exceptions in the Logging Wrapper

What happens: Claude receives None or a misleading result instead of an error.

Fix: log, then re-raise.

Logging Sensitive Inputs and Full Results

What happens: personal data accumulates in logs.

Fix: redact sensitive fields, and log sizes or summaries.

Unstructured Log Messages

What happens: "Tool search_courses called with Django" is hard to filter or aggregate.

Fix: log JSON with consistent field names.

Leaving Debug Logging On in Production

What happens: ANTHROPIC_LOG=debug writes request and response details into production logs.

Fix: use debug logging only during development.

Best Practices

• Log every tool call and every model response, linked by a run ID.
• Record durations, outcomes, error types, and token usage.
• Re-raise exceptions after logging them.
• Redact sensitive inputs, and keep results out of logs by default.
• Use structured JSON logs with consistent fields.
• Use ANTHROPIC_LOG for development diagnostics, not routine production logging.
• Review logs regularly for failing tools, slow tools, and unusual call patterns.

Exercise

1. Add the total number of tool calls and the total tokens to a final run_summary log entry.
2. Change SENSITIVE handling to also redact any string value longer than 200 characters.
3. Write a script that reads the JSON log lines and prints, per tool, the number of calls, the error rate, and the average duration.
4. Explain why the wrapper logs result_chars instead of the full result.

Step 3:

import json
from collections import defaultdict


def tool_stats(lines: list[str]) -> None:
    stats = defaultdict(lambda: {"calls": 0, "errors": 0, "total_ms": 0.0})
    for line in lines:
        entry = json.loads(line)
        if entry.get("event") != "tool_call":
            continue
        s = stats[entry["tool"]]
        s["calls"] += 1
        s["errors"] += entry["outcome"] == "error"
        s["total_ms"] += entry["duration_ms"]
    for tool, s in stats.items():
        print(f"{tool:<16} calls={s['calls']:<4} error_rate={s['errors'] / s['calls']:.0%} "
              f"avg_ms={s['total_ms'] / s['calls']:.1f}")

Step 4: full results can be large, which makes logs expensive to store and slow to search, and they can contain personal or sensitive data from your systems. The result size is usually enough to spot problems, such as empty results or unexpectedly huge ones. When full results are genuinely needed for debugging, log them separately, with restricted access and short retention.