Avoid Unsafe Tool Calls

A Permitted Call Can Still Be Unsafe

Lesson 176 decided which tools a user may use and which resources they may touch. This lesson addresses a different question: even when a call is permitted, what can the arguments make it do? A tool that reads a file is allowed to read files, but a file name that climbs out of the intended folder turns a permitted read into a read of the whole machine. A tool that runs a search is allowed to query a database, but a string pasted into a query can change what the database does.

The arguments to a tool come from the model, and the model's choices can be shaped by the user, by retrieved content, or by an injection (Lesson 175). Treating those arguments as trusted input is the mistake this lesson guards against. Every argument is validated for its meaning, not just its type, and every operation that cannot be undone is held for a person to confirm.

The Patterns That Recur

A small number of unsafe patterns account for most tool vulnerabilities:

Each row names a defense that the earlier lessons already introduced. The point of this lesson is to apply them at the tool boundary, where model-supplied values enter the application.

Path Traversal

A tool that reads files should resolve the requested path against its base folder, and refuse any result that lies outside it. Relative segments such as .. and absolute paths both have to be handled, because either can escape a folder by simple string concatenation:

"""safe_tools.py: validate paths, queries, and destructive actions at the tool boundary."""

import sqlite3
from pathlib import Path

BASE = Path("/srv/app/data")


def safe_path(user_path: str) -> Path | None:
    candidate = (BASE / user_path).resolve()
    try:
        candidate.relative_to(BASE.resolve())
    except ValueError:
        return None
    return candidate


def describe_path(user_path: str) -> str:
    return "denied" if safe_path(user_path) is None else "allowed"


if __name__ == "__main__":
    for user_path in ["reports/q3.txt", "../etc/passwd", "reports/../../secret.txt", "/etc/passwd"]:
        print(f"{user_path}: {describe_path(user_path)}")

Output:

reports/q3.txt: allowed
../etc/passwd: denied
reports/../../secret.txt: denied
/etc/passwd: denied

How It Works

1. resolve() removes the relative segments. reports/../../secret.txt becomes /srv/app/secret.txt, which is visible only after resolution, so the check must run on the resolved path.
2. relative_to() tests containment. It raises an error when the candidate is not under the base folder, and the function turns that error into a refusal.
3. Absolute paths are refused by the same test. Joining with an absolute path replaces the base, so /etc/passwd resolves outside the folder and is caught without any special case.
4. The base is resolved too, so that a base reached through a link is compared correctly with the resolved candidate.

Query Injection

A database tool should never build a query by inserting values into a string. A value that contains quotation marks can rewrite the query, and the database will run whatever it has been given. The defense is parameters: the query text is fixed, and values are sent separately, so they are always treated as data.

db = sqlite3.connect(":memory:")
db.execute("CREATE TABLE courses (slug TEXT, title TEXT)")
db.execute("INSERT INTO courses VALUES ('django-101', 'Django for Beginners')")

hostile = "x' OR '1'='1"
safe_rows = db.execute("SELECT * FROM courses WHERE slug = ?", (hostile,)).fetchall()
unsafe_rows = db.execute(f"SELECT * FROM courses WHERE slug = '{hostile}'").fetchall()
print("rows (parameterized):", len(safe_rows))
print("rows (unsafe string formatting):", len(unsafe_rows))
print("table still exists:", bool(db.execute("SELECT name FROM sqlite_master WHERE name = 'courses'").fetchall()))

The parameterized query treats the hostile value as a single slug, which matches nothing. The formatted query becomes a condition that is always true, and it returns every row. The example uses an in-memory database, so the table exists only for the demonstration.

Destructive Actions Need a Person

Some tool calls cannot be undone: a cancellation, a deletion, a payment, a message sent to a customer. For these, a permission check is not enough, because the permission may be correct and the instruction may still be wrong. A confirmation gate holds the action until a person approves it. The gate is part of the tool's dispatch, not something the model can skip:

DESTRUCTIVE = {"cancel_enrollment", "delete_file", "send_message"}


def gate(tool: str, human_confirmed: bool) -> str:
    if tool in DESTRUCTIVE and not human_confirmed:
        return "pending confirmation"
    return "run"


print("cancel_enrollment:", gate("cancel_enrollment", human_confirmed=False))
print("search_courses:", gate("search_courses", human_confirmed=False))

The confirmation shows the person the exact action, with its arguments, before anything happens. The model's description of the action is not enough, because the arguments are what will be executed.

Never Execute Model-Supplied Code

A tool that evaluates an expression or runs a script from the model gives the model the full power of the application. Lesson 100 showed why eval is dangerous, and the same reasoning applies to any tool that runs code from a model. If a tool needs computation, it should accept a structured request and perform a fixed operation, not run text the model wrote.

Summary

Permitted tool calls can still be unsafe because their arguments come from the model. Path traversal is defeated by resolving and checking containment, query injection by passing parameters, and command injection by never using a shell. Code execution from model output is never allowed, and network targets are restricted to known hosts. Destructive actions wait for a person to confirm the exact operation. Each defense is applied at the tool boundary, where the model's values enter the application. The next lesson turns to the other direction: the results that tools return to the model.