Logging safely without exposing confidential content

Unit 12, Lesson 7 touched on safe logging in the specific context of moderation — noting that flagged content shouldn't be logged in full. This lesson treats logging as its own discipline across the entire application, not just around moderation events, because logs are one of the most common places sensitive data quietly accumulates. A team that carefully protects API keys (Lessons 1-3) and redacts PII before sending it to the model (Lesson 7) can still leak all of it right back out through an unguarded logger.info(f"Request: {prompt}") line.

Why logs are a distinct risk surface

Logs are treated, culturally, as low-stakes debugging exhaust — which is exactly why they're dangerous. Compared to a database, logs typically have:

• Broader access. More engineers have read access to application logs than to the production database, often including on-call staff, contractors, and third-party log-aggregation services (Datadog, Splunk, CloudWatch, and similar).
• Longer, less deliberate retention. Logs are frequently retained by default configuration (30 days, 90 days) without anyone having made an explicit decision to store that specific content for that specific duration.
• Less structure and less review. A database schema is designed; log statements are added ad hoc by whoever is debugging a feature at 2 a.m., with no equivalent review process to catch a stray sensitive field.

A full user prompt or a full model completion can easily contain the exact PII, secrets, or proprietary business content you've worked to protect everywhere else in your stack. Logging it as a debugging convenience undoes that work.

Deciding what to log and what not to

The guiding question for every log statement touching an AI request should be: does this field help me debug or monitor the system, and if so, do I need its full content or just metadata about it? Most operational needs are met by metadata alone.

This table generalizes the point from Unit 12, Lesson 7 (don't log full flagged content) to the entire request/response lifecycle: metadata is almost always what you actually need for monitoring, alerting, and debugging aggregate behavior; full content is rarely needed and is exactly the material you spent Lessons 1, 2, 3, and 7 of this unit trying to protect.

Building a redacting log filter

Rather than relying on every engineer to remember to redact manually at every call site — which fails the moment someone is in a hurry — build the redaction into your logging setup itself, so it happens automatically regardless of what an individual log statement contains.

import logging
import re

_SECRET_PATTERN = re.compile(r"sk-[A-Za-z0-9-]{16,}")
_EMAIL_PATTERN = re.compile(r"[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}")


class RedactingFilter(logging.Filter):
    """
    A logging Filter that scrubs likely secrets and emails from every
    log record's message before it reaches any handler (console, file,
    or a remote log aggregator).
    """

    def filter(self, record: logging.LogRecord) -> bool:
        message = record.getMessage()
        message = _SECRET_PATTERN.sub("[REDACTED_KEY]", message)
        message = _EMAIL_PATTERN.sub("[REDACTED_EMAIL]", message)
        # Overwrite the record so downstream handlers see the redacted text.
        record.msg = message
        record.args = ()
        return True


def build_safe_logger(name: str) -> logging.Logger:
    logger = logging.getLogger(name)
    logger.addFilter(RedactingFilter())
    return logger

A logging.Filter in Python runs on every log record passed through a logger that has it attached, and returning True means "allow this record through" (returning False would suppress it entirely — not what we want here, we want to alter it, not drop it). record.getMessage() renders the final message string (applying any %s-style formatting args first), and then the filter overwrites record.msg with the redacted version and clears record.args so the original, unredacted arguments aren't re-applied later by a handler's formatter.

This is a safety net, not a replacement for deliberate logging discipline — it catches an accidental sk-... string or an email address that slipped into a log line, but it cannot catch every category of sensitive content (a name, a physical address, a proprietary contract clause). Treat it as a last line of defense behind the discipline of choosing what to log in the first place.

import io

def test_redacting_filter_masks_key_and_email():
    logger = logging.getLogger("test.safe_logging")
    logger.setLevel(logging.INFO)
    logger.addFilter(RedactingFilter())

    buffer = io.StringIO()
    handler = logging.StreamHandler(buffer)
    logger.addHandler(handler)

    logger.info("Request failed for user jane@example.com using key sk-proj-abcdef1234567890")
    handler.flush()
    output = buffer.getvalue()

    assert "jane@example.com" not in output
    assert "sk-proj-abcdef1234567890" not in output
    assert "[REDACTED_EMAIL]" in output
    assert "[REDACTED_KEY]" in output
    logger.removeHandler(handler)
    print("PASS: RedactingFilter scrubs emails and key-shaped strings from log output")


test_redacting_filter_masks_key_and_email()

Note: The test above attaches a logging.StreamHandler pointed at an in-memory io.StringIO buffer instead of a real console or file handler — this is dependency injection applied to the logging system, letting the test inspect exactly what was written without touching real output streams.

Structuring logs so redaction is easier to enforce

Unstructured log lines (logger.info(f"User {user_id} asked: {prompt}")) tempt developers into interpolating whatever variables are in scope, prompt included. Structured logging — emitting a dictionary of named fields rather than a single formatted sentence — makes it much easier to enforce a policy of "only these specific fields are allowed," because the fields are explicit and reviewable rather than buried inside an f-string.

import logging
import json

logger = logging.getLogger("app.requests")


def log_model_request(request_id: str, model: str, prompt_char_count: int,
                       latency_ms: float, status: str) -> None:
    """
    Logs metadata about a model request without ever touching the
    actual prompt or completion content.
    """
    logger.info(json.dumps({
        "request_id": request_id,
        "model": model,
        "prompt_char_count": prompt_char_count,
        "latency_ms": latency_ms,
        "status": status,
    }))

Notice that log_model_request's signature simply has no parameter for the raw prompt or completion text at all — it takes prompt_char_count, a length, rather than the prompt itself. This is a structural guarantee, similar in spirit to the tools=[] pattern from Lesson 4: if the function has no way to accept the sensitive value, no call site can accidentally pass it through, no matter how the function evolves later.

Separating debug logging from production logging

During local development, it is often genuinely useful to see full prompts and completions to understand why a model produced a given output. The resolution is not "never log full content," but rather "make sure full-content logging is explicitly scoped to environments and audiences where it's appropriate."

import os

def build_request_logger() -> logging.Logger:
    logger = logging.getLogger("app.requests")
    is_production = os.environ.get("ENVIRONMENT") == "production"
    if not is_production:
        # Verbose, full-content logging only outside production, and only
        # to local output — never shipped to a remote aggregator.
        handler = logging.StreamHandler()
        handler.setLevel(logging.DEBUG)
    else:
        handler = logging.StreamHandler()
        handler.setLevel(logging.INFO)
        handler.addFilter(RedactingFilter())
    logger.addHandler(handler)
    return logger

This makes the tradeoff explicit and environment-gated rather than an accident of whichever log statements happen to exist: verbose debugging is available where it's needed and safe (a developer's own machine, non-production data), and production traffic — which is far more likely to carry real customer data — always goes through the redacting filter.

Common Mistakes

• Logging the full prompt or completion "temporarily" to debug an issue, and forgetting to remove it. Debug logging added under time pressure is exactly the kind of line that survives into production, exactly the outcome Lesson 2's "quick hardcoded key" mistake also describes for a different kind of secret.
• Assuming a third-party log aggregator's access controls make raw content logging safe. The aggregator's access control only matters if it's configured correctly and if every person with access should be trusted with the sensitive content — often a much larger set of people than those who need it for debugging.
• Relying entirely on a single redacting filter and skipping deliberate choices about what to log. A filter is a safety net for patterns it recognizes; it will not catch a customer's name, physical address, or a proprietary business detail embedded in free text.

Best Practices

• Log metadata by default — IDs, counts, timings, statuses — and only add full content logging deliberately, scoped to non-production environments.
• Attach a redacting filter to every production logger as a safety net against secrets and common PII patterns slipping through.
• Prefer structured logging with an explicit, reviewable set of fields over interpolated free-form strings, so what's being logged is visible at the call site.
• Treat log retention and access as seriously as database access — apply the same "who can see this and for how long" scrutiny to your logging pipeline that you'd apply to a database containing the same data.