What the Moderation Endpoint Checks

Unit 11, Lesson 5 introduced guardrails as a way to enforce application-specific rules — refund amounts, prompt injection phrases specific to a support desk. The moderation endpoint addresses a different, narrower, and more general concern: whether a piece of text (something a user submitted, or something the model generated) contains content in categories like hate speech, harassment, self-harm, or violence, independent of any application-specific business logic at all.

from openai import OpenAI

client = OpenAI()

moderation_result = client.moderations.create(input="I want to build a birdhouse this weekend.")

result = moderation_result.results[0]
print(f"Flagged: {result.flagged}")
print(f"Categories flagged: {[cat for cat, flagged in result.categories.__dict__.items() if flagged]}")

Note: The exact set of moderation categories, the specific structure of the categories and category_scores objects, and the moderation model used can all change over time as the platform's safety systems are updated. Confirm the current category list and response structure against the current official documentation before building logic that depends on a specific category name.

result.flagged is a single boolean summarizing whether any category was triggered, while result.categories gives the specific breakdown — checking the specific categories, rather than only the overall flagged boolean, matters when different categories warrant different application responses (a violence-related flag and a self-harm-related flag are both serious, but they call for different follow-up actions, as the next section covers).

Checking Both User Input and Model Output

A moderation check applied only to what a user submits misses an entire category of risk: the model's own generated output, which is not guaranteed to avoid these categories on its own, especially under an adversarial or unusual input.

def is_content_safe(client, text: str) -> bool:
    result = client.moderations.create(input=text).results[0]
    return not result.flagged

def generate_moderated_response(client, user_message: str) -> str:
    if not is_content_safe(client, user_message):
        return "I'm not able to help with that request."

    response = client.responses.create(model="gpt-5.6-terra", input=user_message)
    output_text = response.output_text

    if not is_content_safe(client, output_text):
        return "I generated a response that didn't meet content guidelines, so I've withheld it."

    return output_text

This two-sided check reflects a distinction worth making deliberately: moderating only the input protects against a user submitting harmful content, but does nothing about a model response that turns out to be problematic despite an entirely innocuous input — checking the output as well closes that gap, at the cost of an additional moderation call per request.

Moderation and Guardrails Solve Different Problems

Unit 11, Lesson 5 distinguished application-specific guardrails from platform-wide moderation in the context of a single agent; the same distinction applies across this entire course, and it's worth being explicit about which one to reach for.

Neither one is a substitute for the other in a production system — a support desk (Unit 11's capstone project) benefits from both: platform moderation catching genuinely harmful content in either direction, and application guardrails enforcing business rules specific to that support desk that platform moderation was never designed to know about at all.

Handling Flagged Content Thoughtfully

Not every flagged category deserves an identical response — a blanket "reject everything flagged" policy is simpler to write but often not the most appropriate response for every situation a real application will encounter.

def handle_moderation_result(result) -> dict:
    if not result.flagged:
        return {"action": "proceed"}

    flagged_categories = [cat for cat, is_flagged in result.categories.__dict__.items() if is_flagged]

    self_harm_categories = [cat for cat in flagged_categories if "self-harm" in cat or "self_harm" in cat]
    if self_harm_categories:
        return {"action": "provide_support_resources", "categories": self_harm_categories}

    return {"action": "reject", "categories": flagged_categories}

Note: The exact category names available (and therefore the specific substring matching shown here) can vary across moderation model versions. Confirm current category names against the current official documentation rather than assuming these specific strings.

Content flagged for self-harm-related categories, in particular, deserves a materially different response than content flagged for something like harassment — a blanket rejection is a poor response to someone expressing distress, whereas providing supportive resources (without the application attempting to serve as a substitute for real professional support) is more appropriate; this kind of category-aware handling is exactly why checking specific categories, not just the overall flagged boolean, matters in practice.

Logging Safety Events Without Over-Retaining Sensitive Content

Tracking how often moderation flags trigger, and for which categories, is valuable for understanding an application's actual risk profile over time — but what gets logged, and for how long, deserves the same deliberate care Unit 11, Lesson 6 raised about tracing potentially sensitive tool arguments.

import time

def log_moderation_event(result, request_id: str) -> None:
    if not result.flagged:
        return

    flagged_categories = [cat for cat, is_flagged in result.categories.__dict__.items() if is_flagged]
    # Log which categories triggered and when, without necessarily retaining
    # the full flagged text itself, depending on your organization's data
    # handling and retention requirements.
    print(f"[{time.time()}] request {request_id} flagged: {flagged_categories}")

Logging that a request was flagged, which categories triggered, and when, supports monitoring and trend analysis without necessarily requiring the full flagged content to be retained indefinitely — whether to retain the actual flagged text, and for how long, is a decision that should follow your organization's own data handling and retention requirements rather than a default assumption either way.

Common Mistakes

Moderating only user input and never the model's own generated output, missing the case where a response turns out to be problematic despite a perfectly innocuous input.

Treating every flagged category identically with a blanket rejection, rather than responding differently to categories — such as self-harm-related content — that call for a materially different, more supportive response.

Relying on platform moderation alone as a substitute for application-specific guardrails, when the two address different concerns and a production system generally needs both, as Unit 11, Lesson 5 established.

Logging and retaining full flagged content indefinitely without considering data handling requirements, applying less care to safety-event logs than the application applies elsewhere to sensitive data.

Best Practices

Check moderation on both user input and model-generated output, not input alone, to catch problems in either direction.

Respond differently to different flagged categories, particularly treating self-harm-related flags as calling for supportive handling rather than a blanket rejection.

Combine platform moderation with your own application-specific guardrails, since each addresses a different category of risk that the other was never designed to catch.

Apply the same data handling discipline to safety-event logging as to any other sensitive data, deciding deliberately what to retain and for how long rather than defaulting to indefinite full-content logging.