Limits of Response Chaining

When Response Chaining Isn't Enough

Lesson 3 covered previous_response_id, a convenient way to chain a sequence of responses together without manually managing a growing message list. But it has a real ceiling: response chains depend on OpenAI's default ~30-day storage retention, they're identified by whatever the most recent response's ID happens to be (which changes on every turn), and they're not designed to be looked up, listed, or managed as a first-class object in their own right.

The Conversations API fixes exactly this gap. It gives you a dedicated conversation object with its own stable ID — one that doesn't change turn to turn, doesn't expire on the same schedule as an individual response, and can be explicitly created, retrieved, and managed independently of any specific responses.create() call. If you're building a feature where a user might return to the same conversation days, weeks, or months later — possibly from a different device — this is the tool built for that job.

Setup

pip install openai python-dotenv

OPENAI_API_KEY=your_api_key_here

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

Creating a Conversation

conversation = client.conversations.create()
print(conversation.id)

Expected Output

conv_68f1b2c3d4e5f6a7b8c9d0e1

This single call creates a durable conversation object on OpenAI's servers. Unlike a response ID, which is really just a pointer to one specific turn, this conversation.id refers to the conversation itself — a container that will hold every message and tool interaction exchanged within it, for as long as the conversation exists.

Using a Conversation With the Responses API

To actually have a conversation, you connect a client.responses.create() call to your conversation object using the conversation parameter:

conversation = client.conversations.create()

first_response = client.responses.create(
    model="gpt-5.6-luna",
    conversation=conversation.id,
    input=[
        {"role": "developer", "content": "You are a helpful assistant for a small bakery's ordering system."},
        {"role": "user", "content": "Do you have gluten-free options?"},
    ],
)
print(first_response.output_text)

second_response = client.responses.create(
    model="gpt-5.6-luna",
    conversation=conversation.id,
    input="Great, can I order two of those for pickup tomorrow?",
)
print(second_response.output_text)

Expected Output

Yes! We offer a gluten-free chocolate chip cookie and a gluten-free
banana bread loaf. Would you like details on either?

I can help with that — which item would you like, the chocolate chip
cookies or the banana bread loaf, and what time works for pickup?

Notice the pattern: every call passes the same conversation.id, not a changing response ID. The API automatically appends each new turn's messages and outputs to that conversation, and automatically supplies the full accumulated context to the model on each subsequent call — you never have to track a "latest ID" the way you did with previous_response_id, and you never have to manually rebuild a message list the way you did in Lesson 2. The conversation ID itself is the one piece of state your application needs to hold onto.

Retrieving and Listing Conversation Items

Because a conversation is a genuine, addressable resource — not just an implicit chain — you can inspect its contents directly, independent of making a new model call:

# Fetch metadata about the conversation itself
conversation_details = client.conversations.retrieve(conversation.id)
print(conversation_details)

# List the actual messages and items within it
items = client.conversations.items.list(conversation.id)
for item in items.data:
    print(item.role if hasattr(item, "role") else item.type, "-", item)

This is something neither Lesson 2's manual lists nor Lesson 3's response chains give you directly in the same way: a way to ask OpenAI's servers "what's actually in this conversation right now," independent of continuing it with a new turn. That's genuinely useful for building an interface that needs to display a conversation's history — a "resume this chat" screen in a mobile app, for example — without your own backend needing to have stored a separate copy of every message itself.

Manually Adding Items to a Conversation

You're not limited to only adding messages by calling responses.create(). You can also explicitly add items to a conversation directly, which is useful for seeding a conversation with context before the first model call, or for injecting information that didn't come from a live model turn — like a summary of a prior support ticket, imported from your own database:

conversation = client.conversations.create()

client.conversations.items.create(
    conversation.id,
    items=[
        {
            "role": "developer",
            "content": "The customer has an active order #48213, shipped yesterday, expected to arrive Friday.",
        }
    ],
)

response = client.responses.create(
    model="gpt-5.6-luna",
    conversation=conversation.id,
    input="When will my order arrive?",
)
print(response.output_text)

Expected Output

Your order (#48213) shipped yesterday and is expected to arrive on
Friday.

The model correctly answers using context that was never part of a live back-and-forth exchange — it was injected directly as a conversation item. This pattern is genuinely useful for handing off context from another system (a CRM, a ticketing system, an account database) into a conversation without needing to awkwardly phrase it as a fake prior "message."

A Reusable Conversation Wrapper

import os
from dotenv import load_dotenv
from openai import OpenAI

load_dotenv()
client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])


class DurableConversation:
    """Wraps the Conversations API for a single, resumable conversation."""

    def __init__(self, model: str = "gpt-5.6-luna", conversation_id: str | None = None):
        self.model = model
        if conversation_id:
            # Resuming an existing conversation — just confirm it exists.
            self.conversation_id = client.conversations.retrieve(conversation_id).id
        else:
            self.conversation_id = client.conversations.create().id

    def send(self, user_text: str) -> str:
        response = client.responses.create(
            model=self.model,
            conversation=self.conversation_id,
            input=user_text,
        )
        return response.output_text

    def history(self) -> list:
        return list(client.conversations.items.list(self.conversation_id).data)


# First session
convo = DurableConversation()
print(convo.send("Hi, I'm planning a trip to Lisbon in October."))
saved_id = convo.conversation_id  # persist this somewhere durable — a database row, etc.

# ... time passes, maybe a different process or a different day entirely ...

# Resuming later, from just the saved ID
resumed = DurableConversation(conversation_id=saved_id)
print(resumed.send("What did I say I was planning?"))

Expected Output

That sounds exciting! Lisbon in October has great weather — mild
temperatures and fewer crowds than summer. Anything specific you'd
like help planning?

You mentioned you're planning a trip to Lisbon in October.

The second session correctly recalls the first session's content, even though it's a completely separate DurableConversation instance — potentially running in a different process, on a different day, or even in a different application entirely — because all that's needed to resume is the durable conversation_id, which is exactly the kind of thing you'd store as a column in your own users or chats database table.

Real-World Example: A Cross-Device Personal Assistant

Consider a genuinely realistic use case for this durability: a personal productivity assistant app available on both web and mobile. A user starts a conversation about restructuring their week's schedule on their laptop during a lunch break, then picks their phone back up that evening and wants to continue exactly where they left off.

# When a user starts a new assistant thread (web app)
def start_assistant_thread(user_id: str) -> str:
    conversation = client.conversations.create()
    save_conversation_id_to_user_record(user_id, conversation.id)  # your own database
    return conversation.id


# Any client (web or mobile) sending a message just needs the stored ID
def send_assistant_message(user_id: str, message: str) -> str:
    conversation_id = get_conversation_id_for_user(user_id)  # look up from your database
    response = client.responses.create(
        model="gpt-5.6-luna",
        conversation=conversation_id,
        input=message,
    )
    return response.output_text

Because the conversation_id is a stable, durable reference stored in your own database rather than something that changes turn to turn, both the web app and the mobile app can independently continue the exact same conversation just by looking up that one saved ID — neither client needs to maintain its own local copy of the conversation history, and there's no coordination problem between the two clients, since OpenAI's servers are the single source of truth for what's actually in the conversation. This is precisely the scenario where previous_response_id would be awkward (you'd need to somehow keep both clients in sync on "the latest response ID," and that ID changes every single turn) and where Lesson 2's manual approach would require you to build and maintain your own equivalent conversation-storage system from scratch. The Conversations API gives you this cross-device durability essentially for free.

Deleting a Conversation

Conversations, being durable, persist until you explicitly remove them (subject to whatever OpenAI's data retention policies specify at the account level) — there's no automatic 30-day expiration the way there is for an individual unreferenced response. When a conversation genuinely needs to be permanently removed — a user deleting their account and requesting data deletion, for instance — you do that explicitly:

client.conversations.delete(conversation.id)

This is worth building into your application deliberately if you're storing conversation IDs long-term, particularly for any product with data-deletion obligations under privacy regulations — deleting your own database's reference to the conversation ID isn't sufficient on its own if the underlying conversation and its content still exist on OpenAI's servers.

Attaching Metadata to a Conversation

A conversation object supports arbitrary metadata — small key/value pairs you can attach at creation time or update later, useful for tagging a conversation with information your own application cares about without needing a separate database lookup just to identify what a conversation is about:

conversation = client.conversations.create(
    metadata={"user_id": "usr_4471", "topic": "trip_planning", "app_version": "2.3"}
)

# Later, update metadata on an existing conversation
client.conversations.update(
    conversation.id,
    metadata={"user_id": "usr_4471", "topic": "trip_planning", "status": "resolved"},
)

Metadata is a lightweight, useful place to store small identifying tags — which user or account a conversation belongs to, what feature or topic it relates to, a status flag — directly alongside the conversation, rather than needing to always join against your own database just to know basic facts about it. It's not a substitute for your own database when you need to query across many conversations at scale (searching "all conversations tagged 'billing_dispute' from last month," for instance, is better served by your own indexed database), but for per-conversation bookkeeping, it removes an entire category of lookups your application would otherwise need to perform.

Comparing This to the (Deprecated) Assistants API

If you've encountered older OpenAI tutorials or codebases, you may have seen a "Threads" concept as part of the Assistants API — a similarly-durable, similarly server-managed conversation object. It's worth knowing that the Assistants API is on a deprecation path, being superseded by the Responses API combined with the Conversations API covered in this lesson. If you're starting a new project today, build on client.conversations and client.responses, not the Assistants API's client.beta.threads — the durable-conversation concept you'd have reached for there is now covered by the Conversations API directly, with a simpler, more current API surface and without the "beta" status the older Assistants API carried.

If you're maintaining an existing application built on Assistants API threads, migrating to the Conversations API is a reasonable modernization project: the core concept — a durable, addressable conversation container, separate from any single model call — carries over directly, even though the exact method names and object shapes differ. Check OpenAI's current migration guidance directly before undertaking this, since the specifics of a migration path can change as both APIs evolve.

Using Conversations With Tool Calling

Conversations aren't limited to plain text exchanges — they transparently carry forward tool/function call items too, which matters once you combine this unit's techniques with the function-calling material from Unit 6. If a model call within a conversation invokes a tool, that tool call and its result become part of the conversation's item history automatically, the same way a text message does:

response = client.responses.create(
    model="gpt-5.6-luna",
    conversation=conversation.id,
    input="What's the weather in Lisbon right now?",
    tools=[weather_tool_definition],  # defined elsewhere, per Unit 6
)

# If the model calls the tool, you execute it and send the result back,
# still scoped to the same conversation:
if response.output[0].type == "function_call":
    result = execute_weather_lookup(response.output[0].arguments)
    client.responses.create(
        model="gpt-5.6-luna",
        conversation=conversation.id,
        input=[
            {
                "type": "function_call_output",
                "call_id": response.output[0].call_id,
                "output": result,
            }
        ],
    )

Because the tool call and its eventual output are both part of the same conversation, a later turn in that same conversation can reference "the weather you just looked up" and the model will correctly recall not just the final answer, but the fact that a tool was actually invoked to get it — useful for building assistants that combine ongoing conversational context with real, live actions across multiple turns, rather than treating each tool-using turn as an isolated event.

Handling the Case Where a Conversation No Longer Exists

Because conversations are explicitly deletable (as covered below) and application bugs happen, it's worth defensively handling the case where a stored conversation_id no longer resolves to a real conversation — perhaps it was deleted, or the ID was corrupted somewhere in storage:

from openai import NotFoundError

def resume_or_create(stored_conversation_id: str | None) -> str:
    if stored_conversation_id:
        try:
            existing = client.conversations.retrieve(stored_conversation_id)
            return existing.id
        except NotFoundError:
            # The stored ID no longer resolves to a real conversation —
            # fall through to creating a fresh one rather than crashing.
            pass

    return client.conversations.create().id

This defensive pattern — attempt to resume, fall back to creating fresh on failure — is worth building into any production code that resumes conversations from a stored ID, since a hard crash on a missing conversation is a poor user experience for what's ultimately a recoverable situation: the user simply starts a new conversation, rather than the application breaking outright.

Frequently Asked Questions

Can multiple different model calls, using different models, share the same conversation? Yes — the conversation parameter and the model parameter are independent. You could have one turn use gpt-5.6-luna and a later turn in the same conversation use gpt-6-astra, and the conversation's accumulated context carries forward regardless of which model handles any given turn. This is useful for a "escalate to a more capable model for a harder question" pattern within a single ongoing conversation.

Is there a limit to how many conversations I can create? Conversations are ordinary API resources subject to your account's standard rate and usage limits, but there's no meaningfully low cap on the number of conversations you can create for a typical application — a conversation per user, or even per chat thread per user, is a completely normal usage pattern.

Do I still need to worry about the context window with the Conversations API? Yes, unchanged from Lessons 2 and 3. The Conversations API solves durability and addressability — it does not solve the context window limit. A conversation that accumulates enough content will still eventually need the compaction strategies covered in Lesson 6, regardless of which of this unit's three techniques manages its state.

Can I use the Conversations API and previous_response_id together? Generally, you pick one mechanism per conversation rather than mixing them — once a responses.create() call is scoped to a conversation, that conversation object is the source of truth for continuity, and there's no meaningful reason to also chain via previous_response_id within it. Keep the two approaches conceptually separate in your code, and choose per-feature as discussed in Lesson 3's decision guide.

How is conversation content protected — can other users or applications read it? A conversation is scoped to your API account, accessible only with your API credentials, exactly like any other resource in your account (files, fine-tuned models, stored responses). It is not publicly readable, and access control within your own application — making sure user A's request can only retrieve user A's conversation IDs — is entirely your application's responsibility to enforce, the same way you'd protect access to any other row in your own database.

Side-by-Side: All Three Techniques From This Unit

Having now covered all three conversation-management approaches in depth, it's worth seeing them compared directly, since the right choice for a given feature comes down to weighing these tradeoffs against what that specific feature actually needs.

No single row of this table should be read as "always better" — a well-built application often uses more than one of these approaches across its different features, exactly as Lesson 3's decision guide described, choosing per-feature rather than standardizing on one technique for every conversational surface in a product.

Data Retention and Compliance Considerations

Because the Conversations API is explicitly designed for durability, it's worth thinking about data retention deliberately rather than by accident — durability that you didn't intend can become a liability, particularly for conversations that might contain sensitive information. If your application handles healthcare information, financial details, or other regulated data, review OpenAI's current data usage and retention policies for the Conversations API specifically (these are governed by your organization's account-level settings and any applicable data processing agreements) before relying on indefinite conversation persistence for sensitive content.

A practical pattern worth adopting for any application with real compliance obligations: treat client.conversations.delete() as a first-class part of your data lifecycle, not an afterthought. If your product has a data retention policy — "we delete inactive conversations after 90 days," for instance — that policy needs to actively call conversations.delete() on the OpenAI side, not just clean up your own database's references. A scheduled job that walks your own database for conversations past their retention window and deletes both your local record and the corresponding OpenAI conversation is the kind of infrastructure worth building early if you know compliance will eventually require it, rather than retrofitting it under deadline pressure later.

A Note on Choosing Conversation Granularity

One design decision worth being intentional about: what counts as "one conversation" in your application? For a simple single-purpose chatbot, this is obvious — one conversation per user, or one per chat session the user explicitly starts. But for a more complex product, this deserves real thought. Does a user get one single, ever-growing conversation for their entire relationship with your product, or a fresh conversation per topic, per support ticket, or per day?

There's a real tradeoff here, and it connects directly to Lesson 6's subject matter: a single ever-growing conversation eventually accumulates enough content to run into context-window and cost problems, requiring compaction strategies to manage. Multiple smaller, topic-scoped conversations avoid that problem naturally, at the cost of the model not having context from a different topic's conversation unless you explicitly bring it in (by injecting a summary as a conversation item, for instance, using the technique covered earlier in this lesson). Most production designs favor the second approach — reasonably scoped conversations, started fresh when a genuinely new topic or task begins — precisely because it keeps each individual conversation's context manageable, rather than accumulating an entire account's history into one conversation that eventually becomes unwieldy regardless of which state-management technique is managing it.

Common Mistakes

Treating the Conversations API as required for every feature. Not every conversational feature needs this level of durability. A quick, single-session widget (Lesson 3's article Q&A example) doesn't benefit from the extra API surface and object management the Conversations API introduces — reach for it specifically when cross-session or cross-device durability is a real requirement, not by default for every conversational feature.

Losing track of the conversation_id. The entire value of this API depends on you reliably storing and retrieving the conversation ID — typically as a column in your own database, tied to a user or a specific chat thread. If that ID is lost (never saved, or saved somewhere that gets cleared), the conversation becomes unreachable even though it may still technically exist on OpenAI's servers.

Assuming conversation items are automatically visible in your own application's UI without fetching them. If you want to display conversation history in your own interface, you need to actually call client.conversations.items.list() (or maintain your own parallel copy, as in the hybrid approach from Lesson 3) — the conversation existing on OpenAI's servers doesn't automatically populate anything in your own frontend.

Forgetting that deleting your own database record doesn't delete the underlying conversation. If your application's data-deletion flow only removes your own reference to a conversation_id without calling client.conversations.delete(), the actual conversation content still exists on OpenAI's servers. For any feature with real deletion requirements, both sides need to be handled.

Troubleshooting

client.conversations.retrieve() fails with a not-found error. Confirm the conversation ID is being read and passed correctly — a common cause is a typo or truncation when storing the ID in your database, or accidentally storing a response ID (resp_...) instead of a conversation ID (conv_...), which are different identifier types entirely and not interchangeable.

The model doesn't seem to see context that was added via conversations.items.create(). Double check the item was added with an appropriate role (developer for background context that should always be considered, user if it should read as something the user said) and that it was added to the conversation before the responses.create() call you're troubleshooting, since items are only visible to calls made after they're added.

A very old, resumed conversation behaves unexpectedly or errors on context length. Just like a long response chain or a long manually managed history, a conversation with a large accumulated item count is still bound by the model's context window. Lesson 6's compaction strategies apply here too — a conversation resumed after months of accumulated content may need trimming or summarization before continuing productively.

Two different features seem to be sharing the same conversation. Verify each logical thread — each distinct chat a user might have — gets its own conversation.id, created once and stored per-thread, rather than accidentally reusing one conversation ID across what should be separate, unrelated conversations.

Best Practices

Reach for the Conversations API specifically when a feature genuinely needs cross-session or cross-device durability — a personal assistant, an ongoing support thread a user might return to, anything where "pick up where I left off, from anywhere" is a real requirement. Store the conversation_id as durably as any other important piece of application data — a proper database column, not an in-memory variable or a short-lived cookie. Build explicit conversation deletion into any data-deletion flow your application already has, rather than assuming removing your own reference is sufficient. And don't default to this API for every conversational feature — Lesson 3's previous_response_id remains the better, simpler fit for short-lived interactions that don't need this level of durability, and Lesson 2's fully manual approach remains the right choice when you need complete visibility and editability of the underlying data yourself.

What's Next

You now have three complementary techniques for managing conversation state — full manual control (Lesson 2), lightweight server-side chaining (Lesson 3), and durable, addressable conversation objects (this lesson) — and a clear sense of when each one fits. Lesson 5 puts this knowledge to direct use in a hands-on project: building a command-line chatbot that remembers, using the techniques from this unit to give it real, working conversational memory. Lesson 6 then tackles the problem every one of these approaches eventually runs into as a conversation grows: what happens when the accumulated context exceeds what the model can actually accept, and how you manage that limit gracefully with token counting and compaction strategies, rather than discovering it as a production incident.