Add Chat Memory

History vs. Memory

Chat history (Lesson 62) is the record of one conversation. Chat memory is information that survives across conversations: what the chatbot "knows" about a user when a new session starts.

Without memory, every new session starts from zero: users must repeat their name, goals, and skill level. With memory, a new session can begin with useful context.

Types of Memory

This lesson implements long-term facts, the most common kind of memory in chat applications. Anthropic's documentation also describes a memory tool that lets Claude read and write memory files during agentic tasks; that is useful for agents, and builds on the tool-use lessons (81–100).

How Fact Memory Works

After each successful turn:
  user message ──► small model extracts durable facts ──► store new facts for the user

When a session starts or a message is sent:
  stored facts ──► <user_memory> section in the system prompt ──► Claude uses them when relevant

Step 1: Extract Facts

A fast, inexpensive model reads the user's message and returns only durable facts: name, role, goals, skill level, preferences, and ongoing projects. Structured output (Lesson 36) returns them as a clean list.

Step 2: Store Facts

Facts are stored per user, with duplicates ignored and a cap on how many are kept, so memory stays small.

Step 3: Inject Facts

When building a request, the stored facts are added to the end of the system prompt, in tags:

You are a friendly programming tutor...

<user_memory>
- The user's name is Meena.
- The user is learning FastAPI for an internship project.
</user_memory>
These are facts the user shared in earlier conversations. Use them when relevant.
Do not mention this list unless the user asks what you remember.

Placing memory at the end keeps the stable instruction part identical across users and sessions, which suits prompt caching (Lesson 32).

Memory and Privacy

Memory stores personal information, so it needs deliberate rules:

Warning: The extraction instructions reduce the chance of storing sensitive data, but they are not a guarantee. For applications with strict requirements, add your own filtering and review, and follow the data-protection rules that apply to your users.

Practical Example: A Memory Store

memory.py

"""memory.py: long-term user memory (facts) shared across chat sessions."""

import time

import anthropic
from anthropic import Anthropic
from pydantic import BaseModel

from history_store import HistoryStore

MEMORY_MODEL = "claude-haiku-5-5"
MAX_FACTS = 30

MEMORY_SCHEMA = """
CREATE TABLE IF NOT EXISTS user_memory (
    id         INTEGER PRIMARY KEY AUTOINCREMENT,
    user_id    TEXT NOT NULL,
    fact       TEXT NOT NULL,
    created_at REAL NOT NULL,
    UNIQUE (user_id, fact)
);
"""

EXTRACT_SYSTEM = """You extract long-term facts that a user states about themselves in a chat message.
Keep only durable, useful facts: name, role, goals, skill level, preferences, and current projects.
Never include sensitive information (health, finances, passwords, government IDs, exact addresses)
or temporary details (today's mood, a one-off question).
Write each fact as a short third-person sentence, for example "The user is learning Django."
Return an empty list if there are no new facts. Do not repeat facts listed as already known."""


class ExtractedFacts(BaseModel):
    facts: list[str]


class MemoryStore:
    def __init__(self, store: HistoryStore):
        self.db = store.db
        self.db.executescript(MEMORY_SCHEMA)

    def facts(self, user_id: str) -> list[tuple[int, str]]:
        return self.db.execute(
            "SELECT id, fact FROM user_memory WHERE user_id = ? ORDER BY id", (user_id,)
        ).fetchall()

    def add(self, user_id: str, facts: list[str]) -> None:
        if not facts:
            return
        now = time.time()
        with self.db:
            self.db.executemany(
                "INSERT OR IGNORE INTO user_memory (user_id, fact, created_at) VALUES (?, ?, ?)",
                [(user_id, fact, now) for fact in facts],
            )
            self.db.execute(
                "DELETE FROM user_memory WHERE user_id = ? AND id NOT IN "
                "(SELECT id FROM user_memory WHERE user_id = ? ORDER BY id DESC LIMIT ?)",
                (user_id, user_id, MAX_FACTS),
            )

    def forget(self, user_id: str, fact_id: int) -> bool:
        with self.db:
            cursor = self.db.execute(
                "DELETE FROM user_memory WHERE id = ? AND user_id = ?", (fact_id, user_id)
            )
        return cursor.rowcount == 1

    def forget_all(self, user_id: str) -> int:
        with self.db:
            cursor = self.db.execute("DELETE FROM user_memory WHERE user_id = ?", (user_id,))
        return cursor.rowcount

    def system_section(self, user_id: str) -> str:
        facts = [fact for _, fact in self.facts(user_id)]
        if not facts:
            return ""
        lines = "\n".join(f"- {fact}" for fact in facts)
        return (
            "<user_memory>\n" + lines + "\n</user_memory>\n"
            "These are facts the user shared in earlier conversations. Use them when relevant. "
            "Do not mention this list unless the user asks what you remember."
        )


def extract_facts(client: Anthropic, user_text: str, known: list[str]) -> list[str]:
    known_block = "\n".join(f"- {fact}" for fact in known) or "(none)"
    content = (
        f"<known_facts>\n{known_block}\n</known_facts>\n\n"
        f"<message>\n{user_text[:4000]}\n</message>"
    )
    try:
        response = client.messages.parse(
            model=MEMORY_MODEL,
            max_tokens=1024,
            system=EXTRACT_SYSTEM,
            output_format=ExtractedFacts,
            messages=[{"role": "user", "content": content}],
        )
    except anthropic.APIError:
        return []
    if response.stop_reason != "end_turn" or response.parsed_output is None:
        return []
    return [fact.strip() for fact in response.parsed_output.facts if fact.strip()][:5]

Using Memory Across Sessions

"""demo_memory.py: facts from one session personalize the next one."""

from dataclasses import replace

from anthropic import Anthropic

from chatbot_core import BotConfig
from history_store import HistoryStore
from memory import MemoryStore, extract_facts
from sessions import SessionManager

client = Anthropic()
store = HistoryStore("chat.db")
memory = MemoryStore(store)
BASE = BotConfig(system="You are a friendly programming tutor. Keep answers under 100 words.")
sessions = SessionManager(store, client, BASE)
USER = "meena"


def chat(session_id: str, text: str) -> str:
    bot = sessions.open(session_id, USER)
    section = memory.system_section(USER)
    bot.config = replace(BASE, system=f"{BASE.system}\n\n{section}" if section else BASE.system)
    reply = bot.send(text)
    if reply.ok:
        sessions.save_turn(session_id, USER, reply)
        known = [fact for _, fact in memory.facts(USER)]
        memory.add(USER, extract_facts(client, text, known))
    return reply.text


first = sessions.create(USER)
print(chat(first, "Hi! I'm Meena, a final-year student learning FastAPI for my internship project."))

print("\nStored memory:")
for fact_id, fact in memory.facts(USER):
    print(f"  [{fact_id}] {fact}")

second = sessions.create(USER)
print("\nNew session:", chat(second, "Can you suggest a small project to practice with?"))

Example output:

Hi Meena! FastAPI is a great choice for an internship project. What are you building?

Stored memory:
  [1] The user's name is Meena.
  [2] The user is a final-year student.
  [3] The user is learning FastAPI for an internship project.

New session: Sure, Meena! Since you're learning FastAPI, try building a small task-tracker API with CRUD endpoints, request validation, and a SQLite database. It's a good warm-up for your internship project.

How It Works

1. MEMORY_SCHEMA adds a user_memory table to the same database. UNIQUE (user_id, fact) together with INSERT OR IGNORE prevents storing the same fact twice.
2. extract_facts() sends the new message, plus the facts already known, to a fast model with messages.parse() and the ExtractedFacts model. Passing known facts lets the model skip repeats. Failures, refusals, and truncated responses all return an empty list, so memory problems never affect the chat itself.
3. add() inserts new facts, then keeps only the newest MAX_FACTS per user. The subquery selects the IDs to keep, and everything else for that user is deleted.
4. system_section() formats the facts as a tagged list with instructions: use them when relevant, and do not recite them unprompted.
5. chat() rebuilds the bot's configuration with dataclasses.replace(), which creates a copy of BASE with a new system prompt, so the base configuration is never modified. Memory is added at the end of the system prompt.
6. Extraction runs after each successful turn, in the demo directly after saving. In a web application, run it in a background task, so users do not wait for it.
7. The second session has no message history, but the system prompt contains the remembered facts, so Claude addresses Meena by name and tailors the suggestion to FastAPI and her internship.

Common Mistakes

Storing Entire Conversations as "Memory"

What happens: memory grows huge, costs rise, and relevant facts are buried.

Fix: store a small set of condensed, durable facts.

Storing Sensitive or Temporary Information

What happens: personal data is retained unnecessarily, and irrelevant details clutter memory.

Fix: instruct the extractor precisely, filter results, and let users delete facts.

Hidden Memory

What happens: users are surprised that the bot knows things about them, which undermines trust.

Fix: be transparent, and give users a way to view and clear memory.

Letting Memory Failures Break the Chat

What happens: an extraction error turns into an error message for the user's question.

Fix: treat memory as optional, and catch its errors separately.

Putting Memory at the Start of the System Prompt

What happens: each user's prompt begins differently, which reduces prompt-caching benefits.

Fix: append memory at the end of the system prompt.

Best Practices

• Keep memory small, durable, and useful.
• Extract facts with structured output and a fast model.
• Inject memory at the end of the system prompt, in clearly labeled tags.
• Exclude sensitive data, and give users visibility and control.
• Run extraction outside the user's critical path when possible.
• Cap and expire stored facts.

Exercise

1. Send three messages, one with a durable fact, one with a question only, and one with a temporary detail. Which facts are stored?
2. Add a /memory view and a /forget <id> command to the CLI chat from Lesson 58, using MemoryStore.
3. Add a rule to EXTRACT_SYSTEM that also captures the user's preferred explanation style, such as "prefers short answers with code examples".
4. Move fact extraction into a background thread, so the reply is returned before extraction finishes.

import threading


def remember_in_background(user_id: str, text: str) -> None:
    def work():
        known = [fact for _, fact in memory.facts(user_id)]
        memory.add(user_id, extract_facts(client, text, known))

    threading.Thread(target=work, daemon=True).start()

Call remember_in_background(USER, text) instead of extracting inline. Note that the standard sqlite3 connection object cannot be shared across threads by default. In the worker thread, open a separate HistoryStore and MemoryStore, or create the main connection with sqlite3.connect(path, check_same_thread=False) and protect it with a lock. In web frameworks, use the framework's background task feature (for example, FastAPI's BackgroundTasks) and a database connection per task.