Memory: Conversations That Remember

Here’s a perfectly normal support conversation:

Customer: Can I get a refund for order #1043? Nimbus: Sorry, order #1043 was purchased 20 days ago, and refunds are only possible within 14 days. Customer: OK. What about my other order?

Every human support agent knows what "my other order" means: the customer is still talking about refunds, and probably has a second, more recent purchase. Our Nimbus from Part 5 has no idea. Each call to ask-agent.php starts from nothing. It doesn’t know a refund was discussed, or that order #1043 exists. It can only ask: "Which order do you mean?"

In this part we give Nimbus memory: we store every conversation in MySQL, so follow-up questions work. We’ll also deal with the cost of memory, because conversations grow, and a naive approach makes every question slower and more expensive than the last.

Before you start

• What you’ll build: conversation memory in MySQL, history trimming, an interactive terminal chat and a conversation viewer.
• You need: the setup from Part 5 (all tools, the help-center index and the agent loop).
• Code for this part: GitHub → part-06

🔀 Gemini or OpenAI? The code for this series comes in two versions: Gemini Setup (free tier, recommended for learning) and OpenAI Setup (paid). The agent logic is the same in both. Only the small layer that talks to the AI provider differs, because each provider formats tool calls slightly differently, so the code may vary a bit between the two. Model names and free-tier limits change often: check .env.example for the current defaults.

How AI "memory" actually works

The most important fact about LLM APIs: they are stateless. The model remembers nothing between requests. Every time you call Gemini or OpenAI, it sees only what you send in that request.

You’ve already relied on this. In Part 4, the agent loop "remembered" earlier tool results within one question because we resent the growing $messages array on every step. Memory across questions works the same way: we store the conversation ourselves, and send it again with each new question.

A few terms:

• Conversation: all the messages exchanged with one customer in one chat, under one ID.
• Turn: one customer message plus everything that followed it: tool calls, tool results and Nimbus’s answer.
• Token: the unit models read and charge for. Roughly, one token is about four characters of English text.
• Context window: the maximum number of tokens a model can read in one request. Modern models have large windows, but you pay for every token you send, and long requests are slower.

That last point is why memory needs a strategy. If we resend the full history every time, a 20-turn conversation sends 20 turns of tool calls and results with every new question, and with the agent loop, several times per question. Costs grow with every message.

Our strategy has three layers:

1. Keep recent turns in full. The last two turns are sent exactly as they happened, including tool calls and results, because "the other order" usually refers to something just discussed.
2. Compact older turns. For older turns we keep only the customer’s question and Nimbus’s final answer. The answer already contains the facts the tools found.
3. Drop the oldest turns above a budget. If the history is still larger than a fixed size, the oldest turns are forgotten.

Build it

part-06 starts as a copy of Part 5. The changes: two new tables, a new memory.php, an updated agent.php, and new scripts to chat and to inspect conversations.

Step 1: Tables for conversations and messages

-- part-06/schema.sql (excerpt: the new tables)
-- New in Part 6: conversation memory.
-- One row per conversation; the id is random so nobody can guess another one.
CREATE TABLE conversations (
    id         CHAR(32) PRIMARY KEY,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    updated_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ON UPDATE CURRENT_TIMESTAMP
) CHARACTER SET utf8mb4;

-- One row per message, in our neutral format (see llm.php), stored as JSON.
CREATE TABLE messages (
    id              BIGINT AUTO_INCREMENT PRIMARY KEY,
    conversation_id CHAR(32) NOT NULL,
    role            VARCHAR(20) NOT NULL,        -- user, assistant or tool
    data            JSON NOT NULL,               -- the whole message
    created_at      TIMESTAMP DEFAULT CURRENT_TIMESTAMP,
    INDEX idx_conversation (conversation_id, id),
    FOREIGN KEY (conversation_id) REFERENCES conversations(id) ON DELETE CASCADE
) CHARACTER SET utf8mb4;

Two design choices:

• The conversation ID is random, 32 hex characters from random_bytes(), not an auto-increment number. If IDs were 1, 2, 3…, anyone could guess another customer’s conversation.
• Each message is stored as JSON, in exactly the neutral format our code already uses. That includes provider_raw, the provider’s original message, which keeps Gemini’s thought signatures intact when the history is resent.

Re-run the full setup, since the schema resets all tables:

cd part-06
mysql -u root -p < schema.sql
mysql -u root -p < functions.sql
mysql -u root -p < seed.sql
php ingest.php
php build-index.php

⚠️ Gotcha: Because we store each provider’s own message format, a conversation started with Gemini can’t be continued with OpenAI, and vice versa. In practice you use one provider per deployment, so this rarely matters, but don’t switch .env mid-conversation while testing.

Step 2: Save, load and trim

memory.php holds everything memory-related:

<?php
// memory.php — conversation memory in MySQL.
// Saves every message of a conversation, and loads a TRIMMED version of the
// history for the next question, so long chats don't get slow or expensive.

const MEMORY_MAX_CHARS    = 12000;  // history budget per request (~3,000 tokens)
const MEMORY_RECENT_TURNS = 2;      // newest turns kept in full (with tool calls)

function newConversation(): string {
    $id = bin2hex(random_bytes(16));                    // 32 random hex characters
    db()->prepare('INSERT INTO conversations (id) VALUES (?)')->execute([$id]);
    return $id;
}

function conversationExists(string $id): bool {
    $stmt = db()->prepare('SELECT 1 FROM conversations WHERE id = ?');
    $stmt->execute([$id]);
    return (bool) $stmt->fetchColumn();
}

function saveMessages(string $conversationId, array $messages): void {
    $stmt = db()->prepare(
        'INSERT INTO messages (conversation_id, role, data) VALUES (?, ?, ?)'
    );
    foreach ($messages as $m) {
        $stmt->execute([$conversationId, $m['role'], json_encode($m, JSON_UNESCAPED_UNICODE)]);
    }
    db()->prepare('UPDATE conversations SET updated_at = CURRENT_TIMESTAMP WHERE id = ?')
        ->execute([$conversationId]);
}

function loadMessages(string $conversationId): array {
    $stmt = db()->prepare('SELECT data FROM messages WHERE conversation_id = ? ORDER BY id');
    $stmt->execute([$conversationId]);
    return array_map(fn($json) => json_decode($json, true), $stmt->fetchAll(PDO::FETCH_COLUMN));
}

// A "turn" = one customer message + everything that followed it
// (tool calls, tool results, the answer). We always keep or drop whole turns,
// because a tool call without its result breaks both providers' APIs.
function splitIntoTurns(array $messages): array {
    $turns = [];
    foreach ($messages as $m) {
        if ($m['role'] === 'user' || !$turns) {
            $turns[] = [];
        }
        $turns[count($turns) - 1][] = $m;
    }
    return $turns;
}

// Shrink an older turn to just the question and the final answer.
// The answer already contains the facts the tools found.
function compactTurn(array $turn): array {
    $compact = [$turn[0]];
    foreach (array_reverse($turn) as $m) {
        if ($m['role'] === 'assistant' && trim((string) ($m['content'] ?? '')) !== '') {
            $compact[] = ['role' => 'assistant', 'content' => $m['content']];
            break;
        }
    }
    return $compact;
}

// The history we actually send: recent turns in full, older turns compacted,
// and the oldest turns dropped if we're still over budget.
function buildContext(array $messages): array {
    $turns = splitIntoTurns($messages);
    $keepFullFrom = count($turns) - MEMORY_RECENT_TURNS;
    foreach ($turns as $i => $turn) {
        if ($i < $keepFullFrom) {
            $turns[$i] = compactTurn($turn);
        }
    }
    while (count($turns) > 1 && strlen(json_encode($turns)) > MEMORY_MAX_CHARS) {
        array_shift($turns);                            // forget the oldest turn
    }
    return $turns ? array_merge(...$turns) : [];
}

The most important rule is in splitIntoTurns(): we always keep or drop whole turns. Both providers reject a history where a tool call appears without its result, or a result without its call. Cutting the history at an arbitrary message would break the next request.

compactTurn() keeps the customer’s question and the last assistant message that has text, which is Nimbus’s answer. It drops the tool calls and results in between. If the answer said "Order #1043 isn’t eligible: purchased 20 days ago", the model still knows that later, without the raw JSON.

buildContext() applies the strategy: recent turns in full (MEMORY_RECENT_TURNS = 2), older turns compacted, and the oldest turns dropped while the history exceeds MEMORY_MAX_CHARS (12,000 characters, roughly 3,000 tokens). Measuring characters isn’t as precise as counting tokens, but it needs no extra library and works the same for both providers.

Step 3: The agent, now per conversation

The loop works as in Part 4. What changes is where $messages comes from and what happens at the end:

// part-06/agent.php (excerpt: runAgent and finishTurn)
function runAgent(string $question, string $conversationId, int $maxSteps = MAX_STEPS): string {
    traceStart($question, $conversationId);
    $tools = toolDefinitions();

    // Memory: what was said before in this conversation (trimmed to a budget).
    $history = buildContext(loadMessages($conversationId));
    traceEvent(0, 'memory', ['messages' => count($history), 'chars' => strlen(json_encode($history))],
        'memory: ' . count($history) . ' earlier messages loaded (' . strlen(json_encode($history)) . ' chars)');

    // Everything that happens in THIS turn. Only this part gets saved.
    $turn = [['role' => 'user', 'content' => $question]];
    $callCounts = [];

    for ($step = 1; $step <= $maxSteps; $step++) {
        // The system prompt is never stored: it's always added fresh, so
        // prompt changes apply to old conversations too.
        $messages = array_merge([['role' => 'system', 'content' => SYSTEM_PROMPT]], $history, $turn);

        // 1. Ask the model what to do next.
        $started = microtime(true);
        try {
            $reply = chatWithTools($messages, $tools);
        } catch (RuntimeException $e) {
            traceEvent($step, 'api_error', ['error' => $e->getMessage()], 'API error: ' . $e->getMessage());
            return finishTurn($conversationId, $turn,
                "Sorry, I'm having trouble right now. Please try again in a moment.", 'api_error');
        }
        $ms = (int) round((microtime(true) - $started) * 1000);

        // 2. No tool calls → the model has given its final answer. Done.
        if (empty($reply['tool_calls'])) {
            traceEvent($step, 'final_answer', ['ms' => $ms], "model answered ({$ms} ms)");
            if (!$reply['text']) {
                return finishTurn($conversationId, $turn,
                    "Sorry, I couldn't come up with an answer. Could you rephrase?", 'answered');
            }
            $turn[] = $reply['message'];                // keep the provider's own message
            return finishTurn($conversationId, $turn, $reply['text'], 'answered', false);
        }

        // 3. Otherwise run every tool it asked for, and add the results.
        $turn[] = $reply['message'];

        foreach ($reply['tool_calls'] as $call) {
            $argsJson = json_encode($call['arguments']);
            traceEvent($step, 'tool_call', ['tool' => $call['name'], 'arguments' => $call['arguments'], 'ms' => $ms],
                "model → {$call['name']}($argsJson) ({$ms} ms)");

            $key = $call['name'] . $argsJson;
            $callCounts[$key] = ($callCounts[$key] ?? 0) + 1;

            if ($callCounts[$key] > MAX_REPEAT_CALLS) {
                $result = ['error' => 'You already made this exact call. Use the earlier result.'];
            } else {
                $result = safeRunTool($step, $call['name'], $call['arguments']);
            }
            traceEvent($step, 'tool_result', ['tool' => $call['name'], 'result' => $result],
                "PHP  ← " . json_encode($result));

            $turn[] = [
                'role'         => 'tool',
                'tool_call_id' => $call['id'],
                'name'         => $call['name'],
                'content'      => json_encode($result),
            ];
        }
    }

    // 4. Safety limit reached.
    traceEvent($maxSteps, 'step_limit', [], "stopped: reached the step limit ($maxSteps)");
    return finishTurn($conversationId, $turn,
        "Sorry, this is taking more steps than I expected. "
      . "Please contact our support team and they'll sort it out.", 'step_limit');
}

// Save this turn to memory, close the trace, and return the answer.
// If the answer didn't come from the model (errors, limits), we store it as a
// plain assistant message so the history still reads question → answer.
function finishTurn(string $conversationId, array $turn, string $answer,
                    string $outcome, bool $addAnswer = true): string {
    if ($addAnswer) {
        $turn[] = ['role' => 'assistant', 'content' => $answer];
    }
    saveMessages($conversationId, $turn);
    traceEnd($answer, $outcome);
    return $answer;
}

Read the comments in order:

• Load: the trimmed history for this conversation is loaded once at the start.
• $turn: everything that happens during this question is collected separately, so only the new messages are saved at the end.
• The system prompt is never stored. It’s added fresh on every call. If you improve the prompt next week, old conversations immediately use the new version.
• finishTurn() saves the turn, whatever the outcome. Even when we hit an API error or the step limit, the fallback answer is saved as an assistant message, so the stored history always reads question → answer.

The system prompt also gets one new instruction about follow-ups:

Requests about a specific order: use the order tools. If the customer hasn't given
an order number and you need one, check the earlier conversation first, then ask.
Never invent one. If they refer to "it" or "the other order", work out which order
they mean from the conversation; if it's unclear, ask.

trace.php now records the conversation ID and a memory event showing how much history was loaded. We also moved the growing list of require_once lines into bootstrap.php, which every entry script loads.

Step 4: Chat with Nimbus

chat.php is a small interactive loop in the terminal:

<?php
// chat.php — a real back-and-forth chat with Nimbus in your terminal.
// Usage: php chat.php                       (start a new conversation)
//        php chat.php --resume=<id>         (pick up an old one)
// Type "exit" to quit.
require_once __DIR__ . '/bootstrap.php';

$resume = null;
foreach (array_slice($argv, 1) as $arg) {
    if (str_starts_with($arg, '--resume=')) {
        $resume = substr($arg, 9);
    }
}

if ($resume !== null && conversationExists($resume)) {
    $conversationId = $resume;
    echo "Resuming conversation $conversationId\n";
} else {
    $conversationId = newConversation();
    echo "New conversation $conversationId\n";
}
echo "Ask Nimbus anything (type \"exit\" to quit).\n\n";

while (true) {
    echo "You: ";
    $line = fgets(STDIN);
    if ($line === false || in_array(strtolower(trim($line)), ['exit', 'quit'], true)) {
        break;
    }
    if (trim($line) === '') {
        continue;
    }
    echo "\nNimbus: " . runAgent(trim($line), $conversationId) . "\n\n";
}
echo "Bye! Resume later with: php chat.php --resume=$conversationId\n";

ask-agent.php still works one question at a time, now with --conversation=<id> to continue a conversation. It prints the ID after each answer. A third script, show-conversation.php, prints a stored conversation, or with --context, exactly what the model will see with the next question. That’s the quickest way to check that compaction is doing what you expect.

Prompt spotlight: memory changes what "never guess" means

From Part 1 onwards, our prompt has said: "Never invent an order number." With memory, there’s a new grey area. If the customer says "the other order", is using an order number from earlier in the conversation guessing?

No, as long as it’s clearly established. The updated instruction reflects that:

• "Check the earlier conversation first, then ask." Information the customer already gave counts. Asking again for an order number they mentioned two messages ago is annoying.
• "Work out which order they mean from the conversation; if it’s unclear, ask." If two orders were discussed and "it" could mean either, the model should ask, not pick one.

Memory makes an agent feel smarter, but it also gives it more chances to make an assumption that’s wrong. The prompt has to say where inferring ends and guessing begins.

Try it

php chat.php

New conversation 9f3c2a71d04e4b8c8a1e5f6b7c2d9e10
Ask Nimbus anything (type "exit" to quit).

You: Can I get a refund for order #1043?
  [step 0] memory: 0 earlier messages loaded (2 chars)
  [step 1] model → check_refund_eligibility({"order_id":1043}) (790 ms)
  [step 1] PHP  ← {...,"days_since_purchase":20,"eligible":false,...}
  [step 2] model answered (640 ms)

Nimbus: Sorry, order #1043 was purchased 20 days ago, and refunds are only
possible within 14 days.

You: OK. What about my other order, 1042?
  [step 0] memory: 4 earlier messages loaded (1184 chars)
  [step 1] model → check_refund_eligibility({"order_id":1042}) (820 ms)
  [step 1] PHP  ← {...,"eligible":true,"reason":"Paid and within the refund window (11 days left)."}
  [step 2] model answered (700 ms)

Nimbus: Good news: order #1042 is eligible for a refund. Would you like me to
open a refund request for it?

You: Yes please.
  [step 0] memory: 8 earlier messages loaded (2391 chars)
  [step 1] model → create_support_ticket({"category":"refund","summary":"Customer requests a refund for order 1042 (eligible, 3 days old).","order_id":1042}) (950 ms)
  [step 1] PHP  ← {"created":true,"ticket_id":5001,"status":"open"}
  [step 2] model answered (680 ms)

Nimbus: Done! I've opened ticket #5001 for your refund on order #1042.

You: exit
Bye! Resume later with: php chat.php --resume=9f3c2a71d04e4b8c8a1e5f6b7c2d9e10

"Yes please" contains no order number and no mention of refunds, yet Nimbus opened the right ticket for the right order. All of that came from memory.

Now look at what the model would see with the next question:

php show-conversation.php 9f3c2a71d04e4b8c8a1e5f6b7c2d9e10 --context

Context sent with the next question (2214 chars):
user:     Can I get a refund for order #1043?
assistant: Sorry, order #1043 was purchased 20 days ago, and refunds are only possible within 14 days.
user:     OK. What about my other order, 1042?
assistant: (requested tools)
tool:     check_refund_eligibility → {"order_id":1042,...,"eligible":true,...}
assistant: Good news: order #1042 is eligible for a refund. Would you like me to open a refund…
user:     Yes please.
assistant: (requested tools)
tool:     create_support_ticket → {"created":true,"ticket_id":5001,"status":"open"}
assistant: Done! I've opened ticket #5001 for your refund on order #1042.

The first turn has already been compacted to question and answer; the last two are still in full detail.

💡 Tip (Gemini free tier): memory doesn’t add requests, but it makes each one larger. If you’re testing long conversations, lower MEMORY_MAX_CHARS to see trimming kick in sooner.

Privacy: memory is customer data

Stored conversations contain whatever customers typed: names, order details, sometimes personal information. Treat the messages table like any other customer data:

• Decide how long to keep it. For example, delete conversations after 30 days. The foreign key’s ON DELETE CASCADE removes their messages too:

  DELETE FROM conversations WHERE updated_at < NOW() - INTERVAL 30 DAY;
  

• Never use conversation IDs as the only security. In a real web app, link each conversation to a logged-in customer and check ownership before loading it.
• Remember the trace logs from Part 4 contain the same data. Apply the same retention rule to logs/.

Recap

• LLM APIs are stateless. Memory means storing the conversation yourself and resending it with each question.
• Store conversations in MySQL with random IDs and each message as JSON in the neutral format, including provider_raw.
• Keep the cost under control: recent turns in full, older turns compacted, oldest turns dropped above a budget.
• Always keep or drop whole turns, never a tool call without its result.
• Never store the system prompt; add it fresh so improvements apply to old conversations.
• Memory is customer data: set a retention period and tie conversations to customers in real apps.

Next, Part 7: Guardrails and Human Approval. Nimbus can now read, write and remember. Before it goes near real customers, we need limits: which tools need permission, which actions need a human to confirm, and how to stop text inside documents or tool results from hijacking the agent.

Frequently Asked Questions

Full code for this part: GitHub → Gemini Setup/part-06 · OpenAI Setup/part-06