The conversation that comes back

Part one closed with a warning: restart the node and the conversation evaporates. In production that is not a hypothesis — it is Tuesday afternoon's deploy wiping every conversation open at that moment. If the product promises that the user comes back tomorrow and finds their history, the process alone cannot keep that promise.

The answer is not to move the conversation out of the process. It is to split the roles, the way the end of part one hinted: the database becomes the record of what closed; the process stays where the conversation happens. This part builds both ends — writing each round down, and rebuilding the chain when the user returns.

A two-column transcript

What needs to survive is less than it seems. Not the chain struct, not the callbacks, not the model: who spoke, and what they said.

defmodule MyApp.Repo.Migrations.CreateTranscriptEntries do
  use Ecto.Migration

  def change do
    create table(:transcript_entries) do
      add :conversation_id, :string, null: false
      add :role, :string, null: false
      add :content, :text, null: false
      timestamps(updated_at: false)
    end

    create index(:transcript_entries, [:conversation_id])
  end
end

defmodule MyApp.Transcript do
  import Ecto.Query
  alias MyApp.Repo

  defmodule Entry do
    use Ecto.Schema

    schema "transcript_entries" do
      field :conversation_id, :string
      field :role, Ecto.Enum, values: [:user, :assistant]
      field :content, :string
      timestamps(updated_at: false)
    end
  end

  def record(conversation_id, role, content) do
    Repo.insert!(%Entry{conversation_id: conversation_id, role: role, content: content})
  end

  def entries(conversation_id) do
    Repo.all(from e in Entry, where: e.conversation_id == ^conversation_id, order_by: e.id)
  end
end

Note that there is no conversations table. The conversation_id is the same id the Registry from part one already uses as a key, and giving the same conversation a second identity would be bureaucracy. And the order_by goes by id, not inserted_at: the two writes of a round can land on the same microsecond, and the sequential id never ties.

Write when the round closes

The write lives in part one's handle_call, on the branch where the round closed — with alias LangChain.Message.ContentPart next to the aliases the module already had:

@impl true
def handle_call({:ask, text}, _from, state) do
  case state.chain |> LLMChain.add_message(Message.new_user!(text)) |> LLMChain.run() do
    {:ok, chain} ->
      MyApp.Transcript.record(state.id, :user, text)

      MyApp.Transcript.record(
        state.id,
        :assistant,
        ContentPart.content_to_string(chain.last_message.content)
      )

      {:reply, {:ok, chain.last_message}, %{state | chain: chain}, @idle_timeout}

    {:error, _chain, error} ->
      {:reply, {:error, error}, state, @idle_timeout}
  end
end

The state gained its own id — the init/1 in the next section holds %{id: id, chain: chain} — because until now the process never needed to know its own name; the Registry did.

The user's question only enters the record together with the answer. It is the rule part one's error branch established for the chain — state stays at the last point where a round closed in full — now extended to the database. The alternative, writing the question on arrival and flagging the round as open, buys you a state machine: everything that reads the transcript now has to decide what to do with an unanswered question. Two deferred writes cost less than that.

content_to_string/1 settles the other detail: the model's answer carries its text in a list of ContentPart, the shape that showed up in part three's deltas and part four's asserts, and this library function flattens the list back into a string.

Rebuilding is rereading

@impl true
def init(opts) do
  id = Keyword.fetch!(opts, :id)

  llm =
    Keyword.get_lazy(opts, :llm, fn ->
      ChatAnthropic.new!(%{model: "claude-sonnet-5"})
    end)

  chain =
    %{llm: llm}
    |> LLMChain.new!()
    |> LLMChain.add_message(Message.new_system!(Keyword.fetch!(opts, :system_prompt)))
    |> replay(MyApp.Transcript.entries(id))

  {:ok, %{id: id, chain: chain}, @idle_timeout}
end

defp replay(chain, entries) do
  Enum.reduce(entries, chain, fn entry, chain ->
    message =
      case entry.role do
        :user -> Message.new_user!(entry.content)
        :assistant -> Message.new_assistant!(entry.content)
      end

    LLMChain.add_message(chain, message)
  end)
end

The system prompt did not come from the database, and that is a choice: it keeps arriving through opts, it evolves with the code, and a conversation rebuilt tomorrow gets tomorrow's version, not the fossil. That is why the transcript's role only knows :user and :assistant.

To the chain, there is no difference between a new message and a reread one — the model receives the whole list and answers as if the conversation had never stopped. If the read ever gets heavy, handle_continue takes the query off the path of whoever called start_link; with a few dozen messages per conversation, the reduce above shows up on no graph.

What the record loses

The transcript stores text, and a conversation with tools exchanges more than text. When part two handed lookup_order to the model, its request and the function's result became messages in the chain — and none of that goes to the database. On rebuild, the model sees the answer that mentioned order A-4471, but no longer sees the raw result that produced it.

For a support conversation, that cut usually suffices: whatever mattered in the result is embedded in the answer. When it does not, the way out is storing the whole structs — the chain notes everything the last round exchanged in exchanged_messages, tool calls included — and the serializer becomes your job, because the library does not ship one. More fidelity for more code; postpone it until you need it.

A deploy the test rehearses

Part four left the fake ready, and it proves the whole rebuild without touching the network:

defmodule MyApp.ConversationRestartTest do
  use MyApp.DataCase, async: false

  alias LangChain.Message
  alias LangChain.Message.ContentPart
  alias MyApp.{Conversation, FakeModel}

  test "a rebuilt conversation picks up where it left off" do
    fake =
      FakeModel.new!(%{
        reply: fn messages, _tools ->
          {:ok, Message.new_assistant!("got #{length(messages)} messages")}
        end
      })

    start_supervised!({Conversation, id: "t-3", system_prompt: "keep it short", llm: fake})
    {:ok, _first} = Conversation.ask("t-3", "hi")

    stop_supervised!(Conversation)
    start_supervised!({Conversation, id: "t-3", system_prompt: "keep it short", llm: fake})

    {:ok, reply} = Conversation.ask("t-3", "go on")

    assert [%ContentPart{content: "got 4 messages"}] = reply.content
  end
end

The stop_supervised! in the middle of the test is the deploy: the process dies with everything it held in memory. The four in the answer is the proof — the system prompt, the old question and the old answer reread from the database, the new question. Without the rebuild, it would be two.

The async: false is no whim. The one querying the database is the conversation's process, not the test's, and the DataCase Phoenix generates only shares the sandbox connection with other processes when the test is not async. That is the price of testing through a process: this file runs alone, and the price is paid on the first line.

The database now keeps part one's promise: the user comes back tomorrow and the conversation is there. But look at what the rebuild does — it rereads everything, every time. A conversation spanning weeks rebuilds an ever-larger chain, and part four proved that this whole list travels to the model on every round. The history became a cost that only grows, and it is the subject of the last part.