A conversation is a process

LLMChain is an ordinary struct, and run/1 hands back a new one:

{:ok, chain} =
  %{llm: ChatAnthropic.new!(%{model: "claude-sonnet-5"})}
  |> LLMChain.new!()
  |> LLMChain.add_message(Message.new_user!("hi"))
  |> LLMChain.run()

There is no mutable object keeping the history behind your back. The entire conversation sits in that chain variable, and dropping it ends the conversation. Hence the first architecture decision: who holds that struct between one user message and the next?

Where not to keep it

In the LiveView's assigns, the history dies the moment the user opens a second tab. In a global ETS table, you inherit the problem of knowing when to clean up. In the database, on every message, live state becomes I/O.

On the BEAM, live state lives in a process.

One GenServer per conversation

defmodule MyApp.Conversation do
  use GenServer

  alias LangChain.Chains.LLMChain
  alias LangChain.ChatModels.ChatAnthropic
  alias LangChain.Message

  # A conversation idle for 30 minutes does not need a live process.
  @idle_timeout :timer.minutes(30)

  def start_link(opts) do
    GenServer.start_link(__MODULE__, opts, name: via(Keyword.fetch!(opts, :id)))
  end

  def ask(id, text) do
    GenServer.call(via(id), {:ask, text}, :timer.minutes(2))
  end

  defp via(id), do: {:via, Registry, {MyApp.ConversationRegistry, id}}

  @impl true
  def init(opts) do
    chain =
      %{llm: ChatAnthropic.new!(%{model: "claude-sonnet-5"})}
      |> LLMChain.new!()
      |> LLMChain.add_message(Message.new_system!(Keyword.fetch!(opts, :system_prompt)))

    {:ok, %{chain: chain}, @idle_timeout}
  end

  @impl true
  def handle_call({:ask, text}, _from, state) do
    case state.chain |> LLMChain.add_message(Message.new_user!(text)) |> LLMChain.run() do
      {:ok, chain} ->
        {:reply, {:ok, chain.last_message}, %{state | chain: chain}, @idle_timeout}

      {:error, _chain, error} ->
        # The failed chain is discarded. State stays at the last point where a
        # round-trip with the model completed.
        {:reply, {:error, error}, state, @idle_timeout}
    end
  end

  @impl true
  def handle_info(:timeout, state), do: {:stop, :normal, state}
end

Note the timeout returned in every tuple. If no message arrives within thirty minutes, the process receives :timeout and shuts itself down, with no Process.send_after and no periodic sweep. An abandoned conversation cleans itself up.

The two-minute timeout on GenServer.call/3 is deliberate too. The default is five seconds, and a model doing extended reasoning passes that without trying. When it does, the caller raises a timeout error while the work carries on elsewhere, with nobody left to receive the result.

A conversation that does not exist yet

The via above assumes a Registry, and creating the process on demand calls for a DynamicSupervisor. Both go in the supervision tree:

# lib/my_app/application.ex
children = [
  {Registry, keys: :unique, name: MyApp.ConversationRegistry},
  {DynamicSupervisor, name: MyApp.ConversationSupervisor, strategy: :one_for_one}
]

The function that opens a conversation treats :already_started as success, since two tabs from the same user arriving together is routine:

def open(id, system_prompt) do
  spec = {MyApp.Conversation, id: id, system_prompt: system_prompt}

  case DynamicSupervisor.start_child(MyApp.ConversationSupervisor, spec) do
    {:ok, pid} -> {:ok, pid}
    {:error, {:already_started, pid}} -> {:ok, pid}
    error -> error
  end
end

With strategy: :one_for_one, a conversation that crashes takes down exactly one conversation. In a multi-tenant system, that is what keeps one customer's bug off everyone else's screen.

The limit of the process

Keeping the conversation in a process gives you the history already in memory, isolation for free, and expiry through the OTP timeout. Every one of those costs work once the state lives in the database: a query per message, isolation that depends on your code, a cleanup job running on the side.

The database wins on one point, and on that point there is no middle ground: restart the node and the conversation evaporates. If the product requires the user to come back tomorrow and find their history, you persist messages as they settle and rebuild the chain from them when they return. The database becomes the record; the process stays where the conversation happens while it is alive.

The next part hands one of your functions to the model to call.