The history that never stops growing

Part four proved with two numbers that the chain resends the whole conversation on every round: two messages, then four. The progression does not stop there. Round k sends 2k messages — the system prompt, the k questions, the k − 1 answers — and a fifty-round conversation sends, all told, 2,550 of them. Each one billed as input, most of them reread for the tenth time. And part five made it worse on purpose: the conversation now comes back tomorrow, and it comes back whole.

Prompt caching softens the curve — the history is a stable prefix, and a cached prefix costs about a tenth to reread — but it does not change its shape. The cache expires in minutes, it demands a minimum prefix size (the router that doubled the bill shows what happens when that minimum is not met), and the model's context window remains a hard ceiling. Once you reach it, the round does not get expensive: it gets impossible.

The number is already in the response

Before containing the cost, measure it. No manual counting: the provider returns the token count with every response, and the library hands it over in two ways.

# in the final message's metadata
chain.last_message.metadata.usage
#=> %LangChain.TokenUsage{input: 9280, output: 164, ...}

# or as a callback, next to the ones part three registered
|> LLMChain.add_callback(%{
  on_llm_token_usage: fn _chain, usage ->
    :telemetry.execute([:my_app, :llm, :usage], %{input: usage.input, output: usage.output})
  end
})

input is the number that matters: it is the size of the conversation that was just resent. Hanging that value on :telemetry — or on a Logger.info, to begin with — answers the question that decides the rest of this part: do your conversations actually grow? A support chat that closes in six rounds does not have this problem, and a problem that does not exist needs no fix.

Trimming is easy and breaks easily

The first impulse is a sliding window: keep the system prompt and the last n messages, one Enum.take(messages, -n) and done. It works until the first conversation with tools.

When the model asks for lookup_order, as in part two, the request becomes an assistant message carrying the call, followed by the message carrying the result — a pair the providers validate. A cut that keeps the result and drops the call produces a sequence the API rejects outright, and the conversation breaks precisely because it grew, which is when it was going well. You can cut correctly, hunting for a round boundary and never splitting the pair; but there is a problem no cut solves: the model forgets everything that left the window, including the name the user gave in the very first message.

Summarizing trades detail for a ceiling

The library ships the alternative:

defmodule MyApp.Summarizer do
  alias LangChain.Chains.SummarizeConversationChain
  alias LangChain.ChatModels.ChatAnthropic

  def maybe_compact(chain) do
    %{
      llm: ChatAnthropic.new!(%{model: "claude-haiku-4-5", stream: false}),
      threshold_count: 20,
      keep_count: 6
    }
    |> SummarizeConversationChain.new!()
    |> SummarizeConversationChain.summarize(chain)
  end
end

Three decisions in that block. The model is the cheapest in the catalog, with no streaming, because summarizing is mechanical work and nobody is watching. The threshold_count: 20 makes summarize/2 return the chain untouched, with no call at all, while the conversation stays under twenty messages — calling maybe_compact/1 every round is free most of the time. And keep_count: 6 preserves the last three verbatim exchanges, which is where the continuity the user notices lives.

Past the threshold, the chain comes back different: the system prompt stays, the entire middle becomes a fabricated pair — a question asking for the summary, an assistant answer holding it — and the six final messages remain intact. To the model, the conversation now has a new beginning. To the user, nothing changed: the screen shows part five's transcript, still whole in the database. The chain became working memory; the database, the memory of record. The rebuild, by the way, returns the full chain — whoever comes back tomorrow pays one full first round, and the summary acts right after.

When the summarizer fails, the library logs a warning and returns the original chain: the conversation does not break, it just does not shrink, and the next round tries again. That silent degradation is comfortable and hides the thermometer — the chain's own callbacks: accepts an on_llm_token_usage so the summary shows up in your telemetry like any other call.

What remains is deciding where this runs, and the answer is the OTP piece the series was still missing:

@impl true
def handle_call({:ask, text}, _from, state) do
  case state.chain |> LLMChain.add_message(Message.new_user!(text)) |> LLMChain.run() do
    {:ok, chain} ->
      # the transcript writes (part five) stay here
      {:reply, {:ok, chain.last_message}, %{state | chain: chain}, {:continue, :compact}}

    {:error, _chain, error} ->
      {:reply, {:error, error}, state, @idle_timeout}
  end
end

@impl true
def handle_continue(:compact, state) do
  {:noreply, %{state | chain: MyApp.Summarizer.maybe_compact(state.chain)}, @idle_timeout}
end

{:continue, :compact} in place of the timeout makes the GenServer answer first and work after: the summary's latency comes out of nobody's wait. The cost exists and has an address — messages are processed in order, so a question arriving during the summary waits for it to finish, a few seconds on the rare round when the threshold fires. And since :continue and a timeout cannot ride the same tuple, the @idle_timeout is rearmed on the way out of handle_continue.

The proof still costs nothing

Part four's fake does not know it exists to impersonate a conversation: it implements the ChatModel behaviour, and the summarizer accepts any implementation. In the test, the same fake plays both roles:

test "past the threshold, the middle of the conversation becomes a summary" do
  fake =
    FakeModel.new!(%{
      reply: fn _messages, _tools ->
        {:ok, Message.new_assistant!("Summary: customer is tracking order A-4471.")}
      end
    })

  chain =
    %{llm: fake}
    |> LLMChain.new!()
    |> LLMChain.add_message(Message.new_system!("keep it short"))
    |> LLMChain.add_messages(
      for n <- 1..4,
          message <- [Message.new_user!("question #{n}"), Message.new_assistant!("answer #{n}")] do
        message
      end
    )

  summarized =
    %{llm: fake, threshold_count: 6, keep_count: 2}
    |> SummarizeConversationChain.new!()
    |> SummarizeConversationChain.summarize(chain)

  assert [_system, _request, summary, kept_user, kept_assistant] = summarized.messages
  assert [%ContentPart{content: "Summary: customer is tracking order A-4471."}] = summary.content
  assert [%ContentPart{content: "question 4"}] = kept_user.content
  assert [%ContentPart{content: "answer 4"}] = kept_assistant.content
end

Nine messages go in, five come out, and the asserts account for the rest: the summary in the middle, the last exchange intact at the end. What the test does not claim is the quality of the summary — that is the real model's decision, and part four's ruler still applies: the plumbing is proven with the fake, the behaviour is verified with evals, outside these pages.

This is where the series closes, whole this time. The state lives in a GenServer that expires on its own, the tools validate what the model sent, the streaming crosses all the way to the screen without freezing the LiveView, the database keeps every closed round, the live history has a ceiling — and the suite proves all of it without touching the network. LangChain did its part; the rest was OTP all along.