Testing without spending tokens

The agent from the previous three parts works — and any test that goes through it calls the real API. That is three taxes on every mix test run: money, because every call is billed; time, because each one takes seconds; and trust, because the model phrases its answer differently on every run, so an assert on the text passes today and breaks tomorrow with nothing changed in your code.

The way out sits in how the library uses the model:

# inside the library, simplified
%module{} = chain.llm
module.call(chain.llm, chain.messages, chain.tools)

It pulls the module out of whatever struct sits in llm and calls call/3 on it — right after injecting the chain's callbacks into that struct, a detail that will matter in a moment. Any struct qualifies, as long as its module implements the LangChain.ChatModels.ChatModel behaviour. ChatAnthropic is one implementation. A fake of yours is another.

The reply comes from the test

defmodule MyApp.FakeModel do
  @behaviour LangChain.ChatModels.ChatModel

  # The chain injects its callbacks with `%{llm | callbacks: ...}`;
  # without the field, that struct update raises.
  defstruct reply: nil, callbacks: []

  def new!(attrs \\ %{}), do: struct!(__MODULE__, attrs)

  @impl true
  def call(%__MODULE__{} = model, messages, tools), do: model.reply.(messages, tools)

  @impl true
  def retry_on_fallback?(_error), do: false

  @impl true
  def serialize_config(_model), do: %{}

  @impl true
  def restore_from_map(_data), do: {:error, "a fake does not restore"}
end

Four required callbacks, three of them answering with the bare minimum: the fake stays out of fallback and serialization. The entire behaviour lives in reply, a function each test hands over ready-made. The fake decides nothing, and that is the difference that matters: in the test, the one playing the model is you.

The seam in the GenServer

Part one's init/1 built the ChatAnthropic inside itself, and what is born inside cannot be swapped from outside. The change is the model becoming an option with a default:

@impl true
def init(opts) do
  llm =
    Keyword.get_lazy(opts, :llm, fn ->
      ChatAnthropic.new!(%{model: "claude-sonnet-5"})
    end)

  chain =
    %{llm: llm}
    |> LLMChain.new!()
    |> LLMChain.add_message(Message.new_system!(Keyword.fetch!(opts, :system_prompt)))

  {:ok, %{chain: chain}, @idle_timeout}
end

Production passes no :llm and keeps the same ChatAnthropic as before; the test passes the fake. That is full dependency injection in one line, no library required. The get_lazy spares building a model that gets thrown away whenever the test brings its own.

What two numbers prove

defmodule MyApp.ConversationTest do
  use ExUnit.Case, async: true

  alias LangChain.LangChainError
  alias LangChain.Message
  alias LangChain.Message.ContentPart
  alias MyApp.{Conversation, FakeModel}

  test "the chain resends the whole conversation every round" do
    fake =
      FakeModel.new!(%{
        reply: fn messages, _tools ->
          {:ok, Message.new_assistant!("got #{length(messages)} messages")}
        end
      })

    start_supervised!({Conversation, id: "t-1", system_prompt: "keep it short", llm: fake})

    {:ok, first} = Conversation.ask("t-1", "hi")
    {:ok, second} = Conversation.ask("t-1", "go on")

    assert [%ContentPart{content: "got 2 messages"}] = first.content
    assert [%ContentPart{content: "got 4 messages"}] = second.content
  end
end

The fake returns the size of the list it received, and the numbers tell the story. First round: the system prompt and the question, two messages. Second: those two, plus the model's previous answer, plus the new question — four. With no network and in milliseconds, the test just proved that the GenServer accumulates the history and resends all of it every round, which is exactly what part one promised.

Matching on a list of ContentPart is no whim: it is the shape a message carries text in, the same one that showed up in the previous part's deltas. And start_supervised! starts the conversation under the test's own supervisor, which tears it down at the end; with a unique id per test, the application's Registry sees no collision even under async: true.

The provider's worst day, in one line

test "a provider error does not kill the conversation" do
  fake =
    FakeModel.new!(%{
      reply: fn _messages, _tools ->
        {:error, LangChainError.exception(type: "overloaded", message: "provider overloaded")}
      end
    })

  pid = start_supervised!({Conversation, id: "t-2", system_prompt: "keep it short", llm: fake})

  assert {:error, %LangChainError{type: "overloaded"}} = Conversation.ask("t-2", "hi")
  assert Process.alive?(pid)
end

This is the path the real model almost never lets you rehearse: the provider does not overload itself on your testing schedule. With the fake, its worst day costs one line. The final assert is the one that matters: the conversation swallowed the error and lives on — the behaviour part one's handle_call chose when it discarded the failed chain and kept the last good state.

And reply reaches further. Returning a message with tool_calls makes part two's tool loop run entirely inside the test, with your validation in the middle. A Process.sleep before answering rehearses the slow model and part one's timeouts. Every provider behaviour that ever cost you a late night becomes a three-line function.

What the fake does not prove

The fake proves the plumbing: the history grows, the error does not kill, the tool runs, the delta arrives. What it cannot prove is the one thing the real model does — decide. Whether the prompt leads the model to call lookup_order at the right moment, whether the answer comes out in the tone you asked for: only a real call answers that. Verifying behaviour with real calls has a name of its own, evals, and it falls outside this series.

The division that stands: the plumbing is proven with the fake, on every mix test, for free; the model's behaviour is verified separately, with real calls, few and deliberate.

The agent now proves itself for free on every mix test. What it still cannot do is survive: part one warned that a node restart evaporates the conversation — and deploys happen on weekdays, with conversations open. The next part writes each round to the database without moving the process out of the center.