LangChain on the BEAM
A conversational agent in Elixir: the chain inside a process, tools the model can call, and streaming all the way to the screen.
Elixir LangChain handles each provider's message format and the tool-calling loop. Beyond that it has no opinion, which leaves you three decisions: where the conversation state lives, what to do when the model takes its time, and who receives the pieces of the answer while they arrive.
OTP has had answers to those three for decades: process, supervisor, mailbox. This series uses those pieces to assemble a conversational agent:
- The chain inside a process. Why a conversation is a
GenServer, and how to spin one up per user without leaking processes. - Tools the model can call. What the model controls when you hand it a function, and where to validate.
- The LiveView cannot wait. Token-by-token streaming, and where the call to the model has to run.
The code assumes {:langchain, "~> 0.9.0"} and a Phoenix nearby. Each part stands on its own, but they were written to be read in order.
03Parts
-
A conversation is a process
LLMChain is an immutable struct, so something has to hold it between one message and the next. That something is a GenServer.
-
Tools the model can call
The model decides whether to call your function and with which arguments. Validating what it sent is still on you.
-
The LiveView cannot wait
Token-by-token streaming, and why the call to the model has to run in another process.