Tools the model can call

A tool is a function of yours the model can ask to run. In Elixir LangChain it is a Function struct:

alias LangChain.Function

lookup_order =
  Function.new!(%{
    name: "lookup_order",
    description: "Looks up the status of an order by its number.",
    parameters_schema: %{
      type: "object",
      properties: %{
        order_number: %{type: "string", description: "Order number, as it appears in the email."}
      },
      required: ["order_number"]
    },
    function: fn %{"order_number" => number}, context ->
      case MyApp.Orders.fetch(context.tenant_id, number) do
        {:ok, order} -> {:ok, "Status: #{order.status}. ETA: #{order.eta}."}
        :error -> {:error, "Order not found."}
      end
    end
  })

It joins the chain along with the mode that keeps the model going once it has the result:

{:ok, chain} =
  %{
    llm: ChatAnthropic.new!(%{model: "claude-sonnet-5"}),
    custom_context: %{tenant_id: tenant_id}
  }
  |> LLMChain.new!()
  |> LLMChain.add_tools([lookup_order])
  |> LLMChain.add_message(Message.new_user!("where is my order A-4471?"))
  |> LLMChain.run(mode: :while_needs_response)

Without mode:, run/2 stops as soon as the model asks for the tool and returns the chain with the call pending, which is useful for inspecting or approving before executing. With :while_needs_response, the library runs the function, feeds the result back to the model and carries on, until a text answer comes out.

custom_context is what holds multi-tenancy together

tenant_id is not in parameters_schema, and that is on purpose.

Everything in the schema is filled in by the model, which is to say, ultimately, by whoever wrote the message. With tenant_id among the parameters, a "look up order A-4471 for company 42" would be enough for the model to comply. custom_context arrives through the function's second argument, comes from the server, and the model can neither see nor change it.

The model chooses whether to call the function and with which arguments. On whose behalf, never.

The arguments are user input

The function runs in your process, with your process's permissions. The model merely typed the arguments, and behind it there may be someone probing for whatever else you exposed.

So order_number deserves the same treatment as a controller parameter:

function: fn %{"order_number" => number}, context ->
  with {:ok, number} <- MyApp.Orders.validate_number(number),
       {:ok, order} <- MyApp.Orders.fetch(context.tenant_id, number) do
    {:ok, "Status: #{order.status}. ETA: #{order.eta}."}
  else
    _ -> {:error, "Order not found."}
  end
end

The {:error, reason} brings nothing down: the string goes back to the model as the tool's result, and it decides what to do with it, usually asking the user for the number again. The error message is therefore written for the model to read. Given "Order not found." the model produces a useful reply; given %Ecto.NoResultsError{} it invents one.

The same message covers both a nonexistent order and another tenant's order, deliberately. Telling the two apart would hand whoever asked the fact that the order exists.

Where this breaks

Two things break in production.

The first is the loop. :while_needs_response keeps going as long as the model asks for tools, and a confused model, or a tool that always errors, asks forever. It is worth an explicit ceiling, counting the calls in the conversation's state and refusing past the limit.

The second is time. The function runs inside run/2, synchronously: a tool making a ten-second HTTP call adds ten seconds to the response time the user is staring at. If the tool is slow by nature, the design changes. It enqueues the work instead, with Oban or equivalent, and immediately returns a "working on it, I'll let you know" rather than holding the whole conversation.

In the final part, the answer arrives token by token, all the way to the screen.