Bring your own LLM or agent

Already have an agent: a LangGraph or LangChain graph, a LiteLLM or vLLM gateway, a service of your own on any cloud? You don’t have to rebuild it. Lokutor handles everything around the model (speech recognition, deciding when the caller has finished, barge-in, speech synthesis, phone lines, tool execution) and your code supplies the intelligence. Both ways use the same conversation format, the OpenAI chat messages, so what you build for one works for the other.

	Endpoint mode	Client mode
Who calls whom	Lokutor’s servers call your URL	Your process answers over the session’s WebSocket
Where your code can run	A public https address	Anywhere: your laptop, a private network, the same program as the SDK
Phone calls	Yes	No: there is no SDK process on a phone call
Secrets	Your endpoint’s key is stored with the agent, or sent per session	Nothing to expose: no URL, no key
Set it in	The agent’s config, or the `llm` option of the SDK	`on_llm_request` (Python) / `onLlmRequest` (JS)

If you can run a public endpoint, or you need phone calls, use endpoint mode. If your model or tools sit behind a firewall, or you are developing on a laptop, use client mode.

Client mode

Your model runs in the same process that holds the session. Lokutor sends each conversation to your function; you return the reply, all at once or as you produce it. Lokutor starts speaking at the first sentence, and tells you to stop when the caller speaks over the reply.

from lokutor import VoiceAgentClient

def on_llm_request(messages, tools):
    # messages: OpenAI chat messages. messages[0] is Lokutor's system prompt (your agent's instructions plus
    # how to speak on a call); use it, extend it or ignore it.
    for token in my_agent.stream(messages):        # a LangGraph graph, a LangChain runnable, an API call ...
        yield token                                # a string; text is spoken as it arrives

client = VoiceAgentClient(
    api_key="YOUR_LOKUTOR_KEY",
    prompt="You are the receptionist of a dental clinic.",
    on_llm_request=on_llm_request,
    llm_tools="none",                              # your agent runs its own tools; see "Tools" below
)
client.start_conversation()

import { VoiceAgentClient } from '@lokutor/sdk';

const client = new VoiceAgentClient({
  apiKey: 'YOUR_LOKUTOR_KEY',
  prompt: 'You are the receptionist of a dental clinic.',
  llmTools: 'none',
  async *onLlmRequest(messages, tools, signal) {
    for await (const token of myAgent.stream(messages, { signal })) yield token;   // signal aborts when the caller interrupts
  },
});
await client.connect();

The handler may return a string, or a generator (async generators in JavaScript) that yields strings. An exception, or a handler that does not answer within 20 seconds, ends that turn with no reply and is reported in the session as an error message. The Python SDK runs each request on its own thread, so a slow model never blocks the microphone.

LangGraph in the same process

from langgraph.prebuilt import create_react_agent

graph = create_react_agent(model, tools=[lookup_order, refund])   # your tools run in YOUR process

def on_llm_request(messages, tools):
    for chunk, meta in graph.stream({"messages": messages}, stream_mode="messages"):
        if chunk.content and meta.get("langgraph_node") == "agent":   # the node that writes to the caller (name depends on your graph)
            yield chunk.content

Endpoint mode

Point an agent, or one session, at an OpenAI-compatible chat completions endpoint.

client = VoiceAgentClient(
    api_key="YOUR_LOKUTOR_KEY",
    prompt="You are the receptionist of a dental clinic.",
    llm={"base_url": "https://agent.example.com/v1", "api_key": "YOUR_ENDPOINT_KEY",
         "model": "my-agent", "headers": {"X-Tenant": "acme"}, "tools": "auto"},
)

const client = new VoiceAgentClient({
  apiKey: 'YOUR_LOKUTOR_KEY',
  prompt: 'You are the receptionist of a dental clinic.',
  llm: { baseUrl: 'https://agent.example.com/v1', apiKey: 'YOUR_ENDPOINT_KEY', model: 'my-agent', headers: { 'X-Tenant': 'acme' } },
});

{
  "llm_provider": "custom",
  "llm_base_url": "https://agent.example.com/v1",
  "llm_api_key": "YOUR_ENDPOINT_KEY",
  "llm_model": "my-agent",
  "llm_headers": { "X-Tenant": "acme" },
  "llm_tools": "auto"
}

Field	Required	Meaning
`base_url` / `llm_base_url`	Yes	The chat completions URL, or the API base: `https://host/v1` gets `/chat/completions` added. Must be https on a public address.
`api_key`	No	Sent as `Authorization: Bearer ...`. Not needed if you authenticate with `headers`.
`model`	No	The `model` field of the request. Left out when you don’t set one.
`headers`	No	Up to 10 extra request headers (a tenant id, a different auth scheme). `Host`, `Content-Type`, `Content-Length`, `Accept`, `User-Agent` and similar are reserved. An `Authorization` header here replaces the key.
`tools`	No	`auto` (default): the agent’s tools are sent as OpenAI `tools`. `none`: never sent; your agent runs its own.
`speculative_llm`	No	Off by default for your endpoint (see below).

Don’t put a secret in browser code. For a browser, set the endpoint on the agent (the key stays on Lokutor’s side) or keep the endpoint behind a gateway that needs no key.

What your endpoint receives

A standard request. Lokutor asks for a stream:

POST /v1/chat/completions
Authorization: Bearer YOUR_ENDPOINT_KEY
Content-Type: application/json

{
  "model": "my-agent",
  "stream": true,
  "stream_options": { "include_usage": true },
  "messages": [
    { "role": "system", "content": "# Identity ... # Conversation Context\nYou are the receptionist of a dental clinic. ..." },
    { "role": "assistant", "content": "Hola, clínica Sonrisa, ¿en qué puedo ayudarte?" },
    { "role": "user", "content": "Quería una cita para una limpieza." }
  ],
  "tools": [ { "type": "function", "function": { "name": "book_appointment", "parameters": { } } } ],
  "tool_choice": "auto"
}

It answers with a server-sent-event stream of chat.completion.chunk (content deltas, optional tool_calls deltas, an optional last chunk with usage, then data: [DONE]). A server that ignores stream and returns one chat.completion JSON body works too, and a strict server that rejects stream_options is retried without it. custom_llm_server.py is a complete minimal endpoint (about 60 lines of FastAPI) to wrap any agent.

Tools

Your agent may use Lokutor’s tools, its own, or both.

You want	Do this
Lokutor’s tools: the agent’s custom API tools, `end_call`, call transfer, calendar, CRM, knowledge base, web search	Leave `tools` on `auto`. Lokutor sends them as OpenAI `tools`. Reply with a `tool_calls` message (an `llm_tool_call` in client mode); Lokutor runs the tool and calls you again with the conversation plus an `assistant` message holding your call and a `tool` message with the result. This is the standard OpenAI tool loop.
Your own tools, entirely	Set `tools` to `none` and run them inside your agent, answering with text only. Lokutor never sees them. You give up Lokutor’s `end_call` and transfer, since only Lokutor can hang up or transfer a phone call.
Both	Leave `tools` on `auto`. Call Lokutor’s tools through the protocol; run everything else inside your agent. A call to a name that is not one of the tools you were sent isn’t executed and gets an error result.

Tool arguments are checked against the tool’s schema before it runs. A call with missing, blank or mistyped required arguments isn’t executed: you get an error result and can call again or ask the caller. Lokutor’s own ask_expert and background-task tools are not offered when you bring your own model: you are the intelligence.

What your model also receives

Besides the caller’s turns, Lokutor sends a few other chat requests through the same model, so make sure yours treats them as ordinary chat:

the opening line, when the agent speaks first and has no fixed first message;
a check-in after a long silence, with a note that the caller has said nothing;
a summary of a long conversation, to keep it within budget;
after the call, your agent’s evaluation and data-collection rules, when you use them (in client mode there is no session left to answer them, so they are skipped).

What is different from the platform’s own model

Speculative replies are off. The platform’s model starts a reply during the caller’s pause and discards it when they keep talking, about 0.85 extra requests per turn. Sending those to your service would multiply its load, so they are off unless you set speculative_llm to true.
No fallback. If your endpoint or your handler fails or is too slow, that turn gets no spoken reply and the session receives an error message that says why. Lokutor does not answer with its own model in its place. Your endpoint has 20 seconds to start answering.
Latency is yours. Time to first audio is your time to the first sentence plus Lokutor’s recognition, turn detection and speech synthesis. Stream, and start with a short sentence: say “Let me check that” before a slow lookup.
Billing. The platform minute, recognition and speech are billed as usual. Model tokens are not billed: you pay your own endpoint. See Pricing.
Security. Endpoint mode is called from Lokutor’s servers, so an address that is not public is refused: localhost, private ranges, link-local and cloud-metadata addresses, and a hostname that resolves to one, even through a redirect. For local development, use client mode, or a tunnel such as cloudflared tunnel --url http://localhost:8000.

Protocol reference (client mode)

Everything is JSON on the session’s WebSocket (protocol).

Direction	Message
You → Lokutor, once, at connect	`{"type":"llm","data":{"mode":"client","tools":"auto"}}` (the SDKs send it for you)
Lokutor → you	`{"type":"llm_request","request_id":"r1","messages":[...],"tools":[...]}`
You → Lokutor	`{"type":"llm_chunk","request_id":"r1","data":"text"}`, any number
You → Lokutor	`{"type":"llm_tool_call","request_id":"r1","id":"c1","name":"...","arguments":"{...}"}`
You → Lokutor	`{"type":"llm_done","request_id":"r1"}` or `{"type":"llm_error","request_id":"r1","data":"why"}`
Lokutor → you	`{"type":"llm_cancel","request_id":"r1"}` when the caller speaks over the reply: stop generating

Troubleshooting

You see	Cause
An `error` message starting `llm:` about `llm_base_url`	The URL is `http`, local or private. Use https on a public host, or use client mode.
`error` message `LLM error: custom llm endpoint answered HTTP 401: ...`	Your endpoint refused the key or headers; the status and first bytes of its body are in the message.
Silence after the caller speaks, then an error about the endpoint not answering	Your model took more than 20 seconds to start. Stream earlier, or answer a short sentence first.
The agent reads out raw JSON	A tool call came back as text. `tool_calls` arguments must be a JSON string (or object) and the tool name one that was sent.
Requests hit your endpoint twice per turn	`speculative_llm` is on for the agent. Set it to `false`.

Get Started

SDK

API Reference

Resources

Bring your own LLM or agent

Client mode

LangGraph in the same process

Endpoint mode

What your endpoint receives

Tools

What your model also receives

What is different from the platform’s own model

Protocol reference (client mode)

Troubleshooting

​Client mode

​LangGraph in the same process

​Endpoint mode

​What your endpoint receives

​Tools

​What your model also receives

​What is different from the platform’s own model

​Protocol reference (client mode)

​Troubleshooting

Client mode

LangGraph in the same process

Endpoint mode

What your endpoint receives

Tools

What your model also receives

What is different from the platform’s own model

Protocol reference (client mode)

Troubleshooting