> ## Documentation Index
> Fetch the complete documentation index at: https://docs.lokutor.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Bring your own LLM or agent

> Keep your own model, graph or agent. Lokutor listens, decides when the caller has finished, runs your tools and speaks; your code supplies the reply. Two ways: an OpenAI-compatible HTTPS endpoint, or your own process answering over the session socket.

Already have an agent: a LangGraph or LangChain graph, a LiteLLM or vLLM gateway, a service of your own on any cloud? You don't have to rebuild it. Lokutor handles everything around the model (speech recognition, deciding when the caller has finished, barge-in, speech synthesis, phone lines, tool execution) and your code supplies the intelligence.

Both ways use the same conversation format, the OpenAI chat messages, so what you build for one works for the other.

| | **Endpoint mode** | **Client mode** |
| :- | :- | :- |
| Who calls whom | Lokutor's servers call **your URL** | Your **process** answers over the session's WebSocket |
| Where your code can run | A **public https** address | **Anywhere**: your laptop, a private network, the same program as the SDK |
| Phone calls | Yes | No: there is no SDK process on a phone call |
| Secrets | Your endpoint's key is stored with the agent, or sent per session | Nothing to expose: no URL, no key |
| Set it in | The agent's config, or the `llm` option of the SDK | `on_llm_request` (Python) / `onLlmRequest` (JS) |

If you can run a public endpoint, or you need phone calls, use endpoint mode. If your model or tools sit behind a firewall, or you are developing on a laptop, use client mode.

## Client mode

Your model runs in the same process that holds the session. Lokutor sends each conversation to your function; you return the reply, all at once or as you produce it. Lokutor starts speaking at the first sentence, and tells you to stop when the caller speaks over the reply.

```python Python theme={null}
from lokutor import VoiceAgentClient

def on_llm_request(messages, tools):
    # messages: OpenAI chat messages. messages[0] is Lokutor's system prompt (your agent's instructions plus
    # how to speak on a call); use it, extend it or ignore it.
    for token in my_agent.stream(messages):        # a LangGraph graph, a LangChain runnable, an API call ...
        yield token                                # a string; text is spoken as it arrives

client = VoiceAgentClient(
    api_key="YOUR_LOKUTOR_KEY",
    prompt="You are the receptionist of a dental clinic.",
    on_llm_request=on_llm_request,
    llm_tools="none",                              # your agent runs its own tools; see "Tools" below
)
client.start_conversation()
```

```typescript JavaScript theme={null}
import { VoiceAgentClient } from '@lokutor/sdk';

const client = new VoiceAgentClient({
  apiKey: 'YOUR_LOKUTOR_KEY',
  prompt: 'You are the receptionist of a dental clinic.',
  llmTools: 'none',
  async *onLlmRequest(messages, tools, signal) {
    for await (const token of myAgent.stream(messages, { signal })) yield token;   // signal aborts when the caller interrupts
  },
});
await client.connect();
```

The handler may return a string, or a generator (async generators in JavaScript) that yields strings. An exception, or a handler that does not answer within 20 seconds, ends that turn with no reply and is reported in the session as an `error` message. The Python SDK runs each request on its own thread, so a slow model never blocks the microphone.

### LangGraph in the same process

```python theme={null}
from langgraph.prebuilt import create_react_agent

graph = create_react_agent(model, tools=[lookup_order, refund])   # your tools run in YOUR process

def on_llm_request(messages, tools):
    for chunk, meta in graph.stream({"messages": messages}, stream_mode="messages"):
        if chunk.content and meta.get("langgraph_node") == "agent":   # the node that writes to the caller (name depends on your graph)
            yield chunk.content
```

## Endpoint mode

Point an agent, or one session, at an OpenAI-compatible chat completions endpoint.

```python Python (one session) theme={null}
client = VoiceAgentClient(
    api_key="YOUR_LOKUTOR_KEY",
    prompt="You are the receptionist of a dental clinic.",
    llm={"base_url": "https://agent.example.com/v1", "api_key": "YOUR_ENDPOINT_KEY",
         "model": "my-agent", "headers": {"X-Tenant": "acme"}, "tools": "auto"},
)
```

```typescript JavaScript (one session) theme={null}
const client = new VoiceAgentClient({
  apiKey: 'YOUR_LOKUTOR_KEY',
  prompt: 'You are the receptionist of a dental clinic.',
  llm: { baseUrl: 'https://agent.example.com/v1', apiKey: 'YOUR_ENDPOINT_KEY', model: 'my-agent', headers: { 'X-Tenant': 'acme' } },
});
```

```json Agent config (every session and phone call) theme={null}
{
  "llm_provider": "custom",
  "llm_base_url": "https://agent.example.com/v1",
  "llm_api_key": "YOUR_ENDPOINT_KEY",
  "llm_model": "my-agent",
  "llm_headers": { "X-Tenant": "acme" },
  "llm_tools": "auto"
}
```

| Field | Required | Meaning |
| :- | :- | :- |
| `base_url` / `llm_base_url` | Yes | The chat completions URL, or the API base: `https://host/v1` gets `/chat/completions` added. Must be **https** on a **public address**. |
| `api_key` | No | Sent as `Authorization: Bearer ...`. Not needed if you authenticate with `headers`. |
| `model` | No | The `model` field of the request. Left out when you don't set one. |
| `headers` | No | Up to 10 extra request headers (a tenant id, a different auth scheme). `Host`, `Content-Type`, `Content-Length`, `Accept`, `User-Agent` and similar are reserved. An `Authorization` header here replaces the key. |
| `tools` | No | `auto` (default): the agent's tools are sent as OpenAI `tools`. `none`: never sent; your agent runs its own. |
| `speculative_llm` | No | Off by default for your endpoint (see below). |

Don't put a secret in browser code. For a browser, set the endpoint on the agent (the key stays on Lokutor's side) or keep the endpoint behind a gateway that needs no key.

### What your endpoint receives

A standard request. Lokutor asks for a stream:

```http theme={null}
POST /v1/chat/completions
Authorization: Bearer YOUR_ENDPOINT_KEY
Content-Type: application/json

{
  "model": "my-agent",
  "stream": true,
  "stream_options": { "include_usage": true },
  "messages": [
    { "role": "system", "content": "# Identity ... # Conversation Context\nYou are the receptionist of a dental clinic. ..." },
    { "role": "assistant", "content": "Hola, clínica Sonrisa, ¿en qué puedo ayudarte?" },
    { "role": "user", "content": "Quería una cita para una limpieza." }
  ],
  "tools": [ { "type": "function", "function": { "name": "book_appointment", "parameters": { } } } ],
  "tool_choice": "auto"
}
```

It answers with a server-sent-event stream of `chat.completion.chunk` (content deltas, optional `tool_calls` deltas, an optional last chunk with `usage`, then `data: [DONE]`). A server that ignores `stream` and returns one `chat.completion` JSON body works too, and a strict server that rejects `stream_options` is retried without it. [`custom_llm_server.py`](https://github.com/lokutor-ai/lokutor-sdk/blob/main/python/examples/custom_llm_server.py) is a complete minimal endpoint (about 60 lines of FastAPI) to wrap any agent.

## Tools

Your agent may use Lokutor's tools, its own, or both.

| You want | Do this |
| :- | :- |
| Lokutor's tools: the agent's custom API tools, `end_call`, call transfer, calendar, CRM, knowledge base, web search | Leave `tools` on `auto`. Lokutor sends them as OpenAI `tools`. Reply with a `tool_calls` message (an `llm_tool_call` in client mode); Lokutor runs the tool and calls you again with the conversation plus an `assistant` message holding your call and a `tool` message with the result. This is the standard OpenAI tool loop. |
| Your own tools, entirely | Set `tools` to `none` and run them inside your agent, answering with text only. Lokutor never sees them. You give up Lokutor's `end_call` and transfer, since only Lokutor can hang up or transfer a phone call. |
| Both | Leave `tools` on `auto`. Call Lokutor's tools through the protocol; run everything else inside your agent. A call to a name that is not one of the tools you were sent isn't executed and gets an error result. |

Tool arguments are checked against the tool's schema before it runs. A call with missing, blank or mistyped required arguments isn't executed: you get an error result and can call again or ask the caller.

Lokutor's own `ask_expert` and background-task tools are not offered when you bring your own model: you are the intelligence.

## What your model also receives

Besides the caller's turns, Lokutor sends a few other chat requests through the same model, so make sure yours treats them as ordinary chat:

* the opening line, when the agent speaks first and has no fixed first message;
* a check-in after a long silence, with a note that the caller has said nothing;
* a summary of a long conversation, to keep it within budget;
* after the call, your agent's evaluation and data-collection rules, when you use them (in client mode there is no session left to answer them, so they are skipped).

## What is different from the platform's own model

* **Speculative replies are off.** The platform's model starts a reply during the caller's pause and discards it when they keep talking, about 0.85 extra requests per turn. Sending those to your service would multiply its load, so they are off unless you set `speculative_llm` to `true`.
* **No fallback.** If your endpoint or your handler fails or is too slow, that turn gets no spoken reply and the session receives an `error` message that says why. Lokutor does not answer with its own model in its place. Your endpoint has 20 seconds to start answering.
* **Latency is yours.** Time to first audio is your time to the first sentence plus Lokutor's recognition, turn detection and speech synthesis. Stream, and start with a short sentence: say "Let me check that" before a slow lookup.
* **Billing.** The platform minute, recognition and speech are billed as usual. Model tokens are not billed: you pay your own endpoint. See [Pricing](https://docs.lokutor.com/pricing.md).
* **Security.** Endpoint mode is called from Lokutor's servers, so an address that is not public is refused: `localhost`, private ranges, link-local and cloud-metadata addresses, and a hostname that resolves to one, even through a redirect. For local development, use client mode, or a tunnel such as `cloudflared tunnel --url http://localhost:8000`.

## Protocol reference (client mode)

Everything is JSON on the session's WebSocket ([protocol](https://docs.lokutor.com/guides/websocket-protocol.md)).

| Direction | Message |
| :- | :- |
| You → Lokutor, once, at connect | `{"type":"llm","data":{"mode":"client","tools":"auto"}}` (the SDKs send it for you) |
| Lokutor → you | `{"type":"llm_request","request_id":"r1","messages":[...],"tools":[...]}` |
| You → Lokutor | `{"type":"llm_chunk","request_id":"r1","data":"text"}`, any number |
| You → Lokutor | `{"type":"llm_tool_call","request_id":"r1","id":"c1","name":"...","arguments":"{...}"}` |
| You → Lokutor | `{"type":"llm_done","request_id":"r1"}` or `{"type":"llm_error","request_id":"r1","data":"why"}` |
| Lokutor → you | `{"type":"llm_cancel","request_id":"r1"}` when the caller speaks over the reply: stop generating |

## Troubleshooting

| You see | Cause |
| :- | :- |
| An `error` message starting `llm:` about `llm_base_url` | The URL is `http`, local or private. Use https on a public host, or use client mode. |
| `error` message `LLM error: custom llm endpoint answered HTTP 401: ...` | Your endpoint refused the key or headers; the status and first bytes of its body are in the message. |
| Silence after the caller speaks, then an error about the endpoint not answering | Your model took more than 20 seconds to start. Stream earlier, or answer a short sentence first. |
| The agent reads out raw JSON | A tool call came back as text. `tool_calls` arguments must be a JSON string (or object) and the tool name one that was sent. |
| Requests hit your endpoint twice per turn | `speculative_llm` is on for the agent. Set it to `false`. |


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.