Voice Chat (Conversational AI)

For the full wire-protocol reference (message-by-message), see WebSocket Protocol. This page is the quick-reference version.

Audio specs

Property	Input	Output
Sample Rate	16 kHz	44.1 kHz
Format	Int16 PCM	Int16 PCM
Channels	1 (mono)	1 (mono)

Agent audio always arrives as a base64-encoded audio JSON message — this endpoint never sends raw binary audio frames to the client (unlike /ws/tts).

Connection

wss://api.lokutor.com/ws/agent?agent_id=your-agent-id

Authenticate the handshake with an Authorization: Bearer or X-API-Key header, or with ?api_key= where headers cannot be set. In a browser, pass a short-lived wst_ ticket, never your key (Authentication). The handshake is refused with 401 for a bad credential, 402 (plain text) when the balance cannot start a session, 404 agent.not_found for an agent_id that is not yours, and 429 auth.rate_limited past your account’s concurrent sessions. Optional query parameters:

Parameter	What it does
`agent_id`	Runs one of your dashboard agents: its prompt, voice, tools, knowledge base and integrations. The `voice`, `language` and `prompt` messages you send after connecting override the agent’s own for this session.
`caller_phone`, `caller_email`	Who the caller is, for an agent with your CRM switched on: the agent looks them up when the call starts and logs the call on their record (Tool Calling → Your CRM). Phone numbers in international form, e.g. `%2B34612345678`.
`disable_external_tools=1`	Leaves out the agent’s tools that call your URLs (webhook tools) for this session. Built-in tools stay.
`nfe`	Denoising steps for this session. Live sessions can only lower it: values are clamped to 1–6, and the default is the maximum. Accepted for compatibility; no effect on versa-2.0.
`cfg`	Voice guidance, clamped to 1–4. Accepted for compatibility; no effect on versa-2.0.
`cfg_text`	Text guidance. Fixed at 1 on live sessions. Accepted for compatibility; no effect on versa-2.0.

Query values override the same keys in the agent’s config.

Agent config

A session with an agent_id runs with that agent’s config, set in the dashboard. Keys documented elsewhere: web_search, google_calendar and the calendar settings, crm and timezone in Tool Calling. This one changes how the agent takes its turn:

Key	Type	Default	What it does
`speculative_llm`	`boolean`	`true`	Speculative replies. The agent starts its reply while the caller pauses, so turns it lands in are answered in about 306 ms instead of about 660 ms. Replies started this way and not used are billed as language-model tokens, about a third of a cent a minute at a typical prompt (Pricing and billing). With `false`, the agent asks the model only once the caller’s turn has ended: no unused replies to pay for, and replies about 255 ms slower on average.

Initialization

Send configuration as JSON messages after connecting, in this order:

{ "type": "voice", "data": "F1" }
{ "type": "language", "data": "en" }
{ "type": "tools", "data": [{ "type": "function", "function": { "name": "get_weather" } }] }
{ "type": "visemes", "data": false }
{ "type": "prompt", "data": "You are a helpful assistant." }
{ "type": "rates", "playback": 44100, "input": 16000 }

Send rates last: receiving it tells the server your configuration is complete and releases the first conversational turn. Each of voice/language/prompt/rates is acknowledged with a status message (voice_set, language_set, prompt_set, rates_set). With an agent_id, every message except rates is optional: send only what you want to override. Without one, send at least prompt. An unknown voice or language is answered with an error frame, {"type": "error", "data": {"code": "validation.invalid_voice", …}} (or validation.invalid_language), and no voice_set / language_set ack; the session keeps its previous value. Language. A language (or the agent’s own) is kept for the whole session: the agent speaks it, and speech recognition listens for it, short replies included. Send an empty string, {"type": "language", "data": ""}, to let the agent follow the caller instead. It recognises the caller’s language from what they say and replies in it in the same voice. A short reply such as “yes” or a number is heard in the language the caller last spoke, and the agent switches when the caller does. With neither an agent_id nor a language message, the agent speaks English and recognition follows the caller.

Client to server

Send microphone audio as binary Int16 PCM frames (16 kHz, mono, ~20ms chunks). You can also send the same audio as {"type":"audio","data":"<base64>"} if binary frames aren’t convenient in your environment. Other client → server messages: pitch (adjust pitch-shift), reset (clear conversation context), tool_result (return a client-executed tool’s result — see Tool Calling).

Server to client

Event	Data	When
`status`	`listening`, `thinking`, `speaking`, `interrupted`, `connected`, or a `*_set`/`reset` ack	State changes and config acks
`transcript`	`{ role, data, partial? }`	Speech transcription (user or agent)
`transcript_truncated`	`{ role: "agent", data, full, emitted }`	The user cut the agent off. `data` is what they actually heard, ending in `…` where they stopped hearing it (empty if they heard nothing). If `emitted` is true, replace the agent `transcript` whose text was `full`; otherwise add `data` as a new agent line
`transcript_revised`	`{ role: "user", data, previous }`	The user paused mid-sentence, the agent barely started, and the user carried on: both halves were transcribed together. Replace the last user line (`previous`) with `data` and drop any agent line after it. Sent instead of a `transcript` for the continuation
`audio`	Base64-encoded PCM chunk + `generation`	Agent speaking
`visemes`	Array of `{ v, c, t }`	Lip-sync data
`tool_call`	`{ name, arguments, call_id }`	Tool execution
`server_rates`	`{ playback, input }`	Sent on connect and in response to a client `rates` message
`error`	A structured error object (`code`, `message`, `detail`, `retryable`) or a plain string	Configuration and pipeline errors. Invalid voice or language arrive structured; runtime pipeline errors may be a string. Handle both.

Barge-in

When the user starts speaking over the agent, the server sends status: listening and stops sending new audio for the reply, but the barge-in is only tentative: keep playing what you already have. status: interrupted follows once the interruption is real (the user kept talking for 400 ms, or their utterance is confirmed as speech rather than noise, a backchannel or an echo of the agent); stop playback and clear queued audio then. If it was a false alarm, the server sends status: speaking (or thinking, if the reply was still being generated) and the reply continues under the same generation. See Handling Interruptions. An agent reply is sent as a transcript as soon as it is generated, which is before it has finished playing. When the user interrupts, a transcript_truncated follows with what they actually heard, and the agent’s own memory of the conversation is corrected the same way. If the user was only pausing and the agent had said no more than a word or two, the user’s two fragments become one turn (transcript_revised) and the agent answers the whole sentence. A reply the user never heard any of — they started speaking again before it began to play — is not kept. It is either never sent as a transcript, or, if it already was, withdrawn with a transcript_truncated whose data is empty and emitted true. The agent forgets it too, and answers what the user goes on to say.

Silence

If the user says nothing for 10 seconds after the agent finishes speaking, the agent speaks once, unprompted: it asks whether they are still there or need anything else — or, if their last turn went unanswered, answers it. It arrives like any other reply (transcript, then audio under a new generation), at most once per silence; the next time the user speaks re-arms it. It never acts on the user’s behalf: no tool is called while they are silent, even if the agent’s last line offered to do something.

Idle sessions

A session in which nobody has spoken for 5 minutes ends: the agent says goodbye (as an ordinary transcript and audio), and the socket closes with code 1000 and the reason nobody has spoken for 5 minutes. A session whose account balance runs out closes with 1008 and the reason account balance exhausted.

WebSocket Protocol — full message reference
Handling Interruptions — barge-in mechanism in detail
Tool Calling — tool_call / tool_result flow and built-in tools

Get Started

SDK

API Reference

Resources

Voice Chat (Conversational AI)

Audio specs

Connection

Agent config

Initialization

Client to server

Server to client

Barge-in

Silence

Idle sessions

​Audio specs

​Connection

​Agent config

​Initialization

​Client to server

​Server to client

​Barge-in

​Silence

​Idle sessions

​Related Pages

Audio specs

Connection

Agent config

Initialization

Client to server

Server to client

Barge-in

Silence

Idle sessions

Related Pages