Voice Chat (Conversational AI)

For the full wire-protocol reference (message-by-message), see WebSocket Protocol. This page is the quick-reference version.

Audio specs

Property	Input	Output
Sample Rate	16 kHz	44.1 kHz
Format	Int16 PCM	Int16 PCM
Channels	1 (mono)	1 (mono)

Agent audio always arrives as a base64-encoded audio JSON message — this endpoint never sends raw binary audio frames to the client (unlike /ws/tts).

Connection

wss://api.lokutor.com/ws/agent?api_key=your-api-key

Initialization

Send configuration as JSON messages after connecting, in this order:

{ "type": "voice", "data": "F1" }
{ "type": "language", "data": "en" }
{ "type": "tools", "data": [{ "type": "function", "function": { "name": "get_weather" } }] }
{ "type": "visemes", "data": true }
{ "type": "prompt", "data": "You are a helpful assistant." }
{ "type": "rates", "playback": 44100, "input": 16000 }

Send rates last — receiving it is what tells the server your configuration handshake is complete and releases the gate on the first conversational turn. Each of voice/language/prompt/rates is acknowledged with a status message (voice_set, language_set, prompt_set, rates_set).

Client to server

Send microphone audio as binary Int16 PCM frames (16 kHz, mono, ~20ms chunks). You can also send the same audio as {"type":"audio","data":"<base64>"} if binary frames aren’t convenient in your environment. Other client → server messages: pitch (adjust pitch-shift), reset (clear conversation context), tool_result (return a client-executed tool’s result — see Tool Calling).

Server to client

Event	Data	When
`status`	`listening`, `thinking`, `speaking`, `interrupted`, `connected`, or a `*_set`/`reset` ack	State changes and config acks
`transcript`	`{ role, data, partial? }`	Speech transcription (user or agent)
`transcript_truncated`	`{ role: "agent", data, full, emitted }`	The user cut the agent off. `data` is what they actually heard, ending in `…` where they stopped hearing it (empty if they heard nothing). If `emitted` is true, replace the agent `transcript` whose text was `full`; otherwise add `data` as a new agent line
`transcript_revised`	`{ role: "user", data, previous }`	The user paused mid-sentence, the agent barely started, and the user carried on: both halves were transcribed together. Replace the last user line (`previous`) with `data` and drop any agent line after it. Sent instead of a `transcript` for the continuation
`audio`	Base64-encoded PCM chunk + `generation`	Agent speaking
`visemes`	Array of `{ v, c, t }`	Lip-sync data
`tool_call`	`{ name, arguments, call_id }`	Tool execution
`server_rates`	`{ playback, input }`	Sent on connect and in response to a client `rates` message
`error`	Error message	Pipeline error

Barge-in

When the user speaks while the agent is talking, the server immediately sends status: interrupted (tentatively — audio for the current response stops arriving right away). It then either confirms the interrupt (a second status: interrupted, plus a truncated context) using a fast on-device classifier (GateTurn) or, failing that, the existing speech-based checks — or dismisses it as a false alarm, in which case no further message is sent and audio for the original response simply resumes under the same generation. See Handling Interruptions for the full mechanism. An agent reply is sent as a transcript as soon as it is generated, which is before it has finished playing. When the user interrupts, a transcript_truncated follows with what they actually heard, and the agent’s own memory of the conversation is corrected the same way. If the user was only pausing and the agent had said no more than a word or two, the user’s two fragments become one turn (transcript_revised) and the agent answers the whole sentence.

WebSocket Protocol — full message reference
Handling Interruptions — barge-in mechanism in detail
Tool Calling — tool_call / tool_result flow and built-in tools

Get Started

SDK

API Reference

Resources

Voice Chat (Conversational AI)

Audio specs

Connection

Initialization

Client to server

Server to client

Barge-in

​Audio specs

​Connection

​Initialization

​Client to server

​Server to client

​Barge-in

​Related Pages

Audio specs

Connection

Initialization

Client to server

Server to client

Barge-in

Related Pages