Skip to main content

​
Text-to-speech

First audio arrives in a few hundred milliseconds. Measured on Lokutor’s production path: What you can do:
  • Stream. Use the WebSocket (/ws/tts, or TTSClient) and start playback on the first chunk. POST /tts/synthesize returns only when the whole clip is done.
  • Reuse the connection. A new WebSocket costs a TLS and WebSocket handshake. Over the raw API you can send several requests on one /ws/tts socket. (TTSClient opens a new socket per call.)
  • Measure it. TTSClient.synthesize({ …, onTTFB: (ms) => … }) reports the time from sending the request to the first audio.
steps cannot be tuned: the current model always runs 4 denoising steps (Parameters).

​
Voice agents

A reply’s latency is the sum of detecting that the caller has finished, transcription, the language model’s first sentence, and synthesis of that sentence. The language model is usually the largest part.
  • Speculative replies (speculative_llm, on by default) start the reply while the caller pauses. Turns where it lands are answered in about 306 ms instead of about 660 ms. Turning it off saves the tokens of unused speculative replies and makes replies about 255 ms slower on average (Voice Chat).
  • Keep prompts short. A longer system prompt and knowledge base make the model slower to start.
  • Tools add time. Each tool call is another round trip to your code or to a built-in service; web search adds a few seconds.
  • Clear your queue on interrupted and when a new reply starts (status: thinking with a newer generation), so a new reply is not stuck behind old audio (WebSocket Protocol).