Skip to main content

Error handling

Use isRetryable() to decide whether to retry or show a fatal error:

Timing

  • Use audioContext.currentTime for audio and viseme timing (not Date.now())
  • Use performance.now() for UI timestamps
  • Schedule audio with 10-50ms lead time to avoid glitches

Interruptions

Clear audio queues and viseme timelines when the user interrupts:

Latency

End-to-end latency is STT + LLM + TTS + network.
  • Denoising steps: the current voice model runs a fixed 4 on every endpoint; steps is accepted and has no effect
  • Keep connection open: WebSocket handshake adds 200-500ms. Reuse it.
  • Streaming playback: Play chunks as they arrive — don’t wait for the full response
  • Track TTFB: client.synthesize({ text, onTTFB: (ms) => ... })

Reconnection

The SDK reconnects automatically with exponential backoff (2s, 4s, 8s, capped at 10s, max 5 attempts):

Audio

  • Pre-warm AudioContext inside a user gesture (button click), not when the first chunk arrives
  • Guard against odd-length PCM buffers: trim the last byte if length % 2 !== 0
  • Limit transcript history to avoid memory leaks

Security

  • Never hardcode API keys — use environment variables
  • Validate key presence before creating the client