Error handling
UseisRetryable() to decide whether to retry or show a fatal error:
Timing
- Use
audioContext.currentTimefor audio and viseme timing (notDate.now()) - Use
performance.now()for UI timestamps - Schedule audio with 10-50ms lead time to avoid glitches
Interruptions
Clear audio queues and viseme timelines when the user interrupts:Latency
End-to-end latency is STT + LLM + TTS + network.- Denoising steps: the current voice model runs a fixed
4on every endpoint;stepsis accepted and has no effect - Keep connection open: WebSocket handshake adds 200-500ms. Reuse it.
- Streaming playback: Play chunks as they arrive — don’t wait for the full response
- Track TTFB:
client.synthesize({ text, onTTFB: (ms) => ... })
Reconnection
The SDK reconnects automatically with exponential backoff (2s, 4s, 8s, capped at 10s, max 5 attempts):Audio
- Pre-warm AudioContext inside a user gesture (button click), not when the first chunk arrives
- Guard against odd-length PCM buffers: trim the last byte if
length % 2 !== 0 - Limit transcript history to avoid memory leaks
Security
- Never hardcode API keys — use environment variables
- Validate key presence before creating the client