STT & TTS vendors
VoiceThere cloud runners synthesize and transcribe speech using providers you pick per project. Configure STT and TTS independently (for example Deepgram STT with OpenAI TTS, or VoiceThere for both with no API keys).
Where to configure
- Dashboard: Project Settings → Voice (STT / TTS). Third-party keys go in project secrets (see Agent environment & secrets).
- Control plane API: Patch
stt_provider,tts_provider, and model ids before deploy. See Control plane API.
A project setting applies to new sessions after you Deploy to cloud. A live session can change STT, TTS, or the vendor without reconnecting: agent code calls setVoiceLanguage with sttVendor or ttsVendor. The API key has to already be a project secret on that deploy. See Switching language mid-conversation. Cloud speech from third-party vendors is billed by that vendor (not VoiceThere speech credits). VoiceThere STT/TTS is metered in VoiceThere usage credits.
Vendor setup guides
Each page below covers secrets, example model ids, what the cloud integration does, and known gaps for that provider. The complete allowlist of ids VoiceThere accepts is published in @node-webrtc-rust/voice-catalog and mirrored in dashboard Voice settings.
- VoiceThere (the built-in VoiceThere speech)
- OpenAI
- Deepgram
- ElevenLabs
- Cartesia
- Fish Audio
- AssemblyAI
- Google Cloud Speech
- Groq
- Azure Speech
- AWS Transcribe & Polly
Streaming matrix (high level)
Partial transcripts and progressive TTS depend on the model and provider. Not every row streams live partials on both legs.
| Provider | STT | TTS | Notes |
|---|---|---|---|
| voicethere | Streaming, built in | Streaming, built in | No API keys; metered on VoiceThere. The older ids local-sherpa and cluster-sherpa are deprecated aliases. |
| openai | File/SSE or Realtime WS by model | HTTP or SSE stream | gpt-live-transcribe uses Realtime WS. |
| deepgram | Listen WS (nova-2 / nova-3) | Aura / Flux speak | Flux is not a listen model. |
| elevenlabs | Scribe v2 Realtime | HTTP / WS stream | eleven_v3 skips WS stream TTS. |
| V2 stream or batch REST | Neural2 batch; Chirp3-HD streams | Chirp STT uses V2 streaming when configured. | |
| assemblyai | Live WS | n/a | STT-only. |
| cartesia | n/a | WS TTS | TTS-only. |
| fishaudio | Realtime socket (finals only) | HTTP stream (Drama 3) | No STT partials; barge-in uses voice activity. |
| groq | File upload (no live partials) | HTTP Orpheus | Whisper file models only for STT. |
| azure | REST conversation (finals) | Neural REST | Key, region, and STT resource hostname required. |
| aws | Transcribe streaming | Polly neural | IAM access key, secret, and region. |
See also
- Spoken language identification: VoiceThere catalog ids for
setVoiceLanguage - Advanced voice settings: VAD, barge-in, and speech event tuning
- Billing & usage credits: VoiceThere speech metering