VoiceThere

STT & TTS vendors

VoiceThere cloud runners synthesize and transcribe speech using providers you pick per project. Configure STT and TTS independently — for example, Deepgram STT with OpenAI TTS, or on-cluster Sherpa for both with no API keys.

Where to configure

  • Dashboard: Project overview → Voice (STT / TTS) — choose providers, models, and voices. Third-party keys go in project secrets (see Agent environment & secrets).
  • Control plane API: Patch project voice settings (stt_provider, tts_provider, model ids) before deploy — see Control plane API.

Changes apply to new sessions after you Deploy to cloud on the project overview. Cloud speech from third-party vendors is billed by that vendor; on-cluster Sherpa STT/TTS usage is metered in VoiceThere usage credits.

STT providers

Sherpa (on-cluster) (local-sherpa)

Uses preloaded Sherpa models on the runner volume — no API keys.

Required project secrets

None

Optional env fields

None

OpenAI (openai)

Whisper STT and OpenAI TTS share OPENAI_API_KEY.

Defaults: model: whisper-1, language: en

STT models

  • whisper-1 — Whisper v1
  • gpt-4o-mini-transcribe — GPT-4o mini transcribe

Required project secrets

  • AGENT_OPENAI_API_KEY — OpenAI API key

Optional env fields

None

Deepgram (deepgram)

Deepgram is STT-only in the SDK — pair with an OpenAI (or other) TTS provider.

Defaults: model: nova-2, language: en

STT models

  • nova-2 — Nova 2
  • nova-3 — Nova 3

Required project secrets

  • AGENT_DEEPGRAM_API_KEY — Deepgram API key

Optional env fields

None

AssemblyAI (assemblyai)

AssemblyAI is STT-only — pair with a TTS provider such as OpenAI.

Defaults: model: universal-streaming-english, language: en

STT models

  • universal-streaming-english — Universal streaming (English)

Required project secrets

  • AGENT_ASSEMBLYAI_API_KEY — AssemblyAI API key

Optional env fields

None

Google Cloud Speech (google)

Uses Application Default Credentials or an API key. Store credentials as encrypted project secrets — they are mapped to the runner at deploy.

Defaults: model: latest_long, language: en-US

STT models

  • latest_long — Latest long

Required project secrets

  • AGENT_GOOGLE_APPLICATION_CREDENTIALS — Service account credentials
  • AGENT_GOOGLE_API_KEY — Google API key

Optional env fields

None

TTS providers

Sherpa (on-cluster) (local-sherpa)

Uses preloaded Sherpa models on the runner volume — no API keys.

Required project secrets

None

Optional env fields

None

OpenAI (openai)

Whisper STT and OpenAI TTS share OPENAI_API_KEY.

Defaults: model: tts-1, voice: alloy

TTS models

  • tts-1 — TTS standard
  • tts-1-hd — TTS HD

TTS voices

  • alloy — Alloy
  • echo — Echo
  • fable — Fable
  • onyx — Onyx
  • nova — Nova
  • shimmer — Shimmer

Required project secrets

  • AGENT_OPENAI_API_KEY — OpenAI API key

Optional env fields

None

ElevenLabs (elevenlabs)

ElevenLabs is TTS-only — pair with an OpenAI (or other) STT provider.

Defaults: model: eleven_multilingual_v2, voice: EXAVITQu4vr4xnSDxMaL

TTS models

  • eleven_multilingual_v2 — Multilingual v2

TTS voices

  • EXAVITQu4vr4xnSDxMaL — Rachel (default)

Required project secrets

  • AGENT_ELEVENLABS_API_KEY — ElevenLabs API key

Optional env fields

  • AGENT_ELEVENLABS_VOICE_ID — Voice id

Cartesia (cartesia)

Cartesia is TTS-only — pair with an OpenAI (or other) STT provider.

Defaults: model: sonic-english, voice: default

TTS models

  • sonic-english — Sonic English

TTS voices

  • default — Default

Required project secrets

  • AGENT_CARTESIA_API_KEY — Cartesia API key

Optional env fields

  • AGENT_CARTESIA_VOICE_ID — Voice id

Google Cloud Speech (google)

Uses Application Default Credentials or an API key. Store credentials as encrypted project secrets — they are mapped to the runner at deploy.

Defaults: model: en-US-Neural2-A, voice: en-US-Neural2-A

TTS models

  • en-US-Neural2-A — en-US Neural2 A

TTS voices

  • en-US-Neural2-A — en-US Neural2 A

Required project secrets

  • AGENT_GOOGLE_APPLICATION_CREDENTIALS — Service account credentials
  • AGENT_GOOGLE_API_KEY — Google API key

Optional env fields

None

Sherpa STT models (on-cluster)

When stt_provider is local-sherpa, the runner loads the selected bundle from the preloaded Sherpa volume. Models below are the full catalog — pick one by stt_model_id in project voice settings.

IdLabelDescription
enEnglish (Kroko)Streaming Zipformer transducer (transducer) for English. English (Kroko)
en-legacyEnglish (2023 baseline)Streaming Zipformer transducer (transducer) for English. English (2023 baseline)
en-smallEnglish (20M, small)Streaming Zipformer transducer (transducer) for English. English (20M, small)
esSpanishStreaming Zipformer transducer (transducer) for Spanish. Spanish
frFrenchStreaming Zipformer transducer (transducer) for French. French
deGermanStreaming Zipformer transducer (transducer) for German. German
zhChinese (Mandarin)Streaming Zipformer transducer (transducer) for Chinese. Chinese (Mandarin)
zh-enChinese + English (bilingual)Streaming Zipformer transducer (transducer) for Chinese. Chinese + English (bilingual)
jaJapaneseStreaming Zipformer transducer (transducer) for Japanese. Japanese Shares the same on-cluster bundle as catalog ids: ar, id, ja, th, vi.
arArabicStreaming Zipformer transducer (transducer) for Arabic. Arabic Shares the same on-cluster bundle as catalog ids: ar, id, ja, th, vi.
idIndonesianStreaming Zipformer transducer (transducer) for Indonesian. Indonesian Shares the same on-cluster bundle as catalog ids: ar, id, ja, th, vi.
thThaiStreaming Zipformer transducer (transducer) for Thai. Thai Shares the same on-cluster bundle as catalog ids: ar, id, ja, th, vi.
viVietnameseStreaming Zipformer transducer (transducer) for Vietnamese. Vietnamese Shares the same on-cluster bundle as catalog ids: ar, id, ja, th, vi.
koKoreanStreaming Zipformer transducer (transducer) for Korean. Korean
ruRussianStreaming Zipformer transducer (transducer) for Russian. Russian
fr-legacyFrench (2023 baseline)Streaming Zipformer transducer (transducer) for French. French (2023 baseline)
en-2023-02English (2023-02 LibriSpeech)Streaming Zipformer transducer (transducer) for English. English (2023-02 LibriSpeech)
bnBengali (South Asia)Streaming Zipformer transducer (transducer) for Bengali. Bengali (South Asia)

Sherpa TTS models (on-cluster)

When tts_provider is local-sherpa, synthesis uses on-cluster neural TTS voices (mostly Piper VITS; Chinese uses Melo). Pick one by tts_model_id.

IdLabelDescription
enEnglish (Amy, Piper medium)Piper VITS neural TTS (vits) for English — English (Amy, Piper medium).
en-amy-lowEnglish (Amy, Piper low)Piper VITS neural TTS (vits) for English — English (Amy, Piper low).
en-lessacEnglish (Lessac, Piper medium)Piper VITS neural TTS (vits) for English — English (Lessac, Piper medium).
en-lessac-highEnglish (Lessac, Piper high)Piper VITS neural TTS (vits) for English — English (Lessac, Piper high).
en-lessac-lowEnglish (Lessac, Piper low)Piper VITS neural TTS (vits) for English — English (Lessac, Piper low).
en-gladosEnglish (GLaDOS)Piper VITS neural TTS (vits) for English — English (GLaDOS).
en-glados-highEnglish (GLaDOS high)Piper VITS neural TTS (vits) for English — English (GLaDOS high).
en-ryanEnglish (Ryan, Piper medium)Piper VITS neural TTS (vits) for English — English (Ryan, Piper medium).
en-ryan-highEnglish (Ryan, Piper high)Piper VITS neural TTS (vits) for English — English (Ryan, Piper high).
en-ryan-lowEnglish (Ryan, Piper low)Piper VITS neural TTS (vits) for English — English (Ryan, Piper low).
en-joeEnglish (Joe, Piper medium)Piper VITS neural TTS (vits) for English — English (Joe, Piper medium).
en-bryceEnglish (Bryce, Piper medium)Piper VITS neural TTS (vits) for English — English (Bryce, Piper medium).
en-johnEnglish (John, Piper medium)Piper VITS neural TTS (vits) for English — English (John, Piper medium).
en-kristinEnglish (Kristin, Piper medium)Piper VITS neural TTS (vits) for English — English (Kristin, Piper medium).
en-normanEnglish (Norman, Piper medium)Piper VITS neural TTS (vits) for English — English (Norman, Piper medium).
en-danny-lowEnglish (Danny, Piper low)Piper VITS neural TTS (vits) for English — English (Danny, Piper low).
en-kathleen-lowEnglish (Kathleen, Piper low)Piper VITS neural TTS (vits) for English — English (Kathleen, Piper low).
en-ljspeechEnglish (LJSpeech, Piper medium)Piper VITS neural TTS (vits) for English — English (LJSpeech, Piper medium).
en-ljspeech-highEnglish (LJSpeech, Piper high)Piper VITS neural TTS (vits) for English — English (LJSpeech, Piper high).
en-arcticEnglish (Arctic multi-speaker, Piper medium)Piper VITS neural TTS (vits) for English — English (Arctic multi-speaker, Piper medium).
en-hfc-femaleEnglish (HFC female, Piper medium)Piper VITS neural TTS (vits) for English — English (HFC female, Piper medium).
en-hfc-maleEnglish (HFC male, Piper medium)Piper VITS neural TTS (vits) for English — English (HFC male, Piper medium).
en-gb-alanEnglish GB (Alan, Piper medium)Piper VITS neural TTS (vits) for English — English GB (Alan, Piper medium).
en-gb-coriEnglish GB (Cori, Piper medium)Piper VITS neural TTS (vits) for English — English GB (Cori, Piper medium).
en-gb-jennyEnglish GB (Jenny, Piper medium)Piper VITS neural TTS (vits) for English — English GB (Jenny, Piper medium).
esSpanish (GLaDOS medium)Piper VITS neural TTS (vits) for Spanish — Spanish (GLaDOS medium).
deGerman (Thorsten medium)Piper VITS neural TTS (vits) for German — German (Thorsten medium).
de-thorsten-highGerman (Thorsten high)Piper VITS neural TTS (vits) for German — German (Thorsten high).
de-gladosGerman (GLaDOS medium)Piper VITS neural TTS (vits) for German — German (GLaDOS medium).
frFrench (Siwis, Piper medium)Piper VITS neural TTS (vits) for French — French (Siwis, Piper medium).
itItalian (Paola, Piper medium)Piper VITS neural TTS (vits) for Italian — Italian (Paola, Piper medium).
ptPortuguese BR (Faber, Piper medium)Piper VITS neural TTS (vits) for Portuguese — Portuguese BR (Faber, Piper medium).
nlDutch (Pim, Piper medium)Piper VITS neural TTS (vits) for Dutch — Dutch (Pim, Piper medium).
plPolish (Gosia, Piper medium)Piper VITS neural TTS (vits) for Polish — Polish (Gosia, Piper medium).
ruRussian (Irina, Piper medium)Piper VITS neural TTS (vits) for Russian — Russian (Irina, Piper medium).
hiHindi (Priyamvada, Piper medium)Piper VITS neural TTS (vits) for Hindi — Hindi (Priyamvada, Piper medium).
zhChinese + English (Melo TTS)Melo TTS (vits) for Chinese — Chinese + English (Melo TTS).

See also

← All documentation