Engine & deployment

Models

Which speech and language models run a call, and how an agent's choices map to them.

A call uses three models: speech-to-text, a language model, and text-to-speech. Which ones run is decided in two places, and it is worth being exact about it.

Two layers

Layer	Decides
The agent	Stores a provider and model name for each stage, plus voice and language. The API accepts and returns these as sent.
The voice engine	Runs the platform's own models by default. For a stage whose provider it has a client for, and whose key the account has saved, it uses that provider for the call.

Providers the engine has a client for today: Deepgram and Sarvam for speech-to-text, ElevenLabs and Sarvam for the voice, and OpenAI, Groq, OpenRouter and Azure OpenAI for the language model. Each needs the account's key, added in Settings → Providers: DEEPGRAM_API_KEY, SARVAM_API_KEY, ELEVENLABS_API_KEY, OPENAI_API_KEY, GROQ_API_KEY, OPENROUTER_API_KEY, or AZURE_OPENAI_API_KEY with AZURE_OPENAI_ENDPOINT (the resource URL; the deployment is llm_config.deployment, else the agent's model, and the API version llm_config.api_version, else 2024-10-21). local is the platform's own models. openai runs on OpenAI when the account has saved OPENAI_API_KEY, and on the platform's own language model when it has not, which is the default for new agents and saves without a warning. The call log notes which was used, in a llm line starting with note:. The model and voice catalogues (GET /user/model/all, GET /api/v1/voice-config/tts*) mark every entry with implemented: false means the engine has no client for it yet. An agent that names such a provider, or one whose key is missing, is saved with an X-Agent-Warnings header, and its calls run on the platform's own models with a note: line in the call log.

Engine backends

Backend	Speech-to-text	Language model	Text-to-speech
`openai` (default)	An OpenAI-style transcription endpoint	An OpenAI-style chat completions endpoint, streamed, with tools	An OpenAI-style speech endpoint
In-house model host	A plain HTTP transcription endpoint	Use `openai` with the server's base URL	A streaming HTTP synthesis endpoint with named voices

"OpenAI-style" means the request and response format, not the company: any server that speaks that format can be used by setting its base URL. New backends can be added in code; the interfaces are stream-shaped.

The `local` provider

When the operator configures the platform's own model servers, they appear to agents as provider local:

in the voice catalog (GET /api/v1/voice-config/tts and /tts/voices), with the voices the text-to-speech server reports;
in the model list, as transcriber local and a language model local;
at zero cost in the price book and the price estimate. The platform fee and telephony are charged as usual.

Naming local for a stage that is not configured is refused with 400 when the agent is saved.

Custom language models

A workspace can register its own OpenAI-style endpoint as a custom model and select it on an agent (/user/model/custom). The engine calls that base_url with the agent's key. The URL may not point at a private address.

What works without any model

Feature	With a model key	Without
Knowledge-base search	Real embeddings	A hashing stand-in: works, but matches words, not meaning
Post-call extraction	One model pass per category	Rule-based
Test chat	The agent's own model	Scripted replies
Mock engine calls	Not needed	Scripted conversation

That is what makes it possible to build and test end to end before any model is connected.

Languages

The platform does not fix a list of languages. What a call can understand and speak is whatever the connected speech models support. The in-house models were exercised in Hindi and English, and the engine's built-in spoken lines (fallback, handover) exist in those two languages.

Limitation: With the in-house speech-to-text, English spoken in a Hindi call can come back transliterated. The engine adds a note to the model's prompt to read such transcripts correctly; short fragments can still be misheard.

Concurrency

The engine caps requests in flight per model server (by default 4 for speech-to-text, 4 for the language model, 2 for text-to-speech). Under load, waiting for a slot adds to the response time and is reported in the latency record of the turn.

Voices and attribution

The platform's own Hindi voices are listed with neutral ids: F1–F9 (female) and M1–M8 (male). F3 is the recommended default for new agents. They are built from speakers in the open IndicVoices-R Hindi corpus by AI4Bharat, licensed CC BY 4.0; the voices are adapted from that material. No voice is named after, or presented as, a real person.