When we started Convey, we tested pretty much every commercial text-to-speech engine we could find. AWS Polly, Google, ElevenLabs, Azure, Resemble — the usual list. For English, all of them were good. For Hindi they were passable. For Telugu and Tamil, sometimes intelligible. For Odia, Assamese, or code-mixed Hinglish, they fell apart.
We needed a TTS that worked equally well across 22+ Indian languages, fast enough for a real-time voice agent on a phone call. So we built one. This is the post about what we learned along the way.
What "good" actually means for Indic TTS
Three things have to be true at once, and most engines manage one or two but not all three.
The first is pronunciation. Proper nouns, place names, brand names — they all have to come out right. "Vijetha" is not "Vai-jay-tha." A buyer who hears their own city pronounced wrong knows immediately something is off, even if they can't articulate what.
The second is prosody. Sentence rhythm and emphasis have to match how a native speaker would actually say it. A TTS that reads "Sir, can I help you book a site visit?" with falling intonation in Telugu sounds robotic. Rising intonation on the right syllable makes the same sentence feel human.
The third is code-mixing. Indians switch languages mid-sentence constantly. "Aap ka loan EMI 5000 rupees per month hai." A TTS that pronounces "EMI" letter by letter as if it were a Hindi word breaks the illusion. So does one that switches to a completely different voice halfway through the sentence.
Why off-the-shelf engines struggle
Most commercial TTS engines were trained on English corpora, with other languages bolted on. Training data for non-Hindi Indian languages is small, voice talent is limited, and the prosody models inherit English assumptions about where the stress falls in a word.
The result is technically intelligible but emotionally flat. A real estate buyer hearing that voice on the phone figures out it's a recording within three seconds and hangs up. We watched it happen on hundreds of calls before we decided to build our own.
What we built
IRA is a streaming neural TTS, and the architecture is shaped by what we needed it to do on a live call.
We went with a unified text-to-audio model instead of the more traditional cascade (text → phonemes → mel-spectrogram → audio). Cascaded systems accumulate error at every stage and run slower. A unified end-to-end model is smaller, faster, and captures cross-language prosody better because it can learn shared patterns across languages.
The model streams in 80-millisecond chunks. That sounds technical but the effect is human — the buyer hears the start of the answer before the model has even finished generating the full sentence. That's what makes sub-200ms latency possible.
One model handles all the languages. Hindi, Telugu, Tamil, Kannada, and 18 more, in a shared phonetic space. Code-mixing works because the model isn't switching engines — it's the same engine, fluent in all of them.
And SSML works. Marketing teams can tweak emphasis on specific words, slow down disclaimers, add pauses. Same SSML standard everyone else uses. No new format to learn.
The hard part wasn't the model
The hardest part of building Indic TTS turned out to be data, not architecture.
For each language we needed roughly 30 hours of clean, single-speaker, neutrally-toned audio with verified transcripts. For Hindi, this kind of dataset exists in the public domain. For Odia and Assamese, it does not. So we recorded our own — native speakers, in a studio, with linguistic supervision to catch transcription errors before they ended up baked into the model.
Recording one language takes about six weeks. We've done it 22 times.
Latency under load is what actually matters
Anyone can quote a latency number on a benchmark page. The question is, latency under what load?
Cold start matters. A TTS API that takes 800 milliseconds to warm up is unusable for a voice agent. The first thing the buyer hears can't be silence.
P99 matters even more than the median. A 200ms median is great until you're on a busy call day and the 99th-percentile request takes 2 seconds because something is queued. A flat 300ms is a better experience.
And concurrency matters. Can you serve 500 simultaneous streams? At what cost? We benchmark IRA at 180ms P50, 240ms P99, with 800 concurrent streams per GPU. Those are the numbers that hold up when a voice agent is on a live call with a buyer at 9 PM on a Friday.
Things we got wrong on the way
A few mistakes worth flagging, if you're considering building your own.
We assumed code-mixed prosody would emerge from training on monolingual and bilingual data. It didn't. We had to explicitly collect Hinglish recordings before the model handled it cleanly.
We over-engineered emotion early on. Real estate buyers don't want a "happy" voice. They want a confident, neutral one. Six months of work on emotion vectors got quietly shelved.
We launched with 8 languages and discovered customers wanted all 22 on day one. Coverage turns out to be binary in this market — there's no minimum-viable language set. A customer in Hyderabad needs Telugu and Tamil and Hindi and Kannada and Malayalam, not "the most important three."
Where it fits
IRA powers ENGAGE voice calls. It's also available as a standalone API for marketing teams who just want studio-quality voiceovers for social ads, IVR menus, and explainers in Indian languages.
Send us text. Get back audio. 22+ languages. 200 milliseconds. That's all of it.