What speech to text does
Speech to text (STT) turns the caller's audio into text the LLM can act on. Every turn of a call runs through it. If STT hears the wrong thing, the LLM answers the wrong question, the workflow branches wrong, and post-call analysis extracts wrong values. Most "the Finn didn't follow my script" reports trace back to a transcription miss, not a prompting problem.
STT is configured on the Finn, in the dashboard. There is no separate STT endpoint. See agents-identity for where these fields live in the Create Finn wizard.
Fields that control STT
| Field | Tab | What it does |
|---|---|---|
| Language | Identity | Primary recognition language. Also drives transcript output and filters the voice library. |
| Language Group | Identity | Voice/STT optimization pack. Defaults to global; regional packs improve accent handling for specific markets. |
| Custom Vocabulary | Behavior | List of words or phrases the recognizer should expect. |
| STT service | AI service configuration | Which speech-to-text engine runs. Default works for most languages. |
Language is the field that matters most. Pick what your callers actually speak, not what your website is in. Setting a Finn to English and pointing it at Tamil-speaking callers does not degrade gracefully — it produces confident nonsense transcripts rather than empty ones.
Language groups
Language Group sits under Identity, next to Language. It selects an optimization pack for voice and recognition. Most accounts leave it on global, which is the correct default when your callers are spread across regions or you do not know their accent profile.
Switch to a regional pack when the bulk of your calls come from one market and you are seeing accent-related errors. The regional pack narrows what the recognizer expects, which raises accuracy inside that market and can lower it for callers outside it. That trade is the whole point of the setting, so do not set a regional pack "just in case."
Accuracy by language tier
STT quality is not uniform across the 46 supported languages. Tiers are graded on voice naturalness, STT accuracy, and regional accent coverage together.
| Tier | STT behavior | Round-trip latency |
|---|---|---|
| Tier 1 | Handles accents, code-switching and ambient noise well | Under 800ms |
| Tier 2 | Works in quiet environments, may struggle with heavy accents | 800-1000ms |
| Tier 3 | Requires good audio quality | 1000-1300ms |
Above roughly 1.5s the conversation stops feeling natural. Tier membership per language is listed in agents-multilingual.
For high-stakes use cases (sales, healthcare, finance), stay on Tier 1. Tier 2 and Tier 3 are acceptable for surveys, notifications, and internal tooling where a misheard word costs little.
Accents
Tier 1 English is robust to most accents, including Indian English and strong regional British accents. It is not immune. Heavy regional accents produce higher error rates, and there is no dashboard readout that tells you your error rate for a given accent group.
The only way to know is to test. Run test calls with speakers who match your real caller base, then read the transcripts in call-logs and compare against what was actually said. Do this before launching, not after.
Things that make accent handling worse, in rough order of impact:
- Poor audio (speakerphone, hands-free car kits, noisy call centers)
- A regional Language Group set to the wrong region
- Short single-word answers with no surrounding context to disambiguate
- Domain terms that are not in Custom Vocabulary
Custom vocabulary
Custom Vocabulary is the highest-leverage STT fix available to you. It tells the recognizer which unusual strings to expect, so brand names and SKUs stop being transcribed as the nearest common word.
Add brand names, product SKUs, drug names, regional slang, and common caller name pronunciations:
["Acme", "FinnAI", "Ozempic", "DELT-450", "Doctor Patel"]
It accepts words in any language and works across languages, which matters when callers code-switch. An English Finn serving Indian customers might carry:
["Ola", "Zomato", "Razorpay", "Aadhaar", "PAN card", "GST"]
Full guidance in agents-vocabulary.
Code-switching
Callers in many regions mix languages inside one sentence — Hindi-English, Spanish-English, French-Arabic. Tier 1 recognition follows this without special configuration. To stop the Finn from responding in a single fixed language, add to Style & Guardrails:
Match the caller's speaking style — if they mix English and Hindi, you can mix too. Don't correct or comment on their language choice.
Tier 2 and Tier 3 code-switching is not reliable. If your callers code-switch heavily and one of the languages is Tier 3, expect transcript corruption in the mixed spans.
What goes wrong
| Symptom | Likely cause | Fix |
|---|---|---|
| Finn answers a question the caller did not ask | Transcription miss upstream | Read the transcript in call logs, add the misheard terms to Custom Vocabulary |
| Brand or product name mangled every call | Not in Custom Vocabulary | Add it, including likely alternate spellings |
| Finn asks the same question twice | Caller's answer did not parse into the expected format | Soften the Ask node parser or rephrase the question. See agents-conversation |
| Accuracy dropped after a config change | Language Group switched to a regional pack that does not match callers | Revert to global |
| Post-call analysis fields blank or wrong | Transcript too short or too corrupted to extract from | Check transcript length first. See post-call-analysis |
| Long silences before the Finn responds | Tier 2/3 latency, or idle timing | Check the tier table above, then tune max_idle_duration |
Related
- agents-multilingual — language list, tiers, and multilingual modes
- agents-vocabulary — custom vocabulary in depth
- agents-tts — the output half of the audio path
- agents-testing — test calls before you launch
- troubleshooting — broader symptom index