Realtime vs Cascade
Two architectures for live interpretation — a direct voice-to-voice model, and a composable STT → translation → TTS pipeline — measured on the same real medical audio. 8 language directions, 24 clips each, 8 configurations, 61 runs.
The trade-off
At a glance
Realtime against three representative cascades. Quality and latency are medians across the 8 directions; cost is the metered total for all of that configuration's runs.
COMETKiwi (reference-free, higher is better), the all-OpenAI cascade vs Realtime, grouped by language pair. The two architectures track each other everywhere except Vietnamese→English — the cascade's one collapse. Realtime was faster in all 8 directions.
All 8 configurations
| Configuration | Quality (neutral) | Typical | Worst case | Cost |
|---|---|---|---|---|
| End-to-end speech | ||||
| OpenAI Realtime | 0.686 | 1420 | 1727 | $13.14 |
| Cascades (STT / translation / TTS) | ||||
| openai / openai / openai | 0.683 | 2254 | 3576 | $0.65 |
| azure-speech / openai / azure-speech | 0.682 | 2421 | 3403 | $0.55 |
| openai / azure / openai | 0.682 | 1786 | 2683 | $0.64 |
| openai / gemini / openai | 0.680 | 4378 | 9346 | $0.62 |
| openai / claude / openai | 0.644 | 2583 | 3852 | $0.77 |
| deepgram / openai / openai | 0.641 | 3508 | 24081 | $0.55 |
| deepgram / openai / elevenlabs | 0.602 | 7670 | 12768 | $2.96 |
Median across the 8 directions; latency in milliseconds; cost is the metered total for all of that configuration's runs, priced at each provider's own list rate. Quality is COMETKiwi, which scores translations without comparing them to a reference — see below for why that matters. OpenAI Realtime is a single voice-to-voice model, not a pipeline, which is why it sits apart from the cascades.
All 8 directions (Realtime vs the all-OpenAI cascade)
| Pair | Direction | Neutral (COMETKiwi) | Overlap (chrF) | Judge (GEMBA) | Typical | Worst case |
|---|---|---|---|---|---|---|
| en ↔ fr | en→fr | 0.685 → 0.695 | 0.738 → 0.677 | 74.3 → 78.1 | 2512 → 1374 | 3858 → 1780 |
| fr→en | 0.640 → 0.615 | 0.651 → 0.550 | 71.6 → 67.6 | 2256 → 1367 | 7114 → 1730 | |
| en ↔ de | en→de | 0.687 → 0.679 | 0.694 → 0.639 | 76.9 → 78.7 | 2270 → 1347 | 3301 → 1496 |
| de→en | 0.682 → 0.712 | 0.709 → 0.657 | 74.6 → 82.7 | 1885 → 1534 | 2976 → 2150 | |
| en ↔ zh | en→zh | 0.715 → 0.683 | 0.411 → 0.340 | 76.9 → 75.1 | 2229 → 1345 | 3706 → 1551 |
| zh→en | 0.719 → 0.712 | 0.611 → 0.545 | 73.8 → 75.4 | 2252 → 1481 | 3446 → 1669 | |
| en ↔ vi | en→vi | 0.679 → 0.689 | 0.710 → 0.657 | 71.4 → 73.3 | 2626 → 1466 | 4825 → 1724 |
| vi→en | 0.600 → 0.648 | 0.438 → 0.448 | 39.9 → 60.0 | 1970 → 1487 | 2596 → 1947 |
Rows are grouped by language pair, each with both directions. Each cell reads cascade → Realtime; latency in milliseconds. COMETKiwi never sees the reference translations, so it is the metric to trust when the three disagree. Judge scores re-ran ±1–2 points on identical input, so gaps under ~2 are not real. Overlap scores are not comparable across writing systems — the low Chinese numbers reflect character density, not worse translation. Worst-case latency is the second-slowest of 24 clips, so one slow utterance moves it by seconds; treat individual rows as indicative and the ranges as the finding.
What we learned
Which providers you pick matters more than which architecture
The five best setups — Realtime and four different cascades — land within 0.006 of each other on the neutral quality metric. But swapping providers inside the cascade moves quality by 0.081, more than ten times that. The architecture question is close to a tie; the provider question is not. "Cascades are worse" is really a claim about one particular stack.
Realtime is faster — and much more consistent
It wins on speed in all 8 directions, but consistency is the real story. Realtime's worst case stayed inside 1.5–2.2s everywhere; the cascade's ranged from 2.6s to 7.1s. Re-running the same setup moves any single worst-case figure by up to 2s, so don't read one row closely — the pattern is that Realtime's tail is tight and the cascade's is erratic. For live interpretation, predictability may matter more than the average.
On quality they are a tie — and getting there took three metrics
Word-overlap scoring picks the cascade in 7 of 8 directions. The LLM judge picks Realtime. Both are biased: our reference translations were machine-made then human-corrected, so they reward the cascade's literal phrasing, while an LLM judge tends to favour natural-sounding output. A third metric scores translations without looking at the references at all — and it puts the two 0.003 apart, 4 directions each. Quality is a wash. Never rank architectures on a single metric.
Realtime pulls ahead where transcription is hardest
Vietnamese→English is the one direction all three metrics agree on: the cascade's worst result and Realtime's best relative showing — 40 → 60 on the judge, and Realtime wins the neutral metric too. In a cascade, a transcription error is faithfully translated and spoken. End-to-end speech has no such handoff to corrupt. (French→English is the mirror image, favouring the cascade on all three.)
Cost is the cascade's moat — by more than expected
The same eight directions cost $13.14 through Realtime and $0.55–0.77 through most cascades — about 20×. Both figures are metered from actual usage, not estimated. At platform volume that is the difference between a premium feature and a default tier. The exception is ElevenLabs speech at $2.96, roughly 5× the other cascades.
The cheapest voice was also the one humans preferred
Azure Speech is the cheapest text-to-speech we tested — about $0.015 per 1,000 characters, the bottom of the price range and roughly 7× below ElevenLabs (~$0.11) — and its cascade tied for the cheapest configuration we metered at $0.55, against a Deepgram stack that scored 0.041 lower. Price didn't cost quality: in human listening review, Azure's voice output sounded the best of the TTS options, and its neutral-metric score (0.682) is statistically tied with the leaders. One boundary: keep it inside its supported language list — pointed at German, which it doesn't list, it produced unusable audio.
The stack we assumed was the default measured worst
Deepgram + ElevenLabs — the pipeline this project originally described as the cascade — came last on quality and cost the most of any cascade. Two independent swaps explain it: using Deepgram instead of OpenAI for speech recognition costs real quality on continuous speech, and ElevenLabs' voice output scored lower than OpenAI's when transcribed back. Both are one-line configuration changes.