[ Own benchmark · reproducible · 2026 ]

Real latency of voicebots vs chatbots
in enterprise.

Controlled end-to-end latency measurement in real enterprise voicebot pipelines (ASR + LLM + TTS) vs equivalent text chatbots. With ceiling above which conversation feels "human" (200 ms) and below which user hangs up (>1,500 ms).

Executed · June 2026 · Datalvar AI Updated · August 2026 Sample · 600 real conversations across 4 industries License · open data and methodology
[ Executive summary ]

TL;DR — 4 conclusions.

  1. Voicebot in production needs < 800 ms end-to-end. Above 1,200 ms user perceives "robotic pause" and hangs up 3x more. Below 500 ms feels natural.
  2. The LLM is not the main bottleneck. In our benchmark, LLM represents 25-35 % of total latency. TTS (35-45 %) and ASR (15-25 %) matter more than everyone thinks.
  3. Real-time streaming architecture is essential. Non-streaming pipelines (waiting for complete response before generating audio) add 400-800 ms unnecessary. Streaming reduces perceived latency 50-70 %.
  4. Latency-optimised voicebot needs specific stack: Deepgram Nova-3 or AssemblyAI Universal-Streaming for ASR, small LLM tiered (Haiku 4.5 / GPT-5 mini) for 70 % of decisions, ElevenLabs Turbo v2.5 or Cartesia Sonic for TTS. Combination reaches 350-500 ms real end-to-end.
[ Methodology ]

How we measured latency.

Test scenarios

  • V1 · Clinic scheduling — 150 conversations. Voicebot receives call, verifies patient DB, offers slots, confirms.
  • V2 · SaaS first-level support — 150 conversations. Voicebot answers queries, queries mock CRM, resolves or escalates.
  • V3 · Post-visit workshop follow-up — 150 conversations. Voicebot calls, asks 4 questions, records structured response.
  • V4 · Insurance qualification — 150 conversations. Voicebot qualifies inbound lead with 6 questions, calculates initial quote.

Stacks compared

  • Stack A · "Latency-first": Deepgram Nova-3 + Claude Haiku 4.5 + ElevenLabs Turbo v2.5. Streaming end-to-end.
  • Stack B · "Quality-first": AssemblyAI Universal + Claude Opus 5 + ElevenLabs Multilingual v2. Streaming end-to-end.
  • Stack C · "Enterprise all-in-one": Azure Speech + GPT-5 + Azure Neural TTS. Streaming end-to-end.
  • Stack D · "Cost-optimised": Whisper large-v3 self-hosted + Llama 3.3 70B + Kokoro TTS. Streaming end-to-end.
  • Stack E · Non-streaming (control): same as Stack A but WITHOUT streaming (wait for complete LLM response before audio generation).

Metrics per turn

  • TTFB (Time To First Byte) — from user finishing speaking → to first audio byte generated
  • End-to-end — from user finishing → to first audio byte user hears (includes network)
  • Breakdown: ASR (final), LLM (first token), TTS (first byte)
  • Complete duration — total system response time

Test conditions

  • Infrastructure: AWS Frankfurt (typical Spanish enterprise)
  • Real internet latency between client (WebRTC endpoint) and server
  • User connection: 4G/5G simulated with 20-40 ms latency
  • Real speakers with different accents (Iberian, Latin American, mixed)
  • System prompt cached with Anthropic prompt caching where applicable
[ Results ]

End-to-end latency per stack.

Total latency (median across 4 scenarios)

StackASR (median)LLM (first token)TTS (first byte)Total E2EPerception
A · Latency-first (streaming)85 ms110 ms140 ms385 msNatural
B · Quality-first (streaming)150 ms210 ms240 ms720 msGood
C · Azure enterprise180 ms180 ms220 ms760 msGood
D · Cost-optimised (self-host)240 ms320 ms180 ms910 msAcceptable
E · Same as A but non-streaming85 ms620 ms *320 ms1,180 msRobotic

* In non-streaming pipeline, LLM latency includes waiting for complete response, not first token.

Real impact on business — hangup rate by latency (V4 scenario, 500 calls)

Median E2E latencyHangup < 30sFull completion ratePost-call NPS
< 500 ms (Stack A)4 %87 %+42
500-800 ms (Stack B/C)7 %82 %+31
800-1,200 ms (Stack D)13 %71 %+8
> 1,200 ms (Stack E non-streaming)28 %52 %−14

Practical conclusion: difference between 385 ms and 1,180 ms is difference between voicebot working in production and voicebot user hangs up on. Streaming architecture and stack choice matter more than LLM used.

Chatbot (text) vs voicebot latency

ChannelAcceptable latencyExcellent latencyPerceptibility threshold
Text chatbot< 3 s< 1 sAbove 5 s user leaves
Voicebot< 800 ms< 500 msAbove 1.2 s user hangs up
Human-human~200 ms~100 msAbove 500 ms feels forced

Text chatbots tolerate 5-10x more latency because user visually sees they're "thinking" (typing indicator, progressive response). Voice has no this signal.

[ Reference architecture ]

Recommended stack for production voicebot Spanish enterprise.

Based on results, this is the reference architecture we deploy in production clients when latency and cost matter:

LayerRecommendationAlternativeWhy
ASRDeepgram Nova-3AssemblyAI Universal-StreamingStreaming ASR < 100 ms with excellent quality in Spanish. Alternatives with similar quality but higher latency.
LLM (routine)Claude Haiku 4.5GPT-5 mini, Gemini 2.5 FlashFirst token < 150 ms + prompt cache active. Good enough for 70 % of decisions.
LLM (complex)Claude Sonnet 4.6GPT-5For decisions needing more depth, activated only when Haiku signals uncertainty.
TTSElevenLabs Turbo v2.5Cartesia Sonic, OpenAI TTS-HDStreaming TTS with first byte < 150 ms and quality indistinguishable from human in Spanish.
OrchestratorLiveKit AgentsPipecat, custom WebRTCReal-time WebRTC with streaming primitives ready. Alternatives require more custom implementation.
Business logicn8n or LangGraphCustom PythonTo handle backend logic (CRM, DB, escalation) outside voicebot layer.

Result: production voicebot with 350-500 ms median end-to-end latency, natural conversation, hangup rate < 5 %, operating cost €0.02-0.05 per minute of conversation. In Spanish enterprise scenarios like clinic scheduling, workshop follow-up, first-level support, this stack absorbs 60-80 % of calls with human quality.

Want this stack for your voicebot?

We measure real latency of your current pipeline, identify bottlenecks and design optimised architecture. Typical implementation: 6-10 weeks up to production.

Book diagnosis → See voice service