Best Low Latency TTS APIs for Real-Time Voice Apps 2026

Vendor latency benchmarks are measured under conditions your production stack will never see. Here is why those numbers lie, and what to measure instead.

The common assumption among engineering teams evaluating voice AI vendors is that published TTFA and P99 numbers from TTS API vendors accurately predict how that API will perform in a live voice application. That assumption is wrong, and the distinction matters enormously when you're the engineer who signed off on the integration and now has to explain to your team why calls that felt snappy in testing sound like a bad phone connection at scale. Platforms built for production voice workloads, like Bland's AI phone infrastructure, are designed around this gap, but understanding why the gap exists is the first step to avoiding it.

According to industry research, vendor-published TTFA figures are typically measured on a single, warm, isolated request with no concurrent load, no queuing effects, and no telephony routing in the path. That is not a benchmark. It is a best-case photograph taken under studio lighting.

The difference between sandbox latency and production latency is routinely 3 to 5 times larger at scale, so a TTS API that posts 200ms TTFA in documentation can exhibit 600ms to 1,000ms in a live deployment. A single warm request hits a ready server with no competition for resources.

Production is the opposite: dozens or hundreds of simultaneous sessions competing for GPU cycles, network bandwidth, and inference queue slots. Under that pressure, P99 latency diverges dramatically from P50.

Developers who track only average response time miss the tail entirely, and tail latency is what callers actually hear. Raw TTS API response time and the latency a caller experiences are two different numbers. SIP signaling, PSTN routing, and WebRTC negotiation each add overhead before a single audio byte reaches the listener. None of that appears in a vendor's published benchmark. Industry analyses of production voice deployments consistently confirm that telephony overhead alone adds 100ms to 300ms on top of any synthesis figure a vendor reports in isolation.

Key takeaways#

• Published TTFA and P99 numbers from TTS vendors are measured on single warm requests against idle servers, they tell you nothing about what happens at 500 concurrent calls under real telephony routing overhead.
• The latency number that matters in production is end-to-end, sub-400ms, sustained under load, not the sandbox figure that made it into the vendor's marketing page.
• Most developers don't buy a voice model; they buy a fragile stack of third-party dependencies that compounds latency at every hop, inference, routing, audio encoding, and compliance each add their own tax.
• Use case matters: outbound call campaigns, 24/7 inbound coverage, and conversational agents each hit different failure modes under load, and a benchmark that passes for one will quietly collapse for another.
• Audio output format is a latency variable, not a compatibility checkbox, the wrong choice multiplies across every concurrent session until your P99 tells a story your sandbox never predicted.
• Six months post-launch is the wrong time to discover your vendor's 80ms demo number had no concurrent sessions, no telephony overhead, and no compliance hops in the path.
• Bland.ai's Lowest Latency tier closes the gap with fine-tuned voice models on dedicated infrastructure, co-located GPUs, and a Voice Delivery Network that routes every call to the nearest server, keeping end-to-end latency consistent when volume scales and compliance constraints are real.

Latency Metrics and Benchmarks Explained - What TTFA, P99, and TTFB Actually Measure#

The common assumption among enterprise buyers and voice application developers is that published TTFA and P99 numbers from TTS API vendors accurately predict how that API will perform in a live voice application. Spend enough time reading TTS vendor documentation and you start to believe that latency is a solved problem. The numbers look precise.

The methodology sounds rigorous. Then you ship to production, real concurrent calls start hitting the stack, and the headline figure you built your evaluation around turns out to describe a condition that will never exist in your environment. This is one of the most consistent friction points for teams building AI voice agents at scale, and it is why claimed pipeline latency figures deserve hard scrutiny before they earn a contract.

Our own research found that Bland Speech v3 generates audio through a single endpoint returning PCM16 WAV at 44.1 kHz, with streaming support over both HTTP chunked transfer and WebSocket (our data).

What TTFA Actually Measures#

Time to first audio measures one thing: how long after a request is submitted before the first audio sample arrives at the client. That sounds definitive. It isn't. Vendors control where the clock starts (after connection is already established, on a warm server, with a short input sentence) and where it stops (first byte of audio, not first intelligible word). A benchmark run against a pre-warmed model with a single concurrent request on a co-located test client will post numbers that look nothing like a cold-start request arriving during a traffic spike.

The more precise question, and the one that governs user experience, is whether the published figure accounts for the full turn detection cycle: from the end of user speech all the way through to the first audio output. That end-to-end window is consistently longer than any isolated TTFA figure suggests, because it includes turn-detection processing, LLM inference, and TTS synthesis before the first audio frame is ever queued. Developers building voice agents on top of bare TTS APIs discover this gap only after they are in production, usually because FCR and CSAT metrics start signaling that something is wrong with the conversation rhythm before they can localize the cause. The best AI phone agent platform for enterprises addresses this directly by treating the full pipeline, not any single API hop, as the unit of measurement.

TTFB Is a Network Metric, Not a Voice Metric#

TTFB is a network metric. It tells you when the first byte of a response left the server and reached the client. For a REST TTS endpoint, that byte might represent a buffered audio payload that still needs to fully transfer before telephony playback can begin, adding meaningful overhead on top of the reported figure. Vendors surface TTFB because it is the smallest defensible number in the latency chain. It is not the number your users experience.

The gap widens further under real PSTN call conditions. Advertised latency figures are typically benchmarked over clean HTTP/2 or WebSocket connections in controlled environments. WebSocket architecture introduces its own framing, buffering, and reconnection considerations that REST-benchmark numbers do not capture. The moment audio has to traverse actual carrier infrastructure, codec transcoding, and network jitter, the TTFB figure becomes a lower bound that production traffic will reliably exceed. Teams that have already integrated with platforms like Amazon Connect understand this acutely: the AI voice layer has to perform inside an existing telephony stack, not beside it, and TTFB alone tells you nothing about whether it will.

P50 vs. P99 - The Median Hides Your Worst Calls#

The median call performing well is not your problem. Your problem is the call at the 99th percentile. In production voice traffic, the gap between P50 and P99 latency routinely represents a significant spread once concurrent sessions, geographic routing, and variable input length enter the picture.

Key takeaway: A common pattern among developers who have shipped voice agents at scale: the P50 number earns the vendor contract, and the P99 number ends the relationship.

This is where platform-level controls matter in ways that raw API benchmarks cannot show. When handling up to 100 concurrent calls and 5,000 calls per day, P99 degradation is not an occasional edge case; it is a predictable consequence of concurrency that must be engineered against. Tracking and improving key performance metrics like FCR, AHT, and CSAT through better tooling and automation requires a pipeline that is stable at the tail, not just fast at the median.

Measuring and improving customer sentiment across every interaction is only possible if the interaction itself is not degraded by latency variance the vendor never disclosed. The best AI phone agent platform for enterprises exists precisely because realism and latency are not separable concerns. A voice that arrives late is indistinguishable, from the caller's perspective, from a voice that sounds wrong.

Use Cases for Low Latency TTS and Why Each One Breaks Differently Under Load#

The failure mode that kills a voice application rarely announces itself in testing. The common assumption is that published TTFA and P99 numbers from TTS API vendors accurately predict how that API will perform in a live voice application. In reality, it shows up at 2 PM on a Tuesday, when call volume spikes and the latency number that looked clean in your sandbox quietly doubles, then doubles again, and suddenly the experience breaks in a way that's specific to your use case, your users, and your business consequences.

"Outbound calling and conversational assistant use cases face distinct latency and turn-taking challenges in production-like environments, highlighting that each use case breaks differently under real load."

Our own research found that callers are rarely in controlled environments, meaning ambient noise is a persistent and common challenge for deployed AI voice agents (our data).

1. Conversational AI Voice Agents - Where 300ms Feels Like a Hung Call#

Human conversation operates on a surprisingly tight clock. Research by Levinson and Torreira found that the modal gap between conversational turns is approximately 200 milliseconds, with speakers beginning to plan their response before the prior turn even ends. That means the full STT-LLM-TTS pipeline must resolve within a window that feels almost impossibly short. When it doesn't, the caller doesn't think "the API is slow." They think the call has dropped.

The problem compounds under concurrent load. Teams building voice AI phone agents consistently find that a TTS API posting sub-150ms TTFB in isolation behaves very differently when 200 sessions are running simultaneously on shared cloud infrastructure. The latency floor rises, the pauses lengthen, and callers, wired to interpret silence as disconnection, hang up.

2. IVR Systems at Peak Call Volume - Latency That Misroutes and Abandons#

IVR latency failures are not subtle. When a prompt takes too long to play, callers press a key early, the system misreads the input, and the call routes to the wrong queue. At scale, this is a measurable misrouting rate that compounds into abandoned calls, re-queued volume, and agent escalations that defeat the cost rationale for the IVR in the first place.

3. Live Sports Broadcast Dubbing - Frame-Accurate Audio Sync at Scale#

Live sports commentary dubbing demands a low latency TTS API capable of sub-200ms TTFB to stay lip-sync-adjacent with broadcast video feeds. Unlike conversational use cases, failure here is visible to millions simultaneously, a 500ms slip during a goal call is a broadcast incident. The compounding load problem is multilingual parallel streams: rendering five language tracks concurrently multiplies GPU demand and can cause one language to lag while others stay on time.

4. Healthcare Voice Assistants - When Latency Erodes Clinical Trust#

Voice-enabled AI in clinical settings, patient triage, medication reminders, post-discharge check-ins, relies on a low latency TTS API to maintain the cadence of a credible, calm interaction. Delays exceeding 600ms cause patients to repeat themselves or disengage, which in triage contexts can mean missed symptom capture. The unique load failure mode here is that hospital systems often spike during emergencies, exactly when reliability matters most and infrastructure is most stressed.

5. Real-Time AI Sports Translation - The Two-Second Wall Nobody Warns You About#

AI sports translation pipelines chain ASR, NMT, and TTS sequentially, meaning a low latency TTS API must absorb whatever budget remains after upstream models consume their share. In practice, TTS is the last stage and the first to get throttled under load. The two-second total latency target is achievable in isolation but breaks when translation models slow under concurrent language-pair demand, leaving TTS with under 100ms to render, a budget most neural voices cannot meet without quality degradation.

Best Low Latency TTS APIs for Real-Time Voice Apps - Vendor Comparison#

Six months after launch is a brutal time to discover that your TTS vendor's 80ms demo number was measured on a single warm request against an idle server, with no concurrent sessions, no telephony routing overhead, and no compliance hops in the path. By then, you have contracts, integrations, and a P99 that looks nothing like the benchmark. The audit should happen before the purchase order, not after the post-mortem.

The core problem is that latency is an infrastructure property, not a model property. The vendor's streaming architecture, whether they own their compute or rent shared cloud capacity, how they handle concurrent sessions, and whether their compliance posture adds processing hops between synthesis and delivery: these factors determine your real-world P99 far more than any published TTFA figure. A TTS API's published benchmark is structurally incapable of predicting production performance for any vendor running on shared public cloud, because benchmark conditions systematically exclude the two forces that dominate real deployments: concurrent-session queuing and third-party routing hops. Infrastructure architecture, not headline TTFA, is the correct first filter when comparing vendors at scale.

Teams who have run high-volume voice workloads know this pattern well. Latency that looks acceptable at one concurrent call starts degrading unpredictably at fifty, and callers begin talking over the agent before it finishes a sentence. The evaluation criteria below treat each vendor as an infrastructure decision, not a feature comparison.

1. Bland.ai - Best Low Latency TTS API for Enterprise Phone AI#

Bland.ai runs fine-tuned voice models on dedicated, co-located GPU infrastructure with a Voice Delivery Network that routes each call to the nearest server, eliminating shared-cloud contention and third-party synthesis hops entirely. Platforms that route synthesis through shared public cloud capacity inherit noisy-neighbor latency variability by design, a dynamic that industry research identifies as a structural characteristic of pooled GPU environments rather than a correctable configuration issue. It is the strongest fit for regulated-industry teams running high-volume concurrent calls where production P99, compliance posture, and infrastructure ownership all need to hold simultaneously.

The trade-off is real: dedicated infrastructure at this level is engineered and priced for enterprise scale. Teams running low call volumes, early-stage prototypes, or cost-sensitive SMB workloads will find the entry point higher than shared-cloud alternatives, and should evaluate whether that infrastructure investment matches their current throughput.

2. ElevenLabs - Best Low Latency TTS API for Ultra-Realistic Streaming Voice#

ElevenLabs Eleven Flash delivers approximately 75ms TTFA via WebSocket streaming across 70-plus languages, according to published figures from ElevenLabs, making it one of the fastest pure-voice synthesis options available. ElevenLabs' voice quality is rated among the highest in independent evaluations, including Inworld AI's 2025 TTS benchmark review, which cites it as a top performer for naturalness, and the 10,000-plus voice library gives product teams real selection depth. The practical limitation for high-concurrency workloads is that ElevenLabs routes synthesis through shared cloud infrastructure, so that 75ms sandbox figure is a best-case number. Teams running hundreds of simultaneous calls should request concurrent-load data before committing.

3. Deepgram Aura - Best Low Latency TTS API for Developer-First Voice Pipelines#

Deepgram's current TTS product is called Flux TTS, and advertises latency as low as 80ms with WebSocket support. The architecture is developer-first, which means the integration burden of stitching STT, LLM, and telephony around it falls on your team. For organizations without dedicated voice infrastructure engineers, that assembly cost is real and often underestimated before the project starts.

4. Cartesia Sonic - Best Low Latency TTS API for Streaming-First Voice Agents#

Cartesia Sonic 3.5 achieves approximately 40ms time-to-first-byte using State Space Models instead of transformer architectures, has cited strong naturalness rankings on public speech evaluation leaderboards, and supports 44 languages with voice cloning from ten seconds of audio, figures drawn from Cartesia's published technical documentation and worth independent verification before a procurement decision. The SSM architecture is purpose-built for chunk-level streaming, which keeps chunk latency low even as session count grows. The honest caveat is that SSM-based models are newer in production at scale, and teams should validate P99 behavior under their specific concurrent call volumes rather than relying on single-session benchmarks.

5. Google Cloud Text-to-Speech - Best Low Latency TTS API for Multilingual Scale#

Google Cloud TTS covers 75-plus languages and integrates cleanly into existing GCP infrastructure, making it the lowest-friction path for teams already running on Google's stack. The latency profile is competitive at low concurrency. The well-documented production complaint is regional variance under high concurrency: teams running globally distributed voice workloads consistently report that P99 climbs and becomes inconsistent across regions when session volume spikes, because the service runs on shared public cloud capacity with no dedicated compute guarantee. For multilingual scale with predictable latency, that tradeoff needs explicit evaluation.

6. AssemblyAI Universal-2 + TTS - Best Low Latency TTS API for End-to-End Voice AI Stacks#

AssemblyAI's Voice Agent API bundles STT, TTS, and pipeline orchestration into a single integrated offering. The flat-rate model simplifies cost forecasting for teams tired of per-character billing surprises. The end-to-end latency target sits around one second for a full pipeline turn, which is acceptable for many phone agent use cases but above the sub-400ms threshold that real-time conversational AI demands. It is the right pick when transcription accuracy and billing simplicity outweigh raw synthesis speed.

Choosing the right vendor is only half the infrastructure decision. How audio actually travels from the API to the caller's ear, in what format, over which protocol, and through which SDK, determines whether that vendor's latency promise survives contact with your real stack. The next section breaks down exactly how those delivery choices function as latency decisions in disguise.

Streaming and Real-Time Audio Delivery - Output Formats, Languages, and SDK Integration Options#

The audio format you choose for streaming TTS output looks like a compatibility checkbox on a vendor comparison sheet. It is a latency variable, and the wrong choice compounds quietly across every concurrent session until your P99 numbers tell a story your sandbox tests never predicted. For teams running high-volume outbound campaigns or 24/7 inbound coverage on platforms like bland.ai, where the Scale plan supports up to 100 concurrent calls and 5,000 calls per day, format selection stops being academic the moment you push real traffic.

PCM vs. Opus vs. MP3 - How Encoding Overhead Silently Inflates Your Streaming Latency#

Raw PCM carries zero encoding overhead. According to industry research, PCM enters your streaming pipeline immediately, with no compression computation in the path. MP3 is the opposite: its psychoacoustic frame-buffering requirement means audio must be batched before the first byte can transmit, adding algorithmic delay that compounds at scale. Tencent MPS's codec comparison puts Opus in a useful middle position, supporting frame sizes as small as 2.5ms and variable bitrates that make it purpose-built for real-time delivery.

The trade-off that vendor benchmarks rarely surface: PCM's zero-encoding advantage disappears under concurrent load. A persistent WebSocket session delivering high-bitrate uncompressed audio imposes greater server-side memory and bandwidth pressure per session than an Opus stream at equivalent perceived quality. The lowest-latency format for a single warm request is not necessarily the lowest-latency format when hundreds of sessions run simultaneously. No published TTFA figure reflects this.

• PCM → Encoding overhead: Zero; enters the pipeline immediately → Streaming latency: Lowest → Concurrent-load behavior: Greater server-side memory and bandwidth pressure per session.
• Opus → Encoding overhead: Low; frame sizes as small as 2.5 ms, with variable bitrates → Streaming latency: Low → Concurrent-load behavior: Purpose-built for real-time delivery; more efficient than PCM at scale.
• MP3 → Encoding overhead: High; psychoacoustic frame-buffering batches audio before the first byte transmits → Streaming latency: Highest → Concurrent-load behavior: Algorithmic delay compounds at scale.

Voice cloning architecture also intersects with format choice in ways most teams underestimate. Streaming-first TTS models make a real architectural tradeoff: they optimize for low first-byte latency at the cost of speaker similarity fidelity. Teams we work with consistently find that while tone, male versus female register, and general cadence are captured reliably, true speaker similarity degrades under the constraints of real-time delivery. Bland.ai addresses this directly with custom-trained models and up to 15 voice clones on the Scale plan (5 on Build, 1 on Start), trained with a fidelity target that is not subordinated to raw streaming speed. For high-volume outbound campaigns where brand voice consistency matters across thousands of calls per day, that distinction is not cosmetic.

Cross-platform deployment compounds these concerns for smaller engineering teams. A TTS stack that performs correctly in a controlled development environment can surface silent failures in production: connection errors that the settings layer reports as successful, or voice assets that resolve locally but fail at runtime under load. These are not edge cases; they are the category of problem that burns sprint cycles and erodes confidence in voice infrastructure before a single call is placed. Bland.ai's integrations platform, including its Amazon Connect integration, is designed to remove that surface area, letting teams substitute or augment human agents within existing call flows without rebuilding the telephony layer from scratch.

WebSocket vs. REST Endpoints - Why the Connection Model Is a Latency Architecture Decision#

WebSocket TTS streaming holds a persistent connection open, which eliminates the TCP handshake and TLS negotiation cost on every utterance. For real-time conversational AI, that connection-setup overhead on a REST call is a repeating tax on every turn in the dialogue. Developers who have built production voice stacks consistently report that switching from REST to WebSocket delivery cut their perceived response lag more than any model-level tuning did.

REST is not wrong for every use case. For async workloads or low-concurrency pipelines, the simpler request-response model is easier to operate and debug. But for high-volume outbound calling, automating maintenance requests, delivery confirmations, appointment scheduling, or sales follow-ups without adding headcount, REST's full-buffer delivery requirement means audio cannot start playing until the entire utterance is encoded and transferred. That is the architectural opposite of what real-time conversational voice demands, and a compounding latency cost at any meaningful call volume.

Tencent MPS frames this clearly in its codec analysis: the codec and transport layer choices made at design time determine the latency floor of the entire system. For teams on bland.ai's Build plan managing up to 50 concurrent calls, or Scale teams running up to 100, that floor is load-bearing. Bland.ai's real-time transcription and premium voice delivery are both included in the per-minute rate across all plans, so the per-session cost of a WebSocket-delivered, low-latency call is predictable at $0.11/min on Scale and $0.12/min on Build, with no separate token charges that inflate unpredictably as session depth grows. For organizations that need to upskill or supplement a small team without adding full-time headcount, that cost predictability is as important as the latency architecture itself, with transfer minutes billed separately at $0.04/min.

How to Evaluate a Low Latency TTS API for Production - Beyond the Benchmark#

Trusting a vendor's published TTFA to predict production behavior is a reasonable starting point. It becomes a liability the moment your call volume scales and the conditions behind that number no longer match your environment.

The Four Production-Readiness Dimensions Vendor Benchmarks Never Report#

Most benchmark tables report one thing well: how fast a single request returns audio on a clean, warm server. What they omit are the four dimensions that actually determine whether a TTS API survives production:

• Concurrent-load latency, how latency behaves when simultaneous requests compete for the same resources
• Infrastructure ownership, whether compute is dedicated or shared, and what that means for variability
• Third-party data exposure, whether sub-processors are disclosed and whether any synthesis hops exist
• Full-stack integration depth, whether the vendor owns the entire pipeline or stitches together third parties

Each of these introduces failure modes that no spec sheet captures. Skipping any one of them during evaluation means discovering it post-launch, under real call volume, with real users on the line.

Concurrent-Load Latency, The Only Latency Number That Predicts Real Costs#

Shared GPU environments expose every tenant to the resource contention of every other tenant, and no configuration change at the application layer can eliminate that structural variability. Dedicated or reserved compute is the only architectural remedy.

Published TTFA figures are measured in isolation and do not reflect how latency behaves when simultaneous requests compete for the same resources.

A vendor posting 80ms TTFA in a sandbox can deliver 600ms or worse when your pipeline stacks STT, LLM inference, TTS, and telephony encoding under real load. Human conversation tolerates roughly a 200ms inter-turn gap; assembled stacks routinely exceed 800ms even when each individual component looks acceptable in isolation. Shared GPU infrastructure creates the noisy-neighbor problem: another tenant's workload spike becomes your latency spike. This pattern holds consistently across shared GPU environments based on our market understanding.

Production Evaluation Checklist, Low Latency TTS API#

Before signing any vendor contract, validate each of the following in writing or via a live stress test:

• Concurrent-load P99 → What to ask/test: Request P99 latency at your expected peak session count → Pass threshold: ≤ 400 ms end-to-end under load.
• Infrastructure ownership → What to ask/test: Dedicated vs. shared GPU? → Pass threshold: Dedicated or reserved capacity confirmed.
• Cold-start latency → What to ask/test: Test WebSocket reconnect from cold Lambda / rotating keys → Pass threshold: < 600 ms including handshake.
• Compliance data path → What to ask/test: Are sub-processors disclosed? Any synthesis hops? → Pass threshold: Full sub-processor list provided in writing.
• Full-stack integration depth → What to ask/test: Does the vendor own STT + TTS + telephony or stitch third parties? → Pass threshold: Single-owner pipeline preferred for regulated use cases.
• Regional P99 consistency → What to ask/test: Test across all target geographies under concurrent load → Pass threshold: < 20% P99 variance across regions.

Action: Add this checklist to your vendor POC kickoff document before any sandbox trial begins.

Next steps#

If your sandbox TTFA numbers looked clean but your production P99 tells a different story at scale, the path forward starts with recognizing that infrastructure architecture, not headline latency figures, is the correct first filter when comparing vendors. Start with the best AI phone agent platform for enterprises.

Vendor benchmarks measure a single warm request against an idle server, which means the WebSocket upgrade handshake cost is never reflected in the numbers you used to make your decision. Cold-start reconnection, concurrent-session queuing, and telephony routing hops each add latency that no published figure captures. Separately, the audio format choice that looks optimal for one warm request stops being optimal when hundreds of sessions run simultaneously, because uncompressed PCM imposes server-side memory and bandwidth pressure that compounds in ways a sandbox never reveals. Together, these two dynamics point to the same conclusion: evaluating any TTS API outside its full assembled stack, under realistic concurrency, gives you a number that will not survive Tuesday afternoon call volume.

Start with bland.ai to see how dedicated infrastructure and a Voice Delivery Network address both failure modes before they reach production.

Frequently Asked Questions#

Why do TTS vendors' advertised latency numbers look so much better than what I see in production?#

Vendor-published TTFA figures are typically measured on a single, warm, isolated request with no concurrent load, no queuing effects, and no telephony routing in the path, conditions that never exist in a live deployment. The real-world gap between sandbox and production latency is routinely 3 to 5 times larger at scale, meaning a TTS API that posts 200ms TTFA in documentation can exhibit 600ms to 1,000ms in a live deployment.

What's the difference between TTFB and TTFA and which one actually matters for voice apps?#

TTFB (Time to First Byte) is a network metric that tells you when the first byte of a response left the server and reached the client, vendors surface it because it is the smallest defensible number in the latency chain, not because it reflects what callers experience. TTFA (Time to First Audio) measures how long after a request is submitted before the first audio sample arrives, which is closer to user experience but still excludes turn-detection processing, LLM inference, and telephony overhead. Neither figure on its own captures the full end-to-end latency a caller actually hears.

Why should I care about P99 latency if my P50 looks fine?#

The median call performing well is not your problem, your problem is the call at the 99th percentile, because that is what callers actually hear during traffic spikes and high concurrency. The P50 number often earns the vendor contract, but the P99 number is what ends the relationship once the application is in production at scale.

How much latency does telephony infrastructure add on top of whatever the TTS API reports?#

SIP signaling, PSTN routing, and WebRTC negotiation each add overhead that never appears in a vendor's published benchmark. Industry analyses of production voice deployments confirm that telephony overhead alone adds 100ms to 300ms on top of any synthesis figure a vendor reports in isolation.

How short does a response delay have to be before callers notice something is wrong in a conversational AI agent?#

Research found that the modal gap between conversational turns is approximately 200 milliseconds, with speakers beginning to plan their response before the prior turn even ends. When the full STT-LLM-TTS pipeline exceeds that window, callers don't think the API is slow, they think the call has dropped.