Voice latency is the metric the customer feels even though they cannot name it. A 600 ms response feels like a normal conversation. A 1.5 s response feels stilted. A 3 s response feels broken. Customers do not say 'the latency was high'; they say 'the AI was weird' and hang up.
The headline number to ask for is p95, the latency on the worst five percent of turns, not the average. A vendor whose average looks good but whose p95 is over three seconds will sound great in the demo and terrible in production.
Latency in a real voice agent is dominated by the LLM thinking time, not the network hops between services. The right way to reduce it is to decompose the pipeline, parallelize where possible, and pin the model. Asking for 'sub-second latency' without those details is asking for marketing copy.
Vorel publishes both p50 and p95 voice latency, and credits the contract SLA automatically if either is breached.