Grading every voice turn. The production loop that catches hallucinations before they ship.

Voice-agent hallucinations are a small percentage of turns and a large percentage of the bad screenshots. A model that gets ninety-eight percent of its turns right and quotes the wrong refund window on the other two percent is still a model your compliance team will not let near production. The right way to think about hallucination defense in a voice deployment is operational, not aspirational: a grader that runs on every turn, in production, against a small set of factual constraints, and routes anything outside that set into a queue a human looks at within minutes. That loop is the difference between a voice agent that ships and one that stays in a staging environment for nine months.

We have been running this loop in production for a while. The grader catches roughly one to three flagged turns per hundred on a well-prompted agent, more on a freshly deployed tenant whose constraint list has not been fully populated yet. The vast majority of flags turn out to be the agent saying something the operator considers true but had not loaded into the constraint list. A small minority are real hallucinations, and those are the ones the operator queue exists to catch. This post is the architecture, the grader's prompt, the alerts pipeline, and the design choices that make the loop trustworthy enough to leave running unattended.

The four claim categories that get caught

Voice-agent hallucinations cluster into a small number of categories, and the categories are remarkably consistent across verticals. The agent confidently quotes a price that is not on the price list. The agent reassures a caller about hours the business is not actually open. The agent commits to a policy the operator does not actually offer, like a thirty-day refund window or a free same-day reschedule. The agent claims a capability it does not have, like transferring a call to a specific manager by name or pulling up a record from a system it has no integration with. Four buckets cover the overwhelming majority of failure modes that show up in production review.

These categories all share one structural property: they are factual claims the agent made without grounded evidence in the prompt context. The model did not look up the price; it generated a plausible one. The model did not consult the policy document; it produced a refund window that sounded reasonable. This is pretraining filling in for missing context, and it is exactly the kind of error a grader checking the utterance against a known-true constraint list can catch.

Hallucinations in a voice agent are not random. They are the model filling in for missing context with whatever sounds plausible, and they cluster into four categories a grader can be tuned for.

Why the weekly batch eval is the wrong tool

The standard industry move for hallucination defense is a batch eval. The team samples a few hundred calls per week, scores them offline against a rubric, and ships fixes the following sprint. This is a fine quality-improvement process and a useless safety mechanism. By the time a Monday-morning eval flags that the agent has been quoting a wrong refund window since last Tuesday, five hundred callers have already heard it. Some of them have made decisions based on what they were told. A subset of those will dispute charges, request refunds the operator never agreed to, or post the exchange publicly. The eval is doing exactly what an eval is designed to do; the point is that an eval is not the right tool to stop a bad turn from reaching a caller in the first place.

The per-turn grader is the tool that does that job. It runs on every agent utterance, in production, with a latency budget low enough that flagging an issue is operationally useful within the same call or shortly after. The eval still runs weekly, as a sanity check on the grader itself; the two systems are complementary, with the eval validating calibration and the grader stopping bad turns from compounding into bad days.

Architecture, prompt, and the four verdicts

The grader is a small LLM call (we use a Haiku-class model on most tenants, sometimes a tuned classifier where the volume justifies it) that takes the agent's utterance, a structured constraint list for the tenant, and a short instruction prompt. It returns a verdict, a rationale, and a constraint reference. The whole call runs asynchronously off the voice loop, so it does not block the caller. Cost lands at around half a tenth of a cent per turn, which works out to a few dollars per thousand calls.

The grader returns one of four verdicts. The verdict supported means the utterance is consistent with the constraint list or makes no factual claim that needs grounding. The verdict unsupported means the utterance contains a factual claim that is absent from (or contradicts) the constraint list. The verdict out_of_scope means the utterance is about something the constraint list does not cover, which is itself a signal worth tracking. The verdict llm_flagged means the grader is uncertain and wants a human to look. Four verdicts, one rationale, one constraint reference, and a confidence score; that is the entire output shape. Here is the abridged prompt and a worked example.

SYSTEM:
You are a factuality grader for a voice-agent utterance. The agent
spoke to a real caller. Your job is to decide whether the agent made
any factual claim that is not grounded in the tenant's constraint list.

You receive:
  - the agent's utterance (one turn of speech)
  - the tenant's factual constraint list (prices, hours, policies,
    capabilities, current promotions)
  - the immediately preceding caller turn for context

You return a JSON object:
  {
    "verdict": "supported" | "unsupported" | "out_of_scope" | "llm_flagged",
    "claim": "<the specific factual claim the agent made, or null>",
    "constraint_ref": "<id of the constraint checked, or null>",
    "rationale": "<one sentence, plain English>",
    "confidence": 0.0 - 1.0
  }

Rules:
  - Claims about price, hours, policy, refund, capability, or
    availability MUST be grounded in the constraint list.
  - Conversational filler ("let me check", "one moment") is supported.
  - If the agent referred the caller to a human or said it could not
    help, that is supported.
  - If the utterance contains a number, a time, or a promise, check it.
  - If you are uncertain, return llm_flagged. Do not guess.

USER:
constraint_list:
{
  "price_list": [
    { "id": "px_basic",   "service": "basic detail",   "price_usd": 149 },
    { "id": "px_premium", "service": "premium detail", "price_usd": 249 }
  ],
  "hours": { "mon_fri": "08:00-18:00", "sat": "09:00-15:00", "sun": "closed" },
  "policies": {
    "refund": "no_cash_refunds_store_credit_only",
    "reschedule": "free_up_to_24h_before"
  },
  "capabilities": ["book_appointment", "reschedule", "answer_pricing"]
}

caller_turn: "How much for a quick detail and can I get a refund if I change my mind?"

agent_turn:  "A quick detail runs about $129, and yeah, we offer a full 30-day money-back guarantee."

GRADER OUTPUT:
{
  "verdict": "unsupported",
  "claim": "basic detail price $129; 30-day money-back guarantee",
  "constraint_ref": "px_basic,policies.refund",
  "rationale": "Quoted price $129 conflicts with price_list ($149). Refund policy quoted as 30-day money-back; constraint says store-credit-only, no cash refunds.",
  "confidence": 0.97
}

An unsupported verdict on a price claim is one of the categories we treat as critical. The flag fires the alert pipeline immediately, and the operator dashboard surfaces the call with the flagged turn highlighted, the rationale visible inline, and a one-click action to acknowledge, override, or escalate. The same flag fires a webhook to the on-call channel for the tenant if the operator has wired one up. Here is the shape we put on the wire.

{
  "event": "hallucination_grader.flag",
  "severity": "critical",                         // info | warn | critical
  "tenant_id": "tnt_8421",
  "call_id":   "call_5af2c4",
  "turn_id":   "turn_5af2c4_018",
  "verdict":   "unsupported",
  "category":  "price",                           // price | hours | policy | capability
  "claim":     "basic detail price $129; 30-day money-back guarantee",
  "agent_utterance": "A quick detail runs about $129, and yeah, we offer a full 30-day money-back guarantee.",
  "caller_utterance":"How much for a quick detail and can I get a refund if I change my mind?",
  "constraint_ref":  ["px_basic", "policies.refund"],
  "rationale":       "Quoted price $129 conflicts with price_list ($149). Refund policy quoted as 30-day money-back; constraint says store-credit-only.",
  "confidence":      0.97,
  "grader_model":    "haiku-grader-v3",
  "constraint_list_version": "tnt_8421_v17",
  "audit_row_ref":   "audit_5af2c4",
  "links": {
    "operator_console": "https://app.vorel.ai/t/tnt_8421/calls/call_5af2c4#turn_018",
    "transcript":       "https://app.vorel.ai/t/tnt_8421/calls/call_5af2c4/transcript",
    "audio":            "https://app.vorel.ai/t/tnt_8421/calls/call_5af2c4/audio"
  },
  "recommended_action": "refund_or_callback_caller_within_2h"
}

The payload is designed for two consumers. The operator console reads it to populate the queue card. The tenant's on-call channel (Slack, Teams, PagerDuty webhook) reads it to fire an alert with enough context that the on-call engineer does not have to open the dashboard to triage. Both consumers see the same evidence: the agent's utterance, the caller's utterance, the constraint that was violated, and the grader's rationale. The recommended_action field gives the on-call a starting opinion, the way the audit row's recommended_next_step gives a triage lead a starting opinion on an escalation. The recommendation is not authoritative; it is a model commitment the human can accept, override, or ignore.

The audit row, and why per-turn beats per-call

Every grader verdict lands on the audit row for the turn, including the supported ones. The audit row exists to answer "what happened on that turn" with structured data, and the grader verdict is part of that data. Compliance teams ask, sooner or later, for a record of what factual claims the agent made and whether they were grounded; the audit row is where they look, and the verdict field is what answers them without anyone listening to the call. The constraint list version is captured alongside the verdict, because the list changes over time and the audit story needs to reproduce the grader's reasoning even after the list has been updated. A flag in May was checked against May's price list, not against today's.

A common shortcut is to run the grader at the end of the call against a summarized transcript. This is faster, cheaper, and useless in the way that matters. The whole point of the loop is to catch the unsupported claim at the moment it was made, on the row that records that moment, with the turn-level context that lets a reviewer judge what was happening. A call-level summary collapses the bad turn into the call's gestalt and obscures exactly the artifact the operator needs to act on. Cost is the usual objection, and the math works out: a five-minute call has roughly twenty agent turns at half a tenth of a cent per turn, so the grader adds about one cent to the call's marginal cost. On a contract priced at forty cents per call, the grader is roughly two and a half percent of cost of goods sold.

A grader flag is useful only if the human who reads it can act on it in under a minute. The payload is the index into the call; the recommended action is a starting opinion the operator can accept, override, or ignore.

Grounded versus ungrounded

The deeper principle behind the loop is simple. The line between a useful voice agent and a liability with a microphone is not the model, the prompt, or the guardrail library; it is whether the system can tell, on every turn, whether the agent stayed inside the set of facts it was authorized to make claims about. The per-turn grader is the mechanism that makes that distinction operational in production rather than aspirational on a slide. Without the loop, the operator is trusting that the model never strays. Models always stray; the only question is how often, how loudly, and how visibly. With the loop, the operator owns a queue of strayed turns, a versioned record of what the constraint list looked like when each turn happened, and an alerts path that turns the worst categories into a real-time signal. That posture is the one a regulated buyer can sign off on, and the one that makes the agent safe enough to leave running outside business hours.

When a buyer evaluates a voice-AI vendor, the question that separates serious deployments from theater is whether the vendor runs a per-turn factuality check in production. Most do not; some run an end-of-call summary check (weaker) and some run a weekly eval (offline). A vendor that runs a per-turn grader can show you the constraint list shape, the verdict schema, an example flag, and the operator-console view where flags land. If the vendor cannot show you those four artifacts, the loop does not exist, and the hallucination defense is whatever the prompt happens to be doing on the day of your trial. The per-turn grader is about a week of engineering once the audit row is already there, and it is the piece of infrastructure that lets a voice agent ship into a regulated buyer's CX with the compliance team's signature on the line.