How prompt caching cut our voice LLM bill by 70%.

Anthropic shipped prompt caching for Claude last quarter. The pitch is straightforward: mark a chunk of your prompt as cacheable, pay a 25% premium the first time the model sees it, and pay roughly 10% of the normal input cost every time you reuse it within the cache TTL. For most chat workloads this is a nice 20-40% cost trim. For voice booking flows it is the single biggest cost lever in the model layer, and we measured a 70% reduction in our Anthropic spend per turn once we wired it correctly.

This post explains why voice booking is the perfect shape for prompt caching, the production numbers we saw, the trade-offs nobody mentions in the launch post, and the places where caching does not help at all. It is the cost-side companion to our latency post: once your voice agent is fast enough to feel good, the model layer is the next thing you cannot ignore.

Why voice booking is the perfect shape

A voice booking turn has a very specific token profile. The system prompt encodes the brand voice, the operator's escalation rules, the booking policy (cancellation windows, deposit terms, who gets priority on the wait list), and a tool catalog with input schemas. The booking context carries the customer record, the appointment slot the agent is currently holding, the calendar state, the parts or chair or technician inventory, and the conversation summary so far. Together these stack up to 5000-8000 tokens before the caller has said anything new.

Then the caller speaks. The variable tail is usually 10-40 tokens. The model thinks about it, calls a tool or two, and replies. Next turn, the prefix has barely moved: maybe one new entry in the conversation summary, maybe a slot was held. The next 5000-8000 tokens look 98% identical to the previous turn. This is exactly what prompt caching was designed to amortize. The marginal turn is mostly cache hits with a tiny uncached suffix.

Compare that to a typical chat workload. A general-purpose support chat has a system prompt of 800 tokens and a conversation that grows linearly. Caching the system prompt buys you a 15% trim because the system prompt is a small fraction of total input. In voice booking the cacheable prefix is the majority of input, so the same caching mechanic moves the cost dial almost an order of magnitude further.

In voice booking the cacheable prefix is the majority of input. The same caching mechanic that trims a chat workload by 15% trims a booking call by 70%.

The pricing math

Anthropic prices cached prompts on three axes: a cache write (first time the model sees the marked block) at roughly 1.25x normal input, a cache read (every subsequent use within the TTL) at roughly 0.1x normal input, and standard pricing on everything outside the cached block. The break-even versus uncached is two reads. Anything past that is pure savings.

per-token cost (relative to standard input = 1.0x):

  cache write     1.25x   (one-time, first turn)
  cache read      0.10x   (every reuse within TTL)
  uncached input  1.00x

example: 6000-token prefix, 8-turn voice call

  without caching
    8 turns x 6000 tokens x 1.00x  =  48000 cost units

  with caching
    1 write x 6000 tokens x 1.25x  =   7500
    7 reads x 6000 tokens x 0.10x  =   4200
    total                          =  11700

  ratio: 11700 / 48000 = 24.4%
  saving: 75.6% on the prefix portion
  blended call saving (incl. uncached tail + output): ~70%

Two things to notice. First, the crossover is two reads. A one-turn conversation pays a 25% penalty for cache write and never gets to amortize it. We do not cache on calls shorter than two turns, which we predict from caller intent classified on the first audio frame (the customer who says "Just confirming my Thursday slot" gets a non-cached one-shot path). Second, the savings are concentrated on the prefix. Output tokens, uncached tail tokens, and tool-call round trips are unchanged. The 70% number is the blended-call saving against our actual production token mix, not a marketing top-line.

The TTL question

Anthropic caches with a 5-minute sliding TTL. Every read resets the clock. The relevant question for voice is whether 5 minutes is long enough that the average call stays inside the window across turns.

Our production data, across roughly 9000 voice turns sampled over four weeks: median gap between adjacent turns in the same call is 7 seconds. p95 is 28 seconds. p99 is 71 seconds. The 5-minute TTL is, in practice, never the bottleneck on intra-call caching. 99.4% of adjacent turns hit the cache.

Where the TTL does show up is across calls. A returning caller in a different session does not hit a cache from their prior call. We initially thought about pre-warming the cache by replaying the prior conversation summary on call answer; we shelved it because the cache-write cost of pre-warming exceeded the savings unless the second call ran past three turns, and we could not predict call length reliably enough at answer-time to justify it. The right answer turned out to be: cache aggressively within a call, do not try to cache across calls.

The booking-context trim trade-off

Caching only pays off when the cached block is identical across turns. The first time we wired this, our cache hit rate was 40%, not the 99% the TTL math predicted. The reason was sloppy prefix construction: we were embedding the current ISO timestamp in the booking context for "now-relative" reasoning, which mutated the cached block on every turn. Every turn produced a fresh cache miss and we paid the 1.25x write penalty for nothing.

The fix is to push the timestamp to the uncached tail. The model still sees "current time" but it lives in the variable suffix next to the user utterance. The cached block is exactly the same bytes on turn 2 as on turn 1. After that fix, hit rate jumped to 98% and the cost number landed where we expected.

The harder trade-off is tenant-specific. Some operators run booking policies that are long and prescriptive (a clinic with 14 cancellation rules, a service shop with parts-pricing tiers). Embedding the full policy in the cached prefix produces the best agent behavior; trimming the policy by 20% (consolidating the longest tail rules into a "see escalation" sentinel) produces a slightly worse agent but a cheaper one. We expose this as a per-tenant knob: prefix size versus per-turn cost. Larger tenants with high call volume pick the trim; smaller tenants who care most about edge-case correctness pick the full prefix. The right answer is genuinely operator-specific.

The cached block has to be byte-identical across turns. The first time we wired this we leaked a current-timestamp into the prefix and got 40% hit rate instead of 98%.

Where caching does not help

Five places caching is the wrong answer, and we will name them honestly because the launch-post framing was tilted toward the wins.

First, the first turn of any call. The first turn pays the cache write at 1.25x and gets nothing back. On a one-shot confirmation call this is a 25% net cost penalty. We classify caller intent on the first audio frame and skip caching on predicted-short calls.

Second, anything that mutates the prefix mid-call. Tool results that you want the model to reason over (a calendar query result, a parts lookup) cannot live in the cached block, because they change every turn. They live in the uncached tail, which is fine, but it means the cache savings only cover the static portion. If the static portion is small relative to the tools-and-results stream, caching is a rounding error.

Third, output tokens. Caching is an input-side mechanic. Output is unchanged. If your agent generates long replies (rare in voice; the LLM stays terse because the TTS layer cannot stream a paragraph naturally), the cache will not help with that cost.

Fourth, model swaps. Cache entries are per-model. The day we route a turn from Sonnet to Haiku for cost (which we do, per the latency post; smaller models on routine turns), that turn does not hit the Sonnet cache. We considered keeping every turn on Sonnet just to preserve cache continuity. The math did not justify it: paying 1x on Haiku is still cheaper than paying 0.1x on Sonnet for the easy turns.

Fifth, multi-tenant prefix sharing. The cache is keyed on exact prefix bytes. Two tenants with similar but not identical booking policies do not share a cache entry. We briefly considered a synthetic "shared base prefix" that all tenants extend; the engineering complexity (and the tenant-isolation security review) was not worth the marginal saving on small tenants.

The deeper principle

Our latency post argued that voice latency is the LLM layer, and that everything else is a rounding error. The cost side of voice has the same shape. Speech-to-text is cheap. Text-to-speech is cheap. The network hops are free. The LLM bill is 80-90% of the per-minute cost on a serious production deployment. Once latency is good enough that the caller cannot feel the model, cost optimization on the model layer is the next surface that matters.

Prompt caching is the highest-leverage move on that surface today. It is also the one with the cleanest engineering shape: you mark the cacheable block, you keep it byte-identical, you ship. There is no model swap, no infrastructure migration, no protocol change. The whole intervention is two API parameters and a one-week refactor to clean up timestamp leakage. The return is a structural cost improvement that scales with call volume.

We expect the rest of the field to land here within two quarters. The vendors who do not are leaving 60-70% of their model cost on the table, and the buyer eventually feels that in the per-minute price. The model layer is where the cost lives. Once you know that, the optimization roadmap writes itself.