Prompt caching explained, with the real break-even

By John M. · Updated 15 August 2026 · 6 min read

Put the current timestamp at the top of your system prompt — a reasonable thing to want, and a line of code — and you can quadruple the input cost of a high-traffic feature. Nothing else changes. The model is the same, the traffic is the same, the prompt is nineteen characters longer. What changed is that no two requests now share a byte-identical prefix, and the cache that was serving 90% of your input tokens at a tenth of the price is serving none of them.

Prompt caching is the largest discount available on the input side of an LLM bill and the one most sensitive to how the prompt is assembled. Claude Sonnet 5 charges $2.00 per million input tokens and $0.20 per million for cached input. Gemini 2.5 Flash charges $0.30 against $0.03. DeepSeek V4 Flash charges $0.14 against $0.0028, a 98% reduction. Those are not small enough to ignore and not automatic enough to assume.

What is actually cached

The provider keeps the computed key-value state for a prefix of your prompt in GPU memory. When the next request arrives, it compares your prompt's opening bytes against what it holds; if they match, it skips the prefill work for that span and starts computing from the first differing token. The requirement is exact and positional. Not "similar content", not "the same instructions in a different order" — the same bytes, from position zero, in the same sequence, with the same tool definitions serialised the same way. Providers also enforce a minimum cacheable length, commonly in the region of 1,000 to 2,000 tokens; a 400-token system prompt gets nothing regardless of how stable it is.

Everything from the first mismatch onwards is billed at the full input rate. A cache is therefore not a percentage of your prompt that happens to be reused — it is a contiguous run from the top, terminated by the first byte that varies.

The break-even, worked in full

Assume a 12,000-token stable prefix on Claude Sonnet 5: system instructions, tool schemas, a style guide, four worked examples.

Uncached, that prefix costs 12,000 / 1,000,000 × $2.00 = $0.024 per request. Read from cache it costs 12,000 / 1,000,000 × $0.20 = $0.0024. But writing to the cache carries a premium: Anthropic bills a five-minute cache write at 1.25× the base input rate, which for Sonnet 5 is $2.50 per million, so the write costs $0.030 — 25% more than simply sending the prompt uncached.

So each cached prefix costs $0.030 once, then $0.0024 per subsequent hit, against $0.024 every time with no cache at all. If a written prefix is reused N times in total:

$0.030 + $0.0024 × (N − 1) = $0.024 × N

which solves to N = 1.28. Expressed as a hit rate — hits divided by total requests, or (N − 1) / N — the break-even is 22%. Above that, caching saves money. Below it, caching costs money.

The one-hour cache tier is priced at 2× base, $4.00 per million on Sonnet 5, so a write is $0.048. Rerunning the same equation gives N = 2.11 and a break-even hit rate of 53%. The longer tier is not a free upgrade; it needs more than twice the reuse to justify itself, and it only makes sense for prefixes whose traffic is steady but sparse.

Now the monthly view. Take 200,000 requests a month against that 12,000-token prefix — 2,400 million prefix tokens.

At 90% the saving is $3,768 a month, 78%. At 30% it is $456, under 10% — real, but smaller than most teams expect from a headline "90% discount on cached input". At 15% caching is losing $372 a month against doing nothing at all. That last row is the one nobody models before switching caching on, and it is not a rare configuration: it is what any low-traffic feature with a long prefix looks like. The prompt caching savings calculator will run your prefix length and hit rate against the current price index.

Three things caching never covers, and each surprises someone. Output tokens are not cached — nothing about generation is reusable, so the $10.00 per million output rate on Sonnet 5 stands whatever your hit rate. Caches are scoped to your organisation and usually to a region, so a multi-region deployment maintains a separate cache per region. And every deployment that changes a single character of the system prompt cold-starts the whole fleet, which is worth knowing before you ship prompt tweaks hourly.

Lifetime is the constraint that decides everything

The default cache lifetime is short — five minutes on the standard Anthropic tier, refreshed on each hit — because the provider is holding GPU memory for you and that memory has other uses. A prefix that goes unused for the window is evicted, and the next request pays the write premium again.

This turns a pricing question into a traffic question. A feature serving 200,000 requests a month averages 4.6 per minute, so a five-minute window is comfortably long and hit rates in the 90s are achievable. A feature serving 2,000 requests a month averages one every 22 minutes: essentially every request is a cache miss followed by a write, which puts that month at 2,400 million × $2.50 = $6,000 against $4,800 uncached — a flat 25% surcharge for a cache that never gets read. Same code, same prompt, opposite outcome.

Density matters per distinct prefix, not per feature. If your prompt embeds a tenant identifier or a per-customer instruction block, forty tenants means forty separate caches each getting a fortieth of the traffic, and a feature with healthy aggregate volume can have terrible per-tenant hit rates. Traffic that clusters in business hours has a hit rate that varies by time of day, and the overnight tail can sit below break-even while the daytime peak sits well above it.

Hit rate is an architecture property, not a setting

There is a flag to enable caching, and it does almost nothing on its own. Three structural decisions determine what your hit rate will be.

Prompt ordering. Everything stable goes first, everything volatile goes last: system instructions, then tool schemas, then few-shot examples, then retrieved chunks, then conversation history, then the user's message. The common failure is inserting retrieved documents above the instructions because that is how the template was written, which puts a per-request variable at position zero and caches nothing. The second most common is a timestamp, a request ID or a personalised greeting near the top.

Tool schema stability. Tool definitions are usually long, usually near the top, and usually generated. If your serialiser does not guarantee key ordering, two logically identical schemas emit different bytes and the cache misses intermittently in a way that is very hard to spot — the bill drifts, nothing errors. Pin the serialisation, sort keys deterministically, and treat any change to a tool description as a deployment that resets your cache across the fleet.

Traffic shape. Covered above, and it is the one you cannot fix in the prompt. If a feature does not have the density, either accept the miss rate and skip caching, or consolidate: one shared prefix across several features usually beats three bespoke ones, because the shared version gets three times the traffic keeping it warm.

Before enabling caching on anything, get two numbers: the length of the genuinely invariant prefix, and requests per five-minute window per distinct prefix. If the second is comfortably above one, cache and expect most of the discount. If it is near one, you are at break-even and the complexity is not worth it. AI chatbot cost calculator will give you the first from a conversation profile; your own logs are the only source for the second.

All figures in this guide come from the price index, last verified 15 August 2026. Every entry links to the vendor page it was read from — see the price index and the methodology.