Eleven ways to cut your LLM bill without switching model
Audit a $9,000 monthly LLM bill line by line and the usual finding is that about a third of it is work and the rest is habit: history nobody needed resent on every turn, a system prompt that accreted for eight months, three retries firing inside 400 milliseconds, and a nightly job running at synchronous prices because that is how the prototype was written. None of that requires a different model to fix. The eleven changes below are ordered by how much they typically return, which is not the order in which they are usually attempted — most teams start with the system prompt because it is the easiest thing to edit, and it is seventh on this list.
The four that pay for a week of engineering time
1. Stop resending the entire conversation on every turn. Savings: 40–70% on any multi-turn feature. A chat interface that appends each exchange to the message array and sends the lot back is paying for the same tokens over and over, quadratically in turn count. Work an example: a 500-token system prompt, 300-token user turns and 200-token replies, over a 20-turn conversation. Turn n sends 800 + 500(n−1) input tokens, which sums to 111,000 input tokens across the conversation. Keep a rolling window of the last four exchanges plus a running summary and the same conversation sends 51,000. At 100,000 conversations a month on Claude Sonnet 5 at $2.00 per million, that is 11,100 million tokens costing $22,200 against 5,100 million costing $10,200 — $12,000 a month, from a windowing function. The AI chatbot cost calculator models the turn-count curve directly.
2. Retrieve instead of stuffing. Savings: 50–90% of input on any document-grounded feature. Long context windows made it possible to paste an entire handbook into every request, and possible is not the same as sensible. A 200,000-token context on Gemini 3.1 Pro at $2.00 per million costs $0.40 per call before the model writes a word; ten thousand calls a month is $4,000. Retrieving the six relevant chunks instead — say 4,500 tokens — costs $0.009 per call, $90 a month, plus embedding costs that round to noise: 200 million tokens of corpus embedded once with OpenAI text-embedding-3-small at $0.02 per million is $4.00, one time. Answer quality usually improves too, because attention over 200,000 tokens of mostly irrelevant material is a real handicap. RAG pipeline cost calculator and context window cost calculator price the two sides against each other.
3. Cache the prefix. Savings: 40–80% of input on high-traffic features with a stable preamble. Cached input on Claude Sonnet 5 is $0.20 per million against $2.00 standard; on Gemini 2.5 Flash it is $0.03 against $0.30. Take a 4,000-token stable prefix — system prompt, tool schemas, few-shot examples — across 500,000 requests a month. Uncached, that prefix alone is 2,000 million tokens × $2.00 = $4,000. At a 90% hit rate, the 50,000 cache writes cost 200 million × $2.50 (Anthropic bills a five-minute write at 1.25× the base input rate) = $500, and the 450,000 hits cost 1,800 million × $0.20 = $360, for $860 total. That is $3,140 a month saved on tokens you were sending anyway. The catch is that the hit rate is not a setting — see prompt caching explained and the prompt caching savings calculator for the break-even, which bites below roughly 20%.
4. Batch everything that does not need an answer while someone waits. Savings: exactly 50%, with no quality change. Anthropic and OpenAI both run batch endpoints at −50% across their lines, returning results within a stated window, usually 24 hours. Classification backfills, nightly enrichment, evaluation runs, moderation sweeps, embedding refreshes and report generation all qualify. A nightly categorisation job pushing 900 million input and 60 million output tokens through Claude Haiku 4.5 at $1.00 / $5.00 costs $900 + $300 = $1,200 synchronously and $600 batched. The engineering work is a queue and a poller. The usual objection is that the job feels real-time — worth testing, because a report a user opens the following morning does not need to be generated the moment the data lands. Batch API savings calculator sizes it.
Worth doing, in most codebases
5. Route the easy requests down a tier. Savings: 40–70% of total spend on mixed workloads. Most production traffic is not uniformly hard. Classification, routing, extraction, short rewrites and yes/no judgements do not need the flagship, and running everything through one model because it is simpler to configure is the most expensive kind of simplicity. Send the cheap 70% to Claude Haiku 4.5 at $1.00 / $5.00 or Gemini 2.5 Flash-Lite at $0.10 / $0.40 and keep Claude Opus 5 at $5.00 / $25.00 for the rest. This does mean adding a model, not switching one — the flagship stays exactly where it is for the requests that need it. The arithmetic on a million requests a month at 3,000 in / 500 out: everything on Opus 5 is 3,000 million × $5.00 + 500 million × $25.00 = $27,500. Move 70% to Haiku 4.5 and that traffic costs 2,100 million × $1.00 + 350 million × $5.00 = $3,850, while the 300,000 hard requests left on Opus cost $8,250 — $12,100 in total, a 56% cut with the difficult work untouched. Build the router as a rule over request features first; a classifier that itself costs an API call has to be very cheap to be worth it. The model switch savings calculator gives the per-tier delta.
6. Cap output length, and mean it. Savings: 20–50% of output spend. Output is billed at three to eight times input on frontier models — Gemini 3.5 Flash is $1.50 in and $9.00 out — so every unnecessary sentence is expensive in a way input padding is not. Two moves: instruct for a specific length in the prompt, and set max_tokens to what the feature can actually consume. Trimming 300 tokens off an average response across 400,000 requests a month is 120 million output tokens; on Gemini 3.5 Flash that is $1,080. Structured output helps twice over, removing narrative scaffolding and making length predictable enough to cap tightly. The mechanics are in why output tokens cost more.
7. Cut the system prompt. Savings: 5–25%. System prompts accumulate: an instruction added for an edge case that was fixed two releases ago, three redundant restatements of the tone rule, a list of eleven examples where four would do. Every token is billed on every call. Going from 900 tokens to 350 across two million requests a month saves 1,100 million input tokens, $2,200 on Claude Sonnet 5. One caveat that matters: if the prefix is cached, those tokens are already costing you $0.20 per million rather than $2.00, so the same edit is worth a tenth as much — do the caching first and then decide whether the trimming is worth the regression testing.
8. Deduplicate identical requests. Savings: 5–40%, entirely dependent on traffic shape. Public-facing features see the same input repeatedly: the same URL summarised, the same product classified, the same FAQ asked in the same words. A hash of the normalised prompt against a key-value store with a sensible TTL is an afternoon's work and permanently removes that fraction of traffic from the bill. A public URL-summarisation endpoint running 250,000 requests a month at 6,000 in / 400 out on GPT-5.6 Terra costs 1,500 million × $2.00 + 100 million × $12.00 = $4,200; a 22% duplicate rate means $924 of that is the same answers computed again. Measure the duplicate rate before building it — a feature with 3% duplicates does not justify a cache layer, one with 25% does, and you cannot guess which you have. This is distinct from prompt caching: here you skip the API call entirely and pay nothing rather than paying a discounted rate.
The ones that matter once the bill is already large
9. Strip boilerplate out of retrieved chunks. Savings: 10–30% of retrieval input. Chunks pulled from HTML, PDFs or wikis carry navigation, cookie banners, headers, footers, licence blocks and repeated document titles. In a typical scrape, 15–25% of every retrieved chunk is furniture, and you pay input rate for it on every single request that retrieves that chunk. Concretely: eight chunks of 700 tokens retrieved per request, a fifth of it furniture, is 1,120 wasted input tokens each time; across 300,000 requests a month on Claude Sonnet 5 that is 336 million tokens and $672 for cookie banners. Clean at ingestion, not at query time, so the cost is paid once. The same pass should collapse whitespace and drop base64 blobs, which tokenise brutally.
10. Kill retry storms with proper backoff. Savings: 2–15% normally, far more during an incident. A failed request that timed out after generating 400 tokens is billed for those tokens. A client that retries three times with no jitter turns a provider hiccup into three times the spend and, because your retries add load, often a longer hiccup. Put numbers on it: 20,000 failures a month, each burning 2,000 input and 350 output tokens before dying, retried three times, is 60,000 wasted calls — 120 million input and 21 million output tokens, $240 + $210 = $450 a month on Claude Sonnet 5 for responses that never reached a user. Exponential backoff with jitter, a retry budget per request, and a circuit breaker that stops calling a failing endpoint are standard practice for every other API and get skipped for LLM clients surprisingly often. Check what your SDK's defaults actually are; several retry twice before your own code sees an error.
11. Instrument per-feature spend. Savings: nothing directly, and it is why the other ten are guesses without it. Tag every API call with the feature, the model, the prompt version and the caller, and record input tokens, output tokens, cached tokens and reasoning tokens separately from the response metadata. Aggregate daily. Within a week you will know which feature owns which share of the bill, and the ranking is reliably surprising — the demo nobody uses that quietly runs a 40,000-token context, the retry loop on a deprecated endpoint, the internal script one team runs hourly. Without this you are optimising the feature you happen to be thinking about; with it you optimise the one that costs money. Price the candidate changes against the LLM API cost calculator before you build them, then verify against the tags afterwards.
Where to start
Work down this list in order and stop when the bill stops mattering. Most teams find the first four are enough.
All figures in this guide come from the price index, last verified 15 August 2026. Every entry links to the vendor page it was read from — see the price index and the methodology.