LLM cost forecasting for a bill that has not happened

By John M. · Updated 15 August 2026 · 7 min read

Finance does not want to know what a token costs, and is right not to. A forecast expressed in tokens per million cannot be checked against anything in the plan, cannot be challenged by anyone outside the engineering team, and cannot be reconciled at month end. A forecast expressed as a cost per document processed can be all three. The translation between those two things is the entire job.

Here is a forecast built the way a finance team will accept it, for a product that extracts structured data from uploaded documents.

One number is not a forecast

Never present a single figure. A single figure is a prediction, and predictions in this category are wrong by factors, not percentages, because the volume driver is unknown and the model mix is still moving. Present three scenarios, each anchored to a volume assumption someone in the business already forecasts for other reasons: documents uploaded, tickets resolved, monthly active users. If the AI forecast rides on a number that already exists in the plan, it inherits that number's credibility and its review cycle. If you invent your own volume driver, you own defending it every quarter.

The three scenarios are not low, medium and high guesses at the same thing. They are three different worlds, each with a stated condition. Conservative means the feature is opt-in and adoption stalls. Base means it ships on by default at the currently forecast document volume. Aggressive means it becomes the primary ingestion path and the enterprise pilot converts.

Build the unit cost first

Everything hangs off one number: what it costs to process one document.

The pipeline reads about 3,600 tokens of extracted text per eight-page document and carries a 900-token system prompt and schema, so 4,500 input tokens. It emits about 1,100 tokens of structured JSON. On Gemini 3.1 Pro at $2.00 input and $12.00 output per million:

• input: 4,500 x $2.00/1,000,000 = $0.0090
• output: 1,100 x $12.00/1,000,000 = $0.0132

A second pass validates the extraction against the source on Gemini 2.5 Flash-Lite at $0.10/$0.40, reading 5,600 tokens and emitting 200: 5,600 x $0.10/M = $0.00056 plus 200 x $0.40/M = $0.00008, so $0.00064. Add 4% for retries and the pipeline costs $0.02284 x 1.04 = $0.0238, call it $0.024 per document.

That is the sentence to take upstairs: processing a document costs 2.4 cents. Everything else in the forecast is that number multiplied by a volume the business already predicts. Check it against your own model mix in the LLM API cost calculator before you commit to it.

The three scenarios

Look at the third column. It does not move.

Flat unit cost is the whole story

This is the single most important thing to communicate upward, and it is the point at which AI spend stops behaving like the infrastructure line it is sitting next to in the budget.

Traditional infrastructure has economies of scale baked into it. Ten times the rows in a database does not cost ten times as much, because you were paying for a machine with headroom, and much of the bill was fixed. Engineers and finance teams have twenty years of instinct built on that curve, and the instinct is that unit costs fall as you grow.

Token pricing has no such curve. Ten times the documents costs almost exactly ten times as much, because you are buying inference by the unit and there is no idle capacity you were already paying for. Volume discounts exist at enterprise scale but they are negotiated percentages, not an order-of-magnitude bend in the line.

The consequences are worth stating plainly to anyone who signs off the budget. Growth does not fix a bad unit cost, it multiplies it. A feature that loses money at 2.4 cents a document at 40,000 documents loses 10 times as much at 400,000. And the levers that do reduce unit cost are engineering choices available at any volume, not rewards for getting bigger: the batch API at 50% off is the obvious one, and applying it to the 60% of the aggressive scenario that is genuinely asynchronous saves 400,000 x 0.6 x $0.024 x 0.5 = $2,880 a month, taking the blended unit cost to $0.0168. That saving is available at 40,000 documents too, proportionally. It is architecture, not scale.

What causes a step change

Unit cost is flat until something in this list happens, and then it jumps. Name these in the forecast so that the jump is a scheduled event rather than a surprise.

Hitting a rate-limit tier. The aggressive scenario is 400,000 x 5,600 tokens, about 2.24 billion tokens a month. Compressed into 22 working days of 10 hours that is roughly 170,000 tokens per minute on average and around half a million at peak. When you hit the ceiling you get three options and all of them change the forecast: queue the work and breach a latency commitment, add a second provider or region and pay for a parallel integration, or take provisioned capacity and convert a purely variable cost into a fixed monthly commitment. The last one is the real step, because it changes the shape of the line, not just its height.

Adding a reasoning model. Route the 20% of documents that fail validation to Claude Opus 5 at $5.00/$25.00, where 4,000 reasoning tokens are billed at the output rate alongside 1,100 visible tokens: 4,500 x $5.00/M = $0.0225 plus 5,100 x $25.00/M = $0.1275, so $0.150 per hard document. The blended unit cost becomes 0.8 x $0.024 + 0.2 x $0.150 = $0.0492. A fifth of the traffic has more than doubled the whole product's unit cost.

Adding retrieval. Six retrieved chunks of 700 tokens add 4,200 input tokens per call, or 4,200 x $2.00/M = $0.0084, a 35% increase on the base unit cost before any embedding or vector storage. Retrieval also introduces the forecast's only genuinely fixed line, the vector store, which is why it needs its own row rather than being folded into the per-document figure.

Adding a second language. Token inflation alone is mild: French or German content runs perhaps 20 to 30% more tokens than the English equivalent for the same meaning. The step change comes from what the inflation forces. If quality on that segment requires Claude Opus 5 rather than Gemini 3.1 Pro, an inflated document costs 5,850 x $5.00/M + 1,430 x $25.00/M = $0.065, and with 30% of volume in that segment the blended unit cost goes to 0.7 x $0.024 + 0.3 x $0.065 = $0.0363, up 51%. The eval set and the human review queue duplicate too, and those are usually larger than the token line.

Do not build in a price cut

The obvious temptation in a twelve-month forecast is to assume prices fall, because they have been falling. Resist it, for two reasons that pull in the same direction.

The first is that price cuts arrive as new models rather than as reductions on the model you are running, and adopting a new model is a re-qualification project with an eval sweep and a regression risk attached, not a line-item change. The second is that teams spend the saving immediately. The cheaper tier arrives, and within a quarter the prompt has grown, retrieval has been switched on, or the hard documents have been routed to something better, because the budget now permits it. Both effects mean a forecast that bakes in a 30% price decline is describing a product that will not exist.

Forecast at today's published prices, note in the assumptions that they are a ceiling rather than a point estimate, and treat any actual decline as a windfall to be spent deliberately.

Reconcile monthly, in unit terms

Divide last month's actual API spend by last month's actual documents and compare it to $0.024. Do this every month, and treat drift beyond about 15% as a defect to be investigated rather than a new baseline to be accepted. Unit cost drifts for identifiable reasons: prompts grow, retry rates climb, someone adds a few-shot example, traffic shifts toward the long tail. Each one is findable within a day if you catch it at 15% and archaeological if you catch it at 200%.

Take the three scenarios and the unit cost into the AI ROI calculator, because the number finance is actually deciding on is not $0.024 a document. It is $0.024 against whatever the document used to cost to process by hand.

All figures in this guide come from the price index, last verified 15 August 2026. Every entry links to the vendor page it was read from — see the price index and the methodology.