Here is a real sentence and the exact arithmetic this page performs on it.
Rewrite the following support ticket as a two-sentence summary for the engineering team. Keep any error codes verbatim, drop pleasantries, and flag whether the customer is blocked.
That string is 180 characters, 27 whitespace-separated words and 5 punctuation marks. The estimator computes three quantities and adds them:
- A character estimate: 180 ÷ 3.9 = 46.15 tokens, weighted at 0.55 → 25.38
- A word estimate: 27 × 1.32 = 35.64 tokens, weighted at 0.45 → 16.04
- A punctuation adjustment: 5 × 0.12 → 0.60
Total 42.02, rounded to 42 tokens, with a floor applied so the result can never fall below the word count. Divide back: 180 ÷ 42 = 4.29 characters per token, which the readout reports as prose tokenising efficiently.
Why two estimates instead of one
A pure character divisor is the standard back-of-envelope rule, and on its own it is wrong in a predictable direction. It over-counts text made of long common words, because "understanding" is one token in most vocabularies despite being thirteen characters, and it under-counts text made of short fragments, because "n't", "(", and "42" each claim a token while contributing one or two characters. A pure word multiplier has the mirror problem: it handles common vocabulary well and collapses on identifiers, URLs and numbers, where a single "word" like customer_id_48213 becomes four or five tokens.
Blending them at 55/45 cancels most of each error. The punctuation term then corrects the residual, because byte-pair vocabularies almost never merge a punctuation mark into a neighbouring word — braces, quotes, colons and commas reliably cost a token each, and dense punctuation is exactly what the character and word heuristics both miss. On ordinary English prose the combination lands within roughly 5–10% of a real tokeniser. That is good enough to size a budget and not good enough to reconcile an invoice.
Why it is not the vendor's number
There is no universal tokeniser. Each model family ships its own vocabulary, learned from its own training corpus by byte-pair encoding, and the vocabularies differ in size, in how aggressively they merge, and in how much non-English text they saw. The same paragraph can differ by 15% between two vendors' counts, and neither is wrong. On top of that, the count you are billed for includes framing your text never contains: chat role markers, message boundaries, tool and function schemas serialised into the request, and — on models that reason before answering — internal tokens billed as output that never reach your application.
This tool also runs entirely in your browser, which is deliberate. Nothing you paste is uploaded. The cost of that guarantee is that it cannot consult a vendor tokeniser, so it will never match one exactly. If you need an exact figure for a contract or a chargeback, run the vendor's own tokeniser library over a representative sample offline. If you need to know whether a feature costs $400 or $4,000 a month, this is sufficient.
What the characters-per-token readout tells you
The diagnostic is more useful than the token count, because it tells you which regime your text is in and therefore how much to distrust the estimate.
Above 4.2 — clean prose, long words, few symbols. Marketing copy, articles, transcribed speech. The blend is at its most accurate here; treat the number as within about 5%.
Between 3.4 and 4.2 — normal mixed content. Support tickets, product descriptions, chat messages with the occasional order number. Expect around 10% error, in either direction.
Below 3.4 — the estimate is soft. This is code, JSON, HTML, heavily accented text, or any non-Latin script. Byte-pair tokenisers trained mostly on English split Cyrillic, Devanagari and CJK characters into multiple byte-level tokens each, so a Japanese paragraph can cost two or three times what its character count suggests. Budget 10–15% above what this page says.
There is one case where the diagnostic itself misleads, and it is worth knowing. Feed it a compact JSON object with almost no spaces — {"user_id": 48213, "plan": "enterprise_annual", "seats_used": 137} — and the word-based half of the blend sees very few "words", so the estimate comes out low while the characters-per-token figure looks reassuringly high. A real tokeniser gives every brace, quote, colon and comma its own token and splits enterprise_annual into pieces. For structured payloads, ignore the ratio verdict and add 30% flat.
Turning an estimate into a budget
Do the multiplication at production volume before you decide the margin, because the margin only matters where the totals are large. Take a summarisation backfill: 2,000,000 documents, an estimated 4,000 input tokens and 600 output tokens each, on Gemini 2.5 Flash at $0.30/$2.50 per million.
Input: 4,000 × 2,000,000 = 8,000,000,000 tokens = 8,000 million × $0.30 = $2,400. Output: 600 × 2,000,000 = 1,200 million × $2.50 = $3,000. Total: $5,400.
Now suppose the estimate ran 12% light on input, which is entirely plausible if the documents carry more markup than the sample you pasted. Input becomes 8,960 million × $0.30 = $2,688, an extra $288. That is a 5% overrun on the total — annoying, not fatal, because output dominates and output length is set by your own instruction rather than by the tokeniser.
Reverse the shape and the margin matters far more. A retrieval feature sending 30,000 input tokens for a 200-token answer on Claude Sonnet 5 at $2.00/$10.00 costs 30,000 ÷ 1,000,000 × $2.00 + 200 ÷ 1,000,000 × $10.00 = $0.062 a call. A 12% input underestimate costs you $0.0072 a call — 11.6% of the bill. Size your safety margin against the input-heavy side of your traffic, not the average.
The rule we use: add 10% for prose, 15% for mixed content, 30% for code or JSON, and then add whatever your system prompt and tool schemas weigh, because those are sent on every request and are easy to forget entirely. Paste them into this box separately and note the number. Then take the result to the LLM API cost calculator to price it across every model at once, and read estimating tokens before launch for the sampling method that keeps a pre-launch estimate honest when you have no production traffic to measure.
Prices used by this calculator were last verified on 15 August 2026 from the vendors' own pricing pages. See the full price index for every figure and its source, the change log for what has moved recently, and the methodology for how the index is maintained and where its limits are.