Half price is an unusual discount to find sitting unclaimed on a pricing page. Every OpenAI, Anthropic and Google model in our index carries a 50% batch discount, applied to both input and output, with no commitment, no negotiation and no minimum spend. The only thing you give up is the right to have the answer immediately.
On the calculator's defaults — 3,000 input tokens, 700 output tokens, 400,000 requests a month on GPT-5.6 Terra at $2.00/$12.00 — a request costs 3,000 ÷ 1,000,000 × $2.00 = $0.006 for input plus 700 ÷ 1,000,000 × $12.00 = $0.0084 for output, so $0.0144 all in. Run everything in real time and that is $5,760 a month. Move 60% of it to the batch tier and the batched 240,000 requests cost $0.0072 each — $1,728 — while the remaining 160,000 stay at $2,304. Total $4,032, a saving of $1,728 a month, $20,736 a year, for a scheduling change.
What a batch endpoint actually is
It is not a different model, a smaller model, or a quantised one. You get the same weights and the same output quality. What changes is the submission mechanism: instead of one HTTP request per prompt held open until the tokens stream back, you upload a file containing many requests — typically JSONL, one request per line, each with an identifier you choose — and receive a job handle. You poll that handle. When the job completes you download a results file and match the outputs back to your inputs by identifier.
The turnaround window is the contractual part. Vendors commit to completing the job within a stated period, usually 24 hours, and in practice most jobs finish far sooner — small batches often return within minutes. But "often within minutes" is not a guarantee you can build a user-facing feature on. Treat the published window as the number, and design as though every batch will take the full length of it. Batch jobs also generally draw on separate, more generous rate limits than the real-time endpoint, which is a second, less-advertised reason to use them: a backfill that would trip your interactive quota for hours can run alongside production traffic without competing for it.
Why the vendor is willing to halve the price
Inference fleets are sized for peak interactive demand, which means they are substantially idle at other times. That idle capacity cannot be stored — an unused accelerator-second is gone. Batch work is schedulable: it can be run at three in the morning, deprioritised behind real-time traffic whenever a spike arrives, paused and resumed, and moved between regions. It also runs more efficiently. Batching many sequences together raises the arithmetic intensity of each forward pass, so the fleet delivers more tokens per accelerator-second on batch work than on latency-sensitive single requests where the hardware waits on memory bandwidth.
So the 50% is not a loss leader or a promotional rate. You are being paid for scheduling flexibility, which has genuine value to the operator, and that is why the discount has been stable across three vendors rather than appearing and disappearing. It is also why the models without one — DeepSeek, Mistral, xAI's Grok 4.6, and the Together- and Fireworks-hosted open-weight models in our index — are not simply being stingy: several are already priced near the floor. DeepSeek V4 Flash at $0.14/$0.28 does not have much room to halve.
Auditing what can wait
The share slider is the field people fill in wrongly, always in the conservative direction. The test is narrow and specific: is a human sitting in front of a screen, blocked, waiting for this particular response? If not, it is batchable.
Applied honestly, that usually catches more than expected. Enrichment and classification pipelines that tag incoming records. Embedding a corpus for retrieval. Evaluation harnesses, which frequently run tens of thousands of prompts and are the single most batchable workload most teams have. Nightly report and digest generation. Moderation queues where a decision is reviewed by a person hours later anyway. Translating a content library. Generating alt text, product descriptions, or metadata for an existing catalogue. Summarising yesterday's support tickets. Generating synthetic training data. Backfilling a feature across historical records after launch.
What genuinely cannot wait is narrower than the org chart suggests: interactive chat, autocomplete, anything in a synchronous web request path, and agentic loops where each tool call feeds the next. Note that the last one is real — an agent doing eight sequential steps cannot batch, because step five does not exist until step four returns.
Where a workload sits on the boundary, the option is not binary. A "results within an hour" feature can run as a batch submitted every fifteen minutes — a product decision worth putting in front of whoever owns the feature rather than settling silently in engineering.
Stack the discount with a model change and the numbers get more dramatic than either lever alone. The same 3,000/700 workload on Gemini 2.5 Flash at $0.30/$2.50 costs $0.00265 a request in real time; batched, $0.001325, or $530 a month for all 400,000 requests against $5,760 on real-time GPT-5.6 Terra. Batch tiers are best claimed on the workloads where a smaller model was already viable — which, since batchable work is mostly classification, extraction and enrichment, is most of them. The LLM API cost calculator has a batch checkbox that prices this combination across all 27 models at once.
Partial failures, which will happen
A batch of 50,000 requests will not return 50,000 clean results. Some lines fail: a prompt that exceeds the context window, a response blocked by a content filter, a malformed row, a transient internal error. Build for it from the start.
Match results to inputs by the identifier you supplied, never by line order — results files are not guaranteed to preserve input ordering, and a pipeline that assumes they do will silently attach the wrong output to the wrong record, which is far worse than a crash. Read the per-request status on every line, not just the job-level status, which reports success for a job that returned 49,000 results and 1,000 errors.
Collect failures into a retry queue and re-submit them as a second batch. The instinct to fall back to the real-time endpoint on failure is what erases the discount: a 5% failure rate handled that way costs you 5% at full price, which is tolerable, but a bad deploy producing a 40% failure rate turns your cheap job into an expensive one automatically and without warning. Cap the fallback.
Finally, decide what happens when the window expires. Vendors return unprocessed requests unbilled rather than running them late, so an expired job leaves you with a partial result set and a decision to make — and re-submitting a whole batch to be safe means paying twice for everything that already succeeded. The batch processing guide covers the job lifecycle and file formats in detail.
Prices used by this calculator were last verified on 15 August 2026 from the vendors' own pricing pages. See the full price index for every figure and its source, the change log for what has moved recently, and the methodology for how the index is maintained and where its limits are.