Run the calculator's defaults and the two options land $153.88 apart on a monthly bill of roughly $6,200. Two RunPod H100 SXM cards at $3.29 an hour cost $3.29 × 2 × 730 = $4,803.40 a month, plus $1,500 for the fractional engineer keeping the thing patched and monitored, giving $6,303.40. At 900 output tokens per second per replica, two replicas and 25% average utilisation, that deployment produces 1,800 × 730 × 3,600 × 0.25 = 1.18 billion output tokens a month, and at a 4:1 input ratio, 4.73 billion input tokens. Buy the identical volume from Together's hosting of Llama 3.3 70B at $1.04/$1.04 per million and you pay 4,730.4 × $1.04 + 1,182.6 × $1.04 = $6,149.52.
A dead heat. And the reason to start there rather than with a headline is that everything downstream of that near-tie is decided by one input: utilisation.
Move it and watch. At 50% utilisation the same hardware produces 2.37 billion output tokens for the same $6,303.40, which is $2.67 per million output tokens against the API's fixed $5.20 — self-hosting wins by roughly half. At 10% it produces 473 million tokens, or $13.32 per million, and the API wins by 2.6×. The break-even sits at 25.6% utilisation: below it, rent tokens; above it, rent hardware. The hardware bill does not move at all across that entire range. That is the whole argument. A GPU you have reserved bills identically whether it is saturated or idle, so self-hosting is a bet that you can keep it busy, and every self-hosting business case that goes wrong goes wrong on exactly that bet.
So what utilisation number should you type? Not your peak, and not your daytime average. Your fleet is sized for peak concurrency, then sits through nights, weekends and the long trough between peaks in a single time zone. A consumer product with a European user base is meaningfully busy perhaps six hours a day, five days a week — 18% of the hours in a month before any headroom for spikes. Teams routinely put 60% or 80% into this field because that is what their load test showed. Production deployments with real diurnal traffic land between 10% and 30%, which is precisely the range where the default comparison above is a coin flip.
Throughput per replica is a batched number, not a speed
The field labelled "throughput per replica" is the other input that gets entered wrongly, and it is worse than the utilisation error because it silently multiplies everything.
What you want is aggregate server throughput: total output tokens per second across all concurrent requests, under a serving stack doing continuous batching. What people enter is single-stream speed — what one user sees, which is what benchmark posts report and what you observe when you curl the endpoint yourself.
Those are not close. Generating tokens for a single sequence is memory-bandwidth-bound: the accelerator loads the entire weight matrix to produce one token, and the arithmetic units idle. Serve 64 sequences concurrently and you load those same weights once to produce 64 tokens. Throughput scales close to linearly with batch size until you exhaust KV cache memory, while per-user speed degrades only modestly. A 70B-class model on a single H100 might give one user 40 tokens a second and still deliver 900 or more in aggregate under load.
Enter 50 instead of 900 and the model tells you break-even utilisation is above 100% — that no amount of traffic justifies the hardware. Enter 3,000 because you read it in a vendor benchmark run at batch size 256 with 128-token outputs, and you will conclude self-hosting is free money. Measure it: run your own serving stack, at your own sequence lengths, at the concurrency you actually expect, and read the aggregate figure off the server's own metrics.
Redundancy doubles the floor and does not raise the ceiling
The replicas field defaults to 2 because one replica is not a production deployment. A single GPU host means every kernel upgrade, driver bug, node preemption and out-of-memory crash is a full outage, and there is nowhere to drain traffic to during a deploy.
Going from one replica to two takes the GPU line from $2,401.70 to $4,803.40 a month — the floor doubles, whether or not the second machine ever serves a request. Whether your cost per token also doubles depends entirely on how you use it. If both replicas take live traffic behind a load balancer, capacity doubles too and the per-token cost is unchanged; break-even utilisation stays at 25.6% and you have bought availability for free in unit terms. If the second is a hot standby carrying nothing, you pay for two and get the throughput of one: at 25% utilisation of a single active replica, 591 million output tokens against $6,303.40 is $10.66 per million — twice the API's $5.20, with no compensating benefit.
Run active-active. It is the difference between redundancy being free and redundancy being the reason the project failed its business case.
The hourly rate is worth shopping too, since it is a straight multiplier on the floor. The same H100 SXM is $3.29 at RunPod, $3.99 at Lambda, $3.99 at Together and $6.155 per GPU at CoreWeave. Swap RunPod for CoreWeave in this scenario and the monthly total goes from $6,303.40 to $10,486.30, pushing break-even utilisation from 25.6% to 42.6% without changing a single thing about your workload. Vast.ai's marketplace H100s start at $1.60, which takes the total to $3,836 and break-even down to 15.6% — with the caveat that marketplace capacity is interruptible and heterogeneous, which is a real operational cost this calculator cannot price.
What is deliberately not in this model
The comparison is hardware and tokens. It excludes the engineer-months to build and harden a serving stack; the evaluation harness you need to prove the open-weight model is good enough for your task, which is work you would not otherwise do; licence review for the model weights; storage and egress for multi-hundred-gigabyte checkpoints; cold-start time when you scale up; and the cost of being paged at three in the morning by something no vendor will fix for you. The $1,500 ops figure is a placeholder for a fraction of one person, and for a genuinely 24/7 service it is optimistic.
It also cannot price the quality gap. Llama 3.3 70B is not Claude Opus 5, and if that substitution costs accuracy, the comparison on this page is not the right one. Self-hosting wins on sustained high volume, on data residency, and on workloads where an open-weight model is genuinely sufficient. It rarely wins on convenience, and it never wins on a workload that is idle two-thirds of the time. The self-hosting reality check goes through the operational side, hidden costs of AI covers the line items above, and the LLM API cost calculator prices the API half against every other model in the index.
Prices used by this calculator were last verified on 15 August 2026 from the vendors' own pricing pages. See the full price index for every figure and its source, the change log for what has moved recently, and the methodology for how the index is maintained and where its limits are.