Calculator
Llama 3.3 70B Instruct: self-host or API?
Self-hosting
Published minimumThe API is cheaper below 654 million tokens a day. Above that, self-hosting wins only if the whole configuration (all GPUs together) delivers at least 15,147 tokens per second, input and output together, while serving at 50% utilization (7,574 if it were busy all the time). A published benchmark (InferenceX, SemiAnalysis) gives a floor of 2,442 tokens per second, input and output together, for a 3:1 input:output mix; the real capacity may be higher, so this does not settle it.
At 50% utilization · Cheapest API offer: $0.095 per million tokens at NanoGPT (3:1 input:output mix)
Hardware rental only: the hourly price the provider publishes for the machine. It does not count the engineering to deploy and run the model, monitoring, redundancy, start-up time, or storage and network costs billed separately.
- Break-even volume
- 654.32 million tokens a day
- Throughput self-hosting needs
- 15,147 tokens per second
- At 50% utilization
- Throughput self-hosting needs
- 7,574 tokens per second
- If busy all the time
At least this many input plus output tokens per second, all GPUs together, while serving.
What published figures show
- At least 1,221 output tokens per second, according to InferenceX (SemiAnalysis)
- We convert this to at least 2,442 tokens per second, input and output together
- Engine of the run: vLLM
- Public run the figure comes from (opens in a new tab)
Rented configuration
1 x MI300X 192GB at DigitalOcean
$62.16 a day · $2.59 an hour · Guaranteed · 192 GB of GPU memory · fp8
Checked Oct 31, 2026 Price page at the provider (opens in a new tab)
Calculation detail
Cheapest of 26 current standard offers, promotions, unstable prices, off-peak prices without published hours and unverified prices far below those of the other providers excluded Checked Oct 31, 2026
Assumptions
- The InferenceX benchmark (SemiAnalysis) served this many output tokens per second, and as many input tokens (1,024 in / 1,024 out). For a mix with at least as much input, we assume reading a prompt token costs no more than generating one, and count it as equal (the least favourable case): at least twice the output figure in input plus output tokens.
- Hardware rental only: the hourly price the provider publishes for the machine. It does not count the engineering to deploy and run the model, monitoring, redundancy, start-up time, or storage and network costs billed separately.
- Configuration: we pick the cheapest one whose weights fit. If a configuration costing up to 25% more has a measured throughput, we pick it instead; failing that, one with a derived throughput; failing that, one with a published lower bound that settles the verdict; among equals, the cheaper. A configuration whose throughput is only a wide estimate is never preferred over the cheapest. This is our rule, not a technical requirement.
- Quantization fp8; guaranteed tier prices as published by the provider; utilization 50%.
- Lower bound: the benchmark's series (InferenceX, SemiAnalysis) stopped while throughput was still rising, so its output figure is a floor of the real capacity. It can decide that self-hosting wins, never that the API is cheaper. It was measured with 1,024-token prompts and 1,024-token answers: longer requests are slower.
- Throughput is aggregate (input and output tokens, 3:1). Published figures in generated tokens are converted with a prefill/generation speed ratio of 3 (prudent), 5 (median) and 8 (optimistic).
- Break-even = daily GPU cost / API price per million tokens (654 million tokens a day). Self-hosting is cheaper only if the average utilization you can sustain exceeds break-even / raw capacity (the break-even utilization of each scenario), whatever the number of GPUs.