The crossover question looks simple: at what monthly token volume does running a model on your own hardware beat paying per token to a hosted API. In practice it has four answers depending on which model class you compare, and there are workload shapes that never cross over at all.
Verified August 2026 against current vendor pricing pages, the Hugging Face API for model metadata, and the Ollama library for tag sizes.
Two Different Cost Curves
Hosted APIs are pure variable cost. You pay for every input token at one rate and every output token at another, the rates change when the model changes, and there is no hardware purchase to amortise. The Anthropic price page lists Claude Sonnet 5 at $2 per million input tokens and $10 per million output tokens; Claude Opus 5 at $5 and $25; Claude Haiku 4.5 at $1 and $5 (Anthropic pricing). The OpenAI page lists GPT-5 at $1.25 and $10, GPT-5-mini at $0.25 and $2.00, GPT-5-nano at $0.05 and $0.40, GPT-4.1-mini at $0.40 and $1.60, GPT-4o at $2.50 and $10 (OpenAI pricing). The Gemini Developer API lists Gemini 2.5 Pro at $1.25 and $10, Gemini 2.5 Flash at $0.30 and $2.50, and Gemini 2.5 Flash-Lite at $0.10 and $0.40 (Google AI Studio pricing). Both vendors also offer a 50% Batch discount on input and output, and the Anthropic page documents cache reads at one tenth of the standard input rate, so any workload that reuses a system prompt cuts the input bill by an order of magnitude before it runs.
Local inference is fixed cost plus electricity. You buy or already own the hardware, you keep it, and the marginal cost per token is whatever the power draw multiplies by. The U.S. average retail electricity price for 2025 was about 13.63 cents per kilowatt-hour (EIA electricity prices). The crossover is the monthly token volume at which the hosted bill catches up to the amortised hardware cost plus the electricity it costs to run.
Choose a Model Class, Not a Single Model
The first mistake is comparing one specific local model to one specific cloud model. The hosted APIs span roughly three price tiers - cheap (the Haiku / Flash-Lite / GPT-5-nano band), mid (Sonnet / GPT-5 / Pro), and top (Opus / GPT-5.x-pro) - and a self-hosted box can plausibly match any one of them if you size it right. Run the crossover calculation for each band separately.
| Hosted tier | Example model | Input $ / MTok | Output $ / MTok | Closest local match | Local download size |
|---|---|---|---|---|---|
| Cheap | Gemini 2.5 Flash-Lite | $0.10 | $0.40 | Qwen3 4B (qwen3:4b) | 2.5 GB (Ollama) |
| Cheap | GPT-5-nano | $0.05 | $0.40 | Qwen3 4B | 2.5 GB |
| Mid | Gemini 2.5 Pro | $1.25 | $10.00 | Qwen3 32B (qwen3:32b) | 20 GB (Ollama) |
| Mid | Claude Sonnet 5 | $2.00 | $10.00 | Qwen3 30B-A3B (MoE) | 19 GB (HF, Ollama) |
| Top | Claude Opus 5 | $5.00 | $25.00 | Qwen3 235B-A22B (MoE) | 142 GB (HF, Ollama) |
Sonnet-class work is the band that matters for most readers. The argument below uses it as the worked example.
The Arithmetic, Then the Catch
Take a single mid-tier card that can hold a 30B-class mixture-of-experts model: qwen3:30b-a3b at 19 GB fits on a 24 GB card with room for context. Approximate the all-in hardware outlay at the published launch price of the card plus a power supply, case and storage; treat that as a one-time cost. The model itself is free (Apache-2.0) and the runtime is free (llama.cpp is MIT).
Assume the box draws roughly 350 watts while running, including the wall-plug overhead that the EIA’s retail price is measured against. At 13.63 cents per kWh (EIA), one million output tokens generated continuously is in the same order of magnitude as a few cents of electricity - far below the $10 the same million output tokens cost on Claude Sonnet 5. The hosted bill dominates for the first few months while the hardware is being amortised. After that, every additional token is electricity and almost nothing else.
So the crossover is real. The catch is the four things this arithmetic hides.
- Hardware is not free. You can be a few hundred dollars in or a few thousand dollars in depending on whether you already own a capable GPU. The crossover point in months scales linearly with what you spent.
- You are not buying one model. VRAM-tier guides on this site list what fits on each card class. The model you want to run dictates the card you need to own, not the other way around.
- Throughput is bounded by what you bought. A single consumer card can serve one user or a handful of concurrent users comfortably. A busy multi-seat installation looks like a different problem and a different bill.
- Quality is not the same model.
qwen3:30b-a3bis not Claude Sonnet 5. It is roughly the same size class on paper, with 3B active parameters per token, but the quality gap matters to some workloads and not to others. For work where the gap is real, the crossover against the cheap tier is the one to compute, not against the mid tier.
Workloads That Never Cross Over
Three workload shapes keep hosted APIs the right answer regardless of volume.
Low volume, any model. If you make a few requests a day, you will not amortise the hardware before it ages out. Hosted APIs are the rational answer for individuals and small teams below roughly a million output tokens per month, which is the volume at which the electricity on a self-hosted box stays in the noise.
Sporadic bursty demand. If usage is heavy for one week a quarter and zero the rest, the API wins on average. The local machine either sits idle or is not sized for the burst. Batch API at 50% off is the explicit hosted answer to this pattern (OpenAI pricing, Anthropic pricing).
Very large models. qwen3:235b-a22b is 142 GB at the default Q4 quantisation (Ollama). That is not a single-card build, it is a multi-GPU rig or a workstation-class machine, and the crossover point moves with the hardware bill. For this class the hosted top tier often stays cheaper than self-hosting unless your volume is genuinely high.
Workloads That Always Cross Over
Three other shapes reliably make the self-hosted side win, because they let the same hardware earn its keep on demand patterns the API penalises.
Long system prompts reused constantly. Coding assistants, RAG, document chat and any tool where the same context is sent every turn. The Anthropic page documents cache reads at 10% of standard input; if you can hit the cache most of the time, the hosted bill drops sharply. If you cannot, the local box wins sooner because every token it serves is at electricity cost from request one.
High-volume chat with no caching. A busy in-house chatbot, a customer support tool, or any pipeline that processes text all day. Once you are above roughly tens of millions of output tokens per month, the hosted mid-tier bill on Sonnet 5 ($10 per million output) has overtaken the amortised cost of the card that runs qwen3:30b-a3b.
Sensitive data where egress is a constraint. Health, legal, internal documents. Hosted APIs route your prompt through someone else’s infrastructure, and even with zero-retention guarantees the routing itself is the cost you are paying. The crossover against hosted is not just dollars; it is the question of whether you want the prompt to leave the building at all. The local AI hub starts from the hardware side; the no-GPU guide starts from machines that already exist.
Bottom Line
The crossover exists. For a mid-tier self-hosted model against a mid-tier hosted API, with a single consumer-class GPU and electricity at the 2025 U.S. average, it lands somewhere between a few months and a year of moderate-to-heavy use, with the exact figure driven by how much you spent on the hardware and how cache-friendly your workload is. Below that volume the API is cheaper and simpler. Above it, electricity is.
The honest limits are the ones that do not show up in the arithmetic: the local model is not the hosted model, throughput is bounded by the card you bought, very large models need very large hardware, and the cheapest workloads on the API side are exactly the workloads the hosted vendors have already cut prices to defend. Run your own numbers on your own usage. The vendors publish their rates; this site will not pretend to know your volume better than you do.