There is no crossover number, none that survives a month or transfers between readers. Whether running a model on your own hardware beats paying per token depends on tokens per day, the input-to-output ratio of that traffic, whether the GPU is dedicated or shared with other work, and the throughput your hardware and quantisation actually reach. Move any one and the answer moves.
One caveat outranks the arithmetic: for a large share of readers, cost is not the deciding variable. Privacy, offline operation and rate-limit independence decide first, and plenty run locally where the API would be cheaper, or stay on the API rather than run hardware. A search for a published analysis that does this arithmetic turns up SEO pages and calculator farms, nothing citable. What follows is a method, not a verdict.
Why Any Single Figure Has a Shelf Life
Two vendor cases carry it. Anthropic’s pricing documentation, read 2026-08-16, states that Claude Sonnet 5’s introductory $2/$10 per million input/output pricing is now standard and that a scheduled increase to $3/$15 on 1 September 2026 will not occur: a rise announced, then cancelled. Google’s Gemini API pricing page, read the same day, runs the other way, listing Gemini 3.7 Flash input at $0.75 per million through 31 December 2026 and $1.50 from 1 January 2027, output at $3.75 rising to $7.50. A doubling, published in advance with a date on it.
Hardware is worse, because the only verifiable figures are launch-day announcements: the RTX 4090 at $1,599 in September 2022, the RTX 4060 Ti 16GB at $499 in May 2023, and the RTX 5090 and 5080 at $1,999 and $999 in January 2025. Those are historical launch prices, and no current retail price is implied: none could be verified, so check a retailer the day you run your numbers, against a tier-by-tier view of the cards.
The Components on Each Side
| Component | Running it yourself | Paying an API |
|---|---|---|
| Capital | purchase price over a life you pick, times the share allocated to inference | none |
| Energy | load watts x hours generating, plus idle watts x idle hours, x price per kWh | inside the token rate |
| Input and output tokens | no marginal charge | separate per-million rates |
| Repeated prefixes, deferrable work | no marginal charge | cache-hit rate, batch rate |
| Ceiling | measured tokens/s x hours available | rate-limit tier |
Input and output are priced apart, so a workload’s ratio matters as much as its volume: on the Gemini page read 2026-08-16, Gemini 2.5 Pro listed $1.25 per million input for prompts up to 200k tokens against $10.00 output, and the archived OpenAI table captured 2026-08-12 listed gpt-5.6-sol at $5.00 input against $30.00 output on the standard tier’s short-context column, with the long-context column on the same row at $10.00 and $45.00.
Two discounts reshape it. Anthropic’s page, read 2026-08-16, listed Sonnet 5 cache hits at $0.20 per million against $2 base input, and batch processing at $1 input and $5 output against $2 and $10 standard, so a workload replaying one long fixed prefix, or one that tolerates delay, prices differently from fresh synchronous input.
The local column has two sourceable terms: NVIDIA’s spec pages, read 2026-08-16, list the RTX 4090 at 450W Total Graphics Power with 19W idle and the 4060 Ti and 4060 at 7W idle with 165W and 115W TGP, though NVIDIA’s own footnote defines those idle figures as measured at the Windows desktop, and a card holding model weights resident in VRAM idles above that, so the published number is a floor rather than the term to use, while the 3090 and 5090 pages give 350W and 575W with no idle figure, so measure that yourself. The EIA’s Electric Power Monthly, Table 5.6.A, read 2026-08-16, put the US residential average at 18.44 cents per kWh for May 2026, its latest published month, on a table running from 12.35 cents in Idaho to 52.00 in Hawaii, so treat it as a placeholder for your own bill.
The Shape of the Arithmetic
Per month, with every slot filled by you:
Local = (purchase price x inference share / useful life in months) + (load watts / 1000 x hours generating x price per kWh) + (idle watts / 1000 x idle hours x price per kWh) + cooling
API = fresh input at the input rate + cached input at the cache-hit rate + output at the output rate, each per million tokens, with batch rates for work that tolerates a delay
The crossover is where those lines meet. Allocation is the term most comparisons drop: a gaming card that also serves a model in the evenings carries almost no capital cost against inference, while one bought to serve models carries all of it. Same hardware, two answers. Cooling and the opportunity cost of a busy card belong in the local column, unquantified because no source put a number on either.
The Term That Can Rule Local Out
Local capacity is measured tokens per second times the hours the machine is available. If daily demand exceeds that at the model size you need, there is no local option to price.
The llama.cpp project keeps benchmark tables in its own repository for CUDA and Apple Silicon, contributor-submitted, the CUDA table carrying a commit hash per row and the Apple table one pinned llama-bench invocation for every entry, so both can be re-run. On the CUDA table (Llama 2 7B, Q4_0, llama.cpp, flash attention off), generation runs at 290.02 tokens/s on an RTX 5090, 186.21 on a 4090, 176.85 on a 5070 Ti and 158.16 on a 3090, while prompt processing over a 512-token prompt runs 14,073, 11,993, 6,952 and 5,175 tokens/s. Prefill is parallel and compute-bound, decode sequential and bandwidth-bound, so input-heavy work is bounded by the prompt-processing figure, output-heavy work by the generation figure.
The Apple Silicon table (Llama 2 7B, Q4_0, llama.cpp Metal backend, its reproduction command specifying llama-7b-v2, opened and curated by the project’s creator) shows generation tracking memory bandwidth: 94.27 tokens/s on a 76-core M2 Ultra at 800 GB/s against 65.95 on a 38-core M2 Max at 400 GB/s. Core count moves with bandwidth across those rows, so read it as the two together rather than bandwidth alone. Both tables fix an old model deliberately and the Apple M5 rows are unfilled, so take the ranking, not the absolutes.
What Actually Decides It
Model size hides inside both columns. If a 7B or 8B class model serves the job, the comparison is real; if it needs a frontier hosted model, it is a smaller local model against a larger hosted one, a capability trade-off in cost clothing. Settle fit first, with which models fit your VRAM and the format you quantise to; with no discrete GPU at all, the CPU-only case changes the capital term, not the method.
Then the non-cost factors. Anthropic’s privacy article on whether inputs are used for training, last updated 16 March 2026 and read 2026-08-16, states that by default the company will not use inputs or outputs from its commercial products, the API included, to train its models. Other vendors’ pages were not verified here, so nothing is claimed about them. Rate limits are tiered and scale with account standing, with no threshold quoted because none could be retrieved. A local model needs no network.
What This Means
The method stands in for a figure because the figure would already be stale: inside one month one vendor cancelled a scheduled price rise and another published a doubling with a date attached, and the only verifiable hardware prices are launch announcements from 2022, 2023 and 2025. A single crossover number would inherit all of them. What lasts is the structure: capital over life and allocation, energy under load and at idle, against per-token input and output rates adjusted for caching and batching, both sides capped by throughput.
The Bottom Line
There is no crossover figure to look up, only components on each side, a shape for the arithmetic, and a rule that every price in it carries the date it was read. For many readers it settles nothing anyway: privacy, offline operation and rate-limit independence decide first, and cost sorts out what is left.