A consumer GPU does not pull its published wattage 24 hours a day. It pulls whatever the workload is doing, plus what the rest of the system needs, plus a baseline when no model is loaded. The cost of running a local language model around the clock is small in absolute dollars and large in the shape it gives the rest of your electricity bill - and it is built out of figures the marketing literature does not publish together.
What TDP Actually Measures
NVIDIA publishes a single number per card under the heading “Total Graphics Power”. The RTX 4090 spec sheet lists 450 W alongside an “Average Gaming Power” of 315 W and an idle of 19 W. Those are different figures for different workloads, and large language model inference is neither of the first two. The 450 W figure is the ceiling the power supply and cooling have to support; it is what the card is allowed to draw, not what it draws.
Two consequences. A workload that does not saturate the GPU leaves headroom on the table, so “what does it cost to run” depends on whether you are always prompting the model or whether it sits idle between requests. Ollama’s FAQ defaults OLLAMA_KEEP_ALIVE to five minutes, after which an idle model is unloaded and the GPU drops back toward idle. A long-running inference server that keeps multiple models resident has a very different bill from one that loads on demand.
The Published Wattage Floor
The table below is the wattage NVIDIA publishes for the consumer cards this site’s VRAM tier index and used-GPU guide point at, plus one Apple data point for comparison. The dollar column is explained below the table.
| Card or machine | Vendor wattage figure | Source | 24/7 cost at 18.34 ¢/kWh |
|---|---|---|---|
| RTX 4060 | 115 W TGP | NVIDIA spec sheet | $15.41 |
| RTX 4070 | 200 W TGP | NVIDIA spec sheet | $26.79 |
| RTX 3080 10GB | 320 W Graphics Card Power | NVIDIA spec sheet | $42.87 |
| RTX 4080 SUPER | 320 W TGP | NVIDIA spec sheet (merged 4080 / 4080 SUPER page) | $42.87 |
| RTX 3090 | 350 W Graphics Card Power | NVIDIA spec sheet | $46.89 |
| RTX 4090 | 450 W TGP | NVIDIA spec sheet | $60.29 |
| RTX 5080 | 360 W TGP | NVIDIA spec sheet | $48.23 |
| RTX 5090 | 575 W TGP | NVIDIA spec sheet | $77.03 |
| Apple Mac Studio M5 Max | 480 W max continuous | Apple specs page | $64.31 |
| NVIDIA H100 SXM | Up to 700 W configurable TDP | NVIDIA product page | $93.78 |
The “24/7 cost” column is the monthly figure, computed as TGP (or max continuous power) times 24 hours times 365.25 days times the June 2026 US average residential rate of 18.34 cents per kilowatt-hour, divided by 1,000 to convert watts to kilowatts, then divided by 12 for the monthly share. None of those figures match what a real workload draws for two reasons, which the next sections address.
What “24/7” Actually Means in Practice
Three adjustments move the published figure in different directions, and they only partially cancel.
PSU efficiency eats first. A 80 PLUS Bronze power supply is rated 85% at 50% load; a Titanium unit is 94% at the same point. Watt-hours drawn from the wall are watt-hours delivered to the components divided by that ratio, so a 300 W component on a Bronze PSU costs about 353 W of electricity, while a Titanium costs about 319 W. PSUs also lose their efficiency at very low load - a 1000 W Titanium running a 100 W draw is far less efficient than its 50% rating, which is the whole reason to size the PSU to the workload rather than buy headroom you cannot use.
Inference is bursty. Prompt processing (the prefill pass over the input) is the heavier job, generation is lighter, and the time between requests is idle. A long chat session looks like one continuous load. An API serving many small requests alternates between fill and drain. A single-user setup that unloads models after five minutes looks like spikes of load separated by idle periods. The honest number for any of these is somewhere between the TGP and the idle figure, weighted by how often the model is doing real work.
Heat is power. Every watt drawn by the GPU becomes a watt of heat in the room. A 300 W GPU plus a 150 W CPU plus a 50 W rest-of-system is a 500 W space heater, which is roughly 1,700 BTU per hour. A small bedroom in winter does not need a separate heater when the inference box is running; the same machine in a small home office in summer is paying for the air conditioning to remove the heat it added.
The CPU-Only Path Costs Less to Buy and Costs More to Run
Local AI without a GPU is a different cost calculation. The hardware is cheaper because there is no graphics card, but the work is done by the CPU and system memory, and high-end desktop CPUs draw 120 W to 170 W TDP and can pull more under sustained all-core load. A machine built for CPU inference typically runs at higher utilisation for longer because every token has to grind through general-purpose silicon rather than tensor cores.
The AMD and Intel CPU wattage figures are published on each chip’s product page; the dollar calculation is the same shape as the GPU table. For an always-on CPU-only inference box, expect to spend less per month than the 300 W+ GPU equivalent but more than an idle workstation.
Where the Numbers Break
Three places the wattage table lies to you, in order of how badly.
TDP is not a measurement. It is the thermal envelope the cooling has to handle. The NVIDIA H100 product page makes the gap explicit with “Up to 700 W (configurable)” for the SXM variant - it lists a configurable operating point, not a measured consumption. Treating TDP as a measured figure multiplies the cost on the high end and overstates efficiency on the low end.
Workload mix matters more than card choice. The same RTX 4090 running constant full-context prefill costs near its 450 W TGP every hour. Running the same model with prompt caching, streaming, and idle gaps between requests might average a third of that. The card’s published wattage is the ceiling of how much it can pull, not a description of how much it does pull.
US averages are not your rates. The EIA residential average covers everyone from Pacific Northwest hydro to New England winter oil heat. Hawaii and California run well above 30 ¢/kWh; parts of the Midwest run below 12 ¢/kWh. At 12 ¢/kWh the RTX 4090 column drops to about $39.46/month; at 35 ¢/kWh it rises to about $115.10/month. Use your own utility bill, not the national average.
Bottom Line
The published wattage figures answer one question well and two badly. They tell you what power supply and circuit you need; they do not tell you what your bill will look like, because they assume the worst case rather than the steady state. A realistic 24/7 local-AI electricity bill for a single 300 W-class GPU at US average residential rates is in the order of tens of dollars a month, not hundreds - and that is before accounting for the duty cycle, the PSU efficiency curve, and the fact that the room is now warmer.
The honest decision is not “can I afford the GPU”. It is “what size card does the work I actually do, what is my local electricity rate, and is the heat load something I want in the room”. The VRAM tier index answers the first question; this page answers the other two, and the answer for almost everyone reading this is that the electricity is a smaller number than the card.