Two numbers decide whether a graphics card is any good for running models at home, and buyers routinely optimise the wrong one.
The first is memory capacity. VRAM sets what you can load at all: a model whose weights plus key-value cache exceed the card either refuses to run or spills into system memory and crawls. The second is memory bandwidth, which sets how fast text comes out once the model is loaded. NVIDIA’s inference optimization guide splits the work in two: reading your prompt is a parallel matrix-matrix operation that “effectively saturates GPU utilization”, while generating the answer is “a matrix-vector operation that underutilizes the GPU compute ability”, one where “the speed at which the data … is transferred to the GPU from memory dominates the latency”.
So capacity decides which models are available, bandwidth decides how quickly they answer, and long prompts lean on compute in a way short chat turns do not. Two 16GB cards can sit in the same tier and behave nothing alike. Every figure below is the manufacturer’s published specification, linked; none of it is a benchmark and no first-hand timing is claimed.
The Options, By What They Unlock
| Card | VRAM | Memory bandwidth | What it unlocks | The honest catch |
|---|---|---|---|---|
| GeForce RTX 5090 | 32 GB GDDR7 | 1,792 GB/s | The largest single-card tier, with the highest bandwidth here | 575W board power and NVIDIA specifies a 1,000W system |
| GeForce RTX 5080 | 16 GB GDDR7 | 960 GB/s | Fast 16GB tier work | 16GB is the ceiling; NVIDIA lists nothing between it and the 5090 |
| GeForce RTX 5060 Ti (16GB) | 16 GB GDDR7 | 448 GB/s | Entry into the 16GB tier on current silicon | A 128-bit bus, roughly half the 5080’s bandwidth at identical capacity |
| GeForce RTX 4090 (used) | 24 GB GDDR6X | Not published by NVIDIA | The 24GB tier on the secondhand market | 450W, no transferable warranty, unknown history |
| GeForce RTX 3090 (used) | 24 GB GDDR6X | Not published by NVIDIA | The older secondhand path to 24GB | Oldest architecture here, 350W, and NVIDIA specifies a 750W system |
| Radeon RX 7900 XTX | 24 GB GDDR6 | 960 GB/s | 24GB with bandwidth matching an RTX 5080 | ROCm, not CUDA, with the distro constraints below |
| Radeon RX 9070 XT | 16 GB GDDR6 | 640 GB/s | Current-generation 16GB on RDNA 4 | Same software tax, and 16GB is 16GB |
| Radeon AI PRO R9700 | 32 GB GDDR6 | 640 GB/s | 32GB in one slot outside the 5090 | Workstation part; bandwidth is roughly a third of the 5090’s |
| Intel Arc Pro B60 | 24 GB GDDR6 | 456 GB/s | 24GB from a third vendor | Lowest bandwidth of the 24GB options; SYCL or Vulkan rather than CUDA |
| Mac with M5 Max | Up to 128 GB unified | 614 GB/s | Models far larger than any consumer GPU holds | Memory is shared with the whole system, and prompt processing is the weak spot |
| NVIDIA DGX Spark | 128 GB unified LPDDR5x | 273 GB/s | Very large models on a desk, slowly | Bandwidth below a midrange desktop card |
Figures come from the NVIDIA comparison page, AMD’s pages for the RX 7900 XTX, RX 9070 XT and AI PRO R9700, the Arc Pro B60 data sheet, Apple’s MacBook Pro specifications and the DGX Spark page.
Two absences, stated precisely. NVIDIA’s pages for the RTX 4090 and RTX 3090, checked on 7 August 2026, give a 384-bit GDDR6X interface but no bandwidth figure, so none is quoted. The comparison page, checked the same day, lists seven RTX 50 series desktop cards and nothing between the 5080’s 16GB and the 5090’s 32GB.
Work Out Which Tier You Need First
Buying by tier is cheaper than buying by brand. The VRAM index maps each memory bracket to the current open-weight pick for chat, coding, vision, speech, translation and agents, and the tier guides state what fits at 6GB, 8GB, 12GB, 16GB, 24GB and 32GB. Read the tier above yours before buying.
The step that changes what you can do is usually 16GB to 24GB, where mid-size open-weight models stop needing an aggressive quantization to fit. Which quantization to pick afterwards is covered in the quantization format guide.
The Used Market Is Where The Capacity Is
A used 24GB card is frequently the better buy against a new 16GB one, because capacity is the constraint that stops a model loading and no amount of bandwidth substitutes for it. The RTX 3090 and RTX 4090 are the obvious candidates, and both remain on the supported list in Ollama’s hardware documentation at compute capability 8.6 and 8.9.
The risks are specific. A used card carries no transferable warranty and an unknown thermal history. Power is the part people forget: NVIDIA rates the RTX 3090 at 350W with a 750W system requirement and the RTX 4090 at 450W with an 850W system requirement, so a bargain card can force a power supply purchase and a case with clearance for a three-slot cooler.
The sharper trap is buying older still. Large-memory datacentre cards appear constantly in secondhand listings, and NVIDIA’s CUDA Toolkit release notes record that support for Maxwell, Pascal and Volta was removed, “corresponding to compute capabilities earlier than Turing”. A 24GB card from that era is one the current toolkit no longer builds for.
AMD And Intel Are Real, And There Is A Tax
The reflex to buy NVIDIA is not baseless, but it is stronger than the evidence now supports.
AMD’s ROCm system requirements list the RX 9070 XT, RX 9070, RX 9060 XT, RX 7900 XTX, RX 7800 XT and the AI PRO R9700 among the supported cards. The practical cost sits in the footnotes: that page restricts the consumer Radeons to “Ubuntu 24.04.4, Ubuntu 22.04.5, RHEL 10.1, and RHEL 9.7”, so the distribution you already run may not be one of the four. Support also lands at different times in different layers. Ollama’s hardware page, checked on 7 August 2026, lists RX 9000 and RX 7000 series cards for ROCm on Linux but only RX 7000 series cards for ROCm on Windows.
Intel’s position is narrower and more interesting than its reputation. The Arc Pro B60 data sheet specifies 24GB of GDDR6, 456 GB/s and 160 Intel Xe Matrix Extensions engines, which is a 24GB card from a third vendor. Intel’s Arc B-series announcement gives 12GB for the B580 and 10GB for the B570; it states capacity but no bandwidth figure, so none is quoted.
The software path exists for both. llama.cpp’s build documentation covers a HIP backend for AMD, a SYCL backend for Intel Arc and a Vulkan backend that spans vendors, and Ollama enables Vulkan by default. The cost of choosing either is not that models fail to run. It is an extra troubleshooting layer, tools that still assume CUDA, and a scheduler working with less information: Ollama’s documentation notes that Vulkan needs elevated capability or root access to report available VRAM, and falls back to approximate model sizes without it.
Apple Silicon Is A Different Question
A Mac is not a slower graphics card, it is a different shape of machine, and the tier tables mislead on it. Unified memory is shared between CPU and GPU rather than dedicated to the GPU, so capacity is generous and bandwidth is the binding constraint. Apple’s specifications put the M5 at 153 GB/s, the M5 Pro at 307 GB/s and the M5 Max at 460 GB/s or 614 GB/s depending on GPU core count, with the M5 Max configurable to 128GB. The Mac Studio specifications list 410 GB/s or 546 GB/s for the M4 Max and 819 GB/s for the M3 Ultra.
So a Mac can hold models no consumer graphics card holds, at bandwidth between a midrange and a high-end desktop card, drawing a fraction of the power. It also means the base chips are poor choices for this work despite the shared badge. How much unified memory the GPU may actually claim, and where Macs are slow, is covered in running LLMs locally on Apple Silicon.
NVIDIA sells the same trade in a different box: DGX Spark pairs 128GB of coherent unified memory with 273 GB/s, more capacity than any GeForce card and less bandwidth than a mainstream one.
What Not To Buy
Do not buy a card where the memory number flatters a narrow bus. The clearest example sits inside one product line: the RTX 5060 Ti in its 16GB configuration and the RTX 5080 both carry 16GB, but NVIDIA’s figures give the former a 128-bit interface at 448 GB/s and the latter a 256-bit interface at 960 GB/s. Same tier, same models available, generation speed governed by the number that differs. The cheaper card is still the right buy if capacity is what you lack, so long as you know which half of the equation it satisfies.
Do not buy 8GB new if local AI is the reason for the purchase. It is a working tier and it is the tier you will want to leave first.
Do not buy capacity you cannot feed. A 32GB card at 640 GB/s and a 32GB card at 1,792 GB/s load the same model and do not deliver the same experience, and the gap widens as context grows.
What This Means
The buying order that follows is short. Decide the model tier you want to run, buy the smallest card that clears it on capacity, then spend what is left on bandwidth. For most people upgrading today that resolves to a used 24GB card; the reasons to go further are long contexts, image generation or a preference for a warranty. Vendor choice is mostly a question of how much troubleshooting time is worth to you, because the AMD and Intel constraints are published rather than hidden.
The Bottom Line
VRAM decides what runs, bandwidth decides how fast it types, and the memory bus quietly decides whether a headline capacity means anything. Check both numbers on the manufacturer’s page before buying, because a card is a multi-year decision and the models it will be asked to run have not been released yet.