Every local LLM guide says buy a big GPU, yet real requirements vary twentyfold with model size, quantization and CPU versus GPU. Here is the math you can do yourself.

📜 Table of Contents

• One formula
• Estimating KV cache
• Model size and quantization
• Four hardware paths
• Bandwidth matters more than compute
• Three tests before buying
• Three budget tiers
• FAQ
• Related

Local LLM Hardware Guide 2026: How Much RAM and VRAM You Really Need

Before buying hardware, do the math instead of reading benchmarks. Local model memory needs can be estimated within a GB or two - enough to make a purchase decision.

One formula

Memory ≈ weights + KV cache + runtime overhead

• Weights = parameters × bits per weight ÷ 8
• KV cache grows linearly with context; several GB at long contexts
• Overhead usually 0.5-1.5GB depending on the runtime

An 8B model at 4-bit is about 4GB of weights, 1-2GB of KV cache at 8K context, plus ~1GB overhead: 6-7GB total. An 8GB card runs it, without much margin.

Estimating KV cache

KV ≈ 2 × layers × KV heads × head dim × sequence length × bytes per element

FP16 is 2 bytes, 8-bit KV is 1. Rules of thumb: an 8B-class model at 8K context uses roughly 1-2GB at FP16; 32K pushes that to 4-8GB; 70B-class models multiply again.

This is why "the weights fit" does not mean "long context works". Most post-purchase OOMs come from cache, not weights.

Model size and quantization

The totals already include cache and overhead. Sizing on weights alone guarantees trouble at long context.

Four hardware paths

Discrete GPU: an order of magnitude faster generation, but fixed and expensive VRAM. Best for interactive use.

CPU plus RAM: slow, but RAM is cheap and large. A 64GB machine runs 32B models no consumer card can hold, at single-digit tokens per second. Good for batch work and privacy.

Hybrid offload: llama.cpp and Ollama can keep some layers on the GPU and the rest in RAM. Best value compromise - 12GB GPU plus 32GB RAM handles 32B. Raise -ngl until you hit OOM, then back off one.

Apple unified memory: shared between CPU and GPU. A 64GB M-series machine runs 32B to 70B quantized models well, with strong performance per watt. The limit is bandwidth below high-end discrete cards, which hurts very long contexts and large batches.

Bandwidth matters more than compute

Generation is bandwidth-bound: every token reads all weights. A useful ceiling estimate: tokens/sec ≈ bandwidth ÷ weight size. With 4GB of weights and 400GB/s, the theoretical ceiling is about 100 tok/s; hitting half of that is respectable. This quickly tells you whether a card is worth it for generation.

Three tests before buying

1. Measure what you run now - ollama ps shows whether layers fell back to CPU, which means VRAM is short.
2. Test long context - same model at 2K and 16K; watch for OOM or a sharp slowdown. That is the real bottleneck.
3. Test your real prompts - not "hello", but the long prompts you actually send. Prefill time is what you feel.

Three budget tiers

Skip exact prices - they vary too much by region; check local quotes.

• Entry: use what you have. 8GB of RAM runs 1.5-3B Q4 for extraction, classification, short summaries. The goal is validating the need, not quality.
• Mainstream: 16GB RAM or 8-12GB VRAM for 7-8B Q4. The most practical 2026 configuration for personal use.
• Advanced: 24GB VRAM or 32-64GB unified memory for 14B-32B. For code, long documents and local agents.

Upgrade order: capacity first, bandwidth second, compute last. Capacity determines whether it runs at all; bandwidth determines how fast; raw compute is secondary during generation.

FAQ

VRAM versus RAM - interchangeable? VRAM is fast but small, RAM is large but slow. Offloading layers to RAM lets you run when VRAM is short, at reduced speed.

Can a laptop run 14B? A 16GB unified-memory ultrabook handles 14B at 4-bit, single-digit tok/s. 32GB is far more comfortable.

Do multiple GPUs pool VRAM? With model sharding, yes, but interconnect overhead often eats the gain on consumer cards. One large card is simpler.

High benchmark scores but slow models? Benchmarks measure compute; generation needs bandwidth. Also check for paging to disk and thread settings.

Do I need a professional card? Not for personal use. Their value is reliability and capacity, not solo inference value.

Note: memory needs vary with architecture, quantizer and context length. Table values are estimates - measure on your own machine.

Related

• Run LLM on CPU: models, speeds and setup
• GPU cloud pricing comparison 2026