Pulling a model too big for your machine is the classic local-LLM failure. This covers the size math, quantization levels, real footprints of common models, and how much context window costs.

📜 Table of Contents

• 1. The size formula is one line
• 2. Reading quantization tags
• 3. Real footprints at 4-bit
• 4. Context window is the hidden cost
• 5. Working backwards from hardware
• 6. Check before downloading
• 7. VRAM vs RAM
• FAQ

Ollama Model Sizes 2026: How Much RAM Each Model Really Needs

Local LLMs fail in one predictable way: you pull a model that looks impressive, it starts loading, and the machine locks up. The model isn't broken — the arithmetic wasn't done. Here is the math, the quantization levels, and a table to work backwards from your hardware.

1. The size formula is one line

Parameters × bits per weight ÷ 8.

• 7B at 16-bit: 7 × 10⁹ × 2 bytes ≈ 14GB
• 7B at 4-bit: ≈ 3.5GB, and real files land around 4GB because a few layers stay at higher precision

That is the whole point of quantization: trade a little accuracy for three quarters of the size. Most people cannot hear the difference in chat; code and math tasks show it.

2. Reading quantization tags

3. Real footprints at 4-bit

Check each model page at ollama.com for exact numbers. The right question is not "what is the largest I can load" but "how much headroom is left".

4. Context window is the hidden cost

Weights are the fixed cost; the KV cache grows with context length. Going from 4K to 32K context is roughly 8× the cache. A 7B model using 5GB of an 8GB card leaves room for only a few thousand tokens; push context higher and Ollama offloads layers to the CPU, and speed drops by an order of magnitude.

ollama run qwen2.5:7b --context-length 8192

Or pin it in a Modelfile with PARAMETER num_ctx 8192.

5. Working backwards from hardware

Rule of thumb: RAM ≥ 2× model size. The extra factor covers the OS and the context cache.

6. Check before downloading

ollama list shows installed sizes, ollama show <model> --modelfile shows layers and parameters, and ollama ps shows the runtime footprint — that last number includes context cache and is the one that matters.

7. VRAM vs RAM

Fully on GPU is fast (tens of tokens/sec). Fully on CPU is slow (a few tokens/sec) but fine for background summarising. Mixed offload works but speed scales with how much got pushed off. For CPU-only machines, more RAM beats a new GPU; with an 8GB card, moving to 16GB VRAM beats more RAM.

FAQ

File says 4.7GB, ollama ps shows 6GB? That gap is the KV cache and runtime overhead; it widens with context length.

q4_K_M vs q4_0? Similar size, but mixed precision protects sensitive layers, so quality is usually better. No reason to prefer q4_0.

Can I load only part of a model? Ollama offloads layers it cannot fit, but the goal is "runs", not "runs fast". If it crawls, use a smaller model.

Does it store two copies? No. Blobs are content-addressed, so shared weights are stored once.

❓ FAQ

Why did my PC freeze?

The model + overhead exceeded RAM and the OS swapped to disk - hard. Reduce model size or context.

What does q4 vs q8 mean for quality?

q8 is closer to full precision (better quality, 2x size); q4 is the standard tradeoff.

Can I run a 70B on 16GB?

Only heavily quantized (q2-q3) and slowly - not recommended.

Does context length affect memory?

Yes - KV cache grows with context; 128k context on a small machine will OOM. Keep it modest.