Local LLM Hardware Requirements

Local LLM Hardware Requirements

3B-7B: min 6GB VRAM. 8B-13B: min 8GB VRAM. 30B-34B: min 16GB VRAM (Q4). 70B+: min 24GB VRAM. CPU-only: 7B at 5-10 t/s with llama.cpp.