Open Source AI Hardware Requirements: Complete Guide to Building Your Local AI Rig

# Open Source AI Hardware Requirements: Complete Guide to Building Your Local AI Rig Building a local AI rig is the best investment you can make in 2026. No API costs, no rate limits, no privacy concerns. ## The Short Answer | Budget | Cost | Can Run | Best For | |--------|------|---------|----------| | Entry | $700-1200 | 7B models | Learning, basic chat | | Mid-Range | $1500-2500 | 7B-13B | Development, coding | | High-End | $2500-5000 | 13B-34B | Production, research | | Enthusiast | $5000+ | 34B-70B+ | Cutting edge | ## Entry Level ($700-1200) **Used RTX 3060 12GB + 32GB RAM build:** - GPU: Used RTX 3060 12GB (~$200) - CPU: Ryzen 5 5600 (~$120) - RAM: 32GB DDR4-3200 (~$60) **Can run:** 7B Q4 at 30-40 tok/s, 13B Q4 at 15-20 tok/s **Cannot:** 34B+ models, long context (8K+) **Mac Alternative:** Mac Mini M4 (16GB) — $599, silent, 7B Q4 at 15-20 tok/s ## Mid-Range ($1500-2500) **RTX 3090 24GB build (best value in 2026):** - GPU: Used RTX 3090 24GB (~$700-900) - CPU: Ryzen 7 7700X (~$300) - RAM: 64GB DDR5-6000 (~$200) **Can run:** 7B-13B at full GPU speed, 34B Q4 at 20-25 tok/s **Mac Alternative:** Mac Studio M2 Max (64GB) — $2200, 34B Q4 at 20+ tok/s ## High-End ($2500-5000) **Dual RTX 3090 + Ryzen 9:** - GPUs: 2x Used RTX 3090 24GB (~$1600) - CPU: Ryzen 9 7950X (~$550) - RAM: 128GB DDR5-6000 (~$400) **Can run:** 34B at full speed, 70B Q4 at 20+ tok/s ## GPU Comparison for LLM Inference | GPU | VRAM | Price (used) | 7B Q4 | 13B Q4 | 34B Q4 | |-----|------|-------------|-------|--------|--------| | RTX 3060 | 12GB | $200 | 35 t/s | 18 t/s | No | | RTX 4060 Ti | 16GB | $350 | 45 t/s | 25 t/s | No | | RTX 3090 | 24GB | $750 | 60 t/s | 40 t/s | 22 t/s | | RTX 4090 | 24GB | $1600 | 85 t/s | 55 t/s | 30 t/s | | 2x RTX 3090 | 48GB | $1500 | --- | --- | 45 t/s | ## RAM and Storage Requirements - 16GB: Minimum for 7B models - 32GB: Comfortable for 7B-13B - 64GB: Good for 34B Q4 - Storage: 2TB NVMe recommended (each model 3-8GB) ## The Most Important Decision: GPU vs RAM If you already have a computer: - GPU VRAM >= 8GB? Use it! Try Ollama. - No? Try CPU inference (7B at 3-8 tok/s). - Too slow? Get used RTX 3060 12GB for ~$200. ## FAQ **Q: Is VRAM or RAM speed more important?** A: VRAM determines what models you can run. **Q: Can I use eGPU?** A: Yes, Thunderbolt 4 eGPUs work. Expect 10-15% loss. **Q: NVIDIA or AMD?** A: NVIDIA is better supported (CUDA). For beginners: get NVIDIA. **Q: Is Apple Silicon worth it?** A: Yes. Unified memory is a huge advantage.