Self-Hosted LLM on Raspberry Pi 5 in 2026: Real Performance Numbers

Can a Raspberry Pi 5 run an LLM? Yes - here are the real token rates you can expect in 2026.

## Self-Hosted LLM on Raspberry Pi 5 in 2026 A Raspberry Pi 5 (8 GB) is the cheapest always-on LLM server you can own. It will not replace your cloud API, but it is perfect for privacy-first home automation. These numbers reflect community benchmarks from 2026. ### What runs on Pi 5 (8 GB) - **Ollama** (177,426 stars) installs in one command and works on ARM64 - **llama.cpp** (122,228 stars) is the recommended engine for CPU-only ARM - Models that fit: 1B-3B at Q4 (e.g., Qwen2.5-1.5B, Llama-3.2-1B, Phi-3-mini) ### Real performance (community benchmarks, 2026) - Qwen2.5-1.5B Q4: ~10-14 tokens/s - usable for simple chat and summarization - Llama-3.2-1B Q4: ~12-16 tokens/s - 3B models: ~5-8 tokens/s - borderline but workable for short prompts - 7B models: 1-3 tokens/s - technically runs, practically painful ### Setup in 3 commands [bash] curl -fsSL https://ollama.com/install.sh | sh ollama pull qwen2.5:1.5b ollama run qwen2.5:1.5b [/bash] ### What it is good for - Home assistant (Home Assistant + local LLM for device control) - Privacy: medical notes, family data never leave the house - Always-on, 5-7W power draw (vs 300W+ for a gaming PC) ### FAQ **Can a Pi run a useful LLM in 2026?** For short tasks, yes - 1.5B models handle summaries and intent parsing fine. **Pi 4 vs Pi 5?** Pi 5 is roughly 2x faster; only Pi 5 with 8 GB is worth it for LLMs.