Ollama on Raspberry Pi 5 2026: Run a Local LLM for Under $100

The Raspberry Pi 5 is cheap and power-efficient - but can it actually run modern LLMs? Yes, with the right models.

## Ollama on Raspberry Pi 5 2026: Run a Local LLM for Under $100 The Raspberry Pi 5 (8GB, ~$80) cannot run a 70B model - but it can run small models fine for chat, summarization, and light agent tasks. With Ollama, setup takes 15 minutes. Here is the honest performance picture. ## Why the Pi 5 Is Viable - 8GB RAM variant handles 3B-8B quantized models - ~5-10 tokens/s on 3B models - usable for chat - 5W power draw - runs 24/7 for pennies - Official Raspberry Pi AI Kit adds 26 TOPS NPU for image models ## Setup ```bash # 1. Install Ollama (official script supports ARM64) curl -fsSL https://ollama.com/install.sh | sh # 2. Pull a small model ollama pull qwen2.5:3b # 3. Chat ollama run qwen2.5:3b ``` ## Model Recommendations by RAM | Pi model | RAM | Best models | Speed | |----------|-----|-------------|-------| | Pi 5 4GB | 4GB | qwen2.5:1.5b, llama3.2:1b | 8-15 tok/s | | Pi 5 8GB | 8GB | qwen2.5:3b, phi-3-mini | 5-10 tok/s | | Pi 5 + AI Kit | 8GB | + small vision models via NPU | faster | ## Real-World Uses That Work - **Home assistant** with a custom system prompt (8GB model, decent answers) - **Offline document summarization** (3B is enough for short docs) - **Privacy-sensitive chat** (data never leaves your network) - **Teaching/learning LLM concepts** (inspect prompts and outputs freely) ## What Does NOT Work - Coding agents and complex reasoning (models too small) - Large context windows (8GB RAM limits context to ~8-16K tokens) - Real-time translation of long text - Anything requiring a 30B+ model ## FAQ **Is it worth it vs cloud APIs?** For learning and privacy, yes. For heavy production workloads, a cloud API or GPU machine is faster per dollar. **Can I cluster multiple Pis?** Yes - llama.cpp supports distributed inference, but network overhead limits gains; 2 Pis is practical, 4+ rarely pays off. **Does the AI Kit help with LLMs?** The NPU accelerates vision/embedding models; text LLMs still run on the CPU. Speedup varies by model.