Ollama Windows Setup Guide: Install and Run Local LLMs in 10 Minutes

# Ollama Windows Setup Guide: Install and Run Local LLMs in 10 Minutes Ollama (176K GitHub stars) makes running local LLMs on Windows dead simple. ## Step 1: Install Ollama on Windows Download the official Windows installer from ollama.com/download. Run it. The installer adds Ollama to your system PATH automatically. After installation, verify: ```bash ollama --version ``` ## Step 2: Enable GPU Acceleration (NVIDIA) Ollama automatically detects NVIDIA GPUs via CUDA. **GPU Memory Requirements by Model Size:** | Model Size | GPU VRAM Needed | Example GPUs | |-----------|----------------|-------------| | 7B Q4 | ~5GB | RTX 3060 12GB, RTX 4060 | | 13B Q4 | ~8GB | RTX 3070, RTX 4070 | | 34B Q4 | ~20GB | RTX 3090, RTX 4090 | | 70B Q4 | ~40GB | Dual RTX 3090, A6000 | ## Step 3: Pull and Run Models ```bash ollama pull llama3.1:8b ollama run llama3.1:8b ``` ## Step 4: Set Up Open WebUI ```bash docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -e OLLAMA_BASE_URL=http://host.docker.internal:11434 -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main ``` Open http://localhost:3000 — a ChatGPT clone powered by your local LLM. ## Step 5: Connect to VS Code (Continue extension) Install Continue extension in VS Code. In config.json: ```json { "models": [{ "title": "Ollama Local", "provider": "ollama", "model": "llama3.1:8b" }] } ``` ## Performance Benchmarks on Windows | GPU | Model | Tokens/sec | |-----|-------|-----------| | RTX 4060 (8GB) | Llama 3.1 8B Q4 | 45-55 t/s | | RTX 3090 (24GB) | Qwen3 14B Q4 | 35-45 t/s | | CPU only (i7-13700) | Llama 3.1 8B Q4 | 5-8 t/s | ## Troubleshooting **Problem: Ollama not found in new terminal** Run `refreshenv` or restart terminal. **Problem: GPU not detected** Install CUDA 12.4+ from NVIDIA. **Problem: Out of memory** Use smaller models or Q4 quantization.