Run Ollama Locally with Docker: Complete 2026 Setup Guide
🩺 Summary
You want to run AI models locally with Docker for isolation and easy management, but you are not sure how to set up Ollama in a container, expose it to other services, and manage model storage efficiently.
📝 Details
Ollama has an official Docker image that makes deployment trivial. With a single docker-compose.yml, you can run Ollama, Open WebUI (ChatGPT-like interface), and n8n together - all talking to your local models.
# Run Ollama Locally with Docker
> The complete guide to running AI models in Docker containers. Isolated, manageable, scalable.
## One-Command Setup
```bash
docker run -d --name ollama -p 11434:11434 -v ollama:/root/.ollama ollama/ollama
```
Then pull a model:
```bash
docker exec ollama ollama pull llama3.1:8b
```
## Full Stack with Open WebUI
The ChatGPT-like interface for your local models:
```yaml
version: '3'
services:
ollama:
image: ollama/ollama
volumes:
- ollama:/root/.ollama
ports:
- "11434:11434"
open-webui:
image: ghcr.io/open-webui/open-webui:main
ports:
- "3000:8080"
environment:
- OLLAMA_BASE_URL=http://ollama:11434
volumes:
- open-webui:/app/backend/data
volumes:
ollama:
open-webui:
```
## GPU Support
For NVIDIA GPUs, add `deploy` section:
```yaml
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
```
## Performance Tips
- Mount models to a fast SSD
- Use Q4_K_M quantized models for 50% RAM savings
- Enable keep-alive to keep models loaded between requests
- Use `OLLAMA_NUM_PARALLEL` for concurrent requests
> Dockerized Ollama gives you the most flexible, portable way to run local AI. One compose file, infinite possibilities.
💬 Comments (0)