Ollama Local Setup Guide 2026
Installing Ollama is the easy part. Where models live, which port it listens on, how concurrency and context are set, and whether other machines can reach it decide whether it is a toy or a usable service.
💡 What You Will Learn
Installing Ollama is the easy part. Where models live, which port it listens on, how concurrency and context are set, and whether other machines can reach it decide whether it is a toy or a usable ser
📜 Table of Contents
Ollama Local Setup Guide 2026
Installing Ollama takes two minutes, which makes people think the job is done. What actually shapes the experience comes after: where models are stored, which address the service binds to, whether other machines can reach it, how many requests run in parallel, and how long the context is. Here is the full path.
1. Install
Windows: download the installer from the official site; Ollama runs in the background and the CLI becomes available.
macOS: dmg or Homebrew, then a menu-bar icon plus CLI.
Linux: the official script creates an ollama system user and a systemd service.
curl -fsSL https://ollama.com/install.sh | sh
ollama --version
systemctl status ollama
2. Model directory: decide first
By default models land in your user directory on the system drive. An 8B Q4 model is four to five GB; a dozen models is tens of GB. This is the usual reason a small system SSD fills up.
| Variable | Purpose | Suggested |
|---|---|---|
OLLAMA_MODELS |
Model storage path | Point at a large data disk |
OLLAMA_HOST |
Bind address and port | 0.0.0.0:11434 only if LAN access needed |
OLLAMA_NUM_PARALLEL |
Parallel requests per model | 2-4 to start, based on VRAM |
OLLAMA_MAX_LOADED_MODELS |
Models kept resident | 1 when VRAM is tight |
OLLAMA_KEEP_ALIVE |
Idle unload timer | 24h for daily-use models |
OLLAMA_CONTEXT_LENGTH |
Default context | Set per workload; longer costs memory |
On Linux use a systemd override rather than editing the packaged unit:
sudo systemctl edit ollama.service
[Service]
Environment="OLLAMA_MODELS=/data/ollama/models"
Environment="OLLAMA_HOST=127.0.0.1:11434"
Environment="OLLAMA_KEEP_ALIVE=24h"
Then sudo systemctl daemon-reload && sudo systemctl restart ollama. On Windows set the same names as system environment variables and restart the tray app.
3. Pull a model and talk to it
ollama pull qwen3:8b
ollama run qwen3:8b
ollama list # installed models
ollama ps # currently loaded models
Model names use name:tag, where the tag usually encodes parameter count and quantization. Rough sizing: about 8GB of VRAM or RAM handles 3B-4B quantized, 16GB handles 8B, 24GB opens up 14B. Chat and summarization are fine at 8B; code and reasoning benefit from larger.
4. Two APIs
Native API, /api/chat for multi-turn:
curl http://localhost:11434/api/chat -d '{
"model": "qwen3:8b",
"messages": [{"role":"user","content":"Explain KV cache in three sentences"}],
"stream": false
}'
OpenAI-compatible endpoint at http://localhost:11434/v1 with any non-empty key:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:11434/v1", api_key="ollama")
r = client.chat.completions.create(
model="qwen3:8b",
messages=[{"role": "user", "content": "Write a quicksort in Python"}],
)
print(r.choices[0].message.content)
Any tool that accepts a custom base_url can now point at your local model without code changes.
5. LAN access and the security boundary
Setting OLLAMA_HOST=0.0.0.0:11434 lets other devices on the network connect - useful when a desktop runs the model and a laptop consumes it. Add a firewall rule for the port.
Ollama ships with no authentication. Do not expose it directly to the internet. For remote access use an SSH tunnel or a reverse proxy with auth:
ssh -L 11434:127.0.0.1:11434 user@server
6. Troubleshooting
| Symptom | Likely cause | Fix |
|---|---|---|
| Will not start, port in use | Another instance, or 11434 taken | Find the process or change port |
| Very slow first response | Model loading from disk | Set OLLAMA_KEEP_ALIVE |
| Speed drops to single digits | VRAM exhausted, layers fell back to CPU | Check ollama ps, use a smaller model or lower quant |
| Truncated answers | Context or output limit hit | Raise context and num_predict |
| Pull hangs | Network or disk space | Check free space, retry |
| LAN clients cannot connect | Still bound to 127.0.0.1, or firewall | Change OLLAMA_HOST, open port |
7. Three settings worth changing
Context length: defaults are short for long documents, but oversized contexts eat memory. Set per task.
Keep-alive: longer for models you use all day; saves the reload penalty each time.
Parallelism: only meaningful with multiple clients. For a single user, raising it splits bandwidth and makes each response slower.
FAQ
Can models live on an external or network drive? Yes, but network storage random reads are slow enough to make loading painful. USB 3.0 external drives are workable.
How do I remove a model? ollama rm name. It must be pulled again to use.
How does Open WebUI fit in? Run it in Docker and point OLLAMA_BASE_URL at the host's Ollama instance for a ChatGPT-like interface.
CPU only? Yes, it works out of the box, just slower. Prefer models under 3B.
Note: variable names, defaults and model tags change between releases. Check the official Ollama documentation.
Related
Written by our editorial team; tools listed here are tested or verified against public sources. Links point to official sites or GitHub repos for reference only — no paid placements.
