LocalAI Usage Tutorial: Deploy Open-Source AI Models on Your Server in 2026

LocalAI Usage Guide

Once LocalAI is running, this guide covers model management, API endpoints, and integration with popular tools.

Managing Models

List models

curl http://localhost:8080/v1/models

Download models

curl -X POST http://localhost:8080/models/apply -H Content-Type:application/json -d model=llama-3.1-8b-instruct

Model config

name: llama-3.1-8b-instruct
parameters:
  context_size: 4096
  n_gpu_layers: 35
  batch: 512
backend: llama-cpp

API Endpoints

Chat Completions

curl http://localhost:8080/v1/chat/completions -H Content-Type:application/json -d {"model":"llama-3.1-8b-instruct","messages":[{"role":"user","content":"Explain RAG"}]}

Image Generation

curl http://localhost:8080/v1/images/generations -H Content-Type:application/json -d {"model":"stable-diffusion-3.5","prompt":"cat in spacesuit"}

LangChain Integration

from langchain_openai import ChatOpenAI
llm = ChatOpenAI(
    model=llama-3.1-8b-instruct,
    openai_api_base=http://localhost:8080/v1
)
response = llm.invoke(What is LocalAI?)

Performance Tuning

• Batch size: start at 512 for GPU throughput
• Context size: lower to 2048 if OOM
• Quantization: Q4_K_M for best balance

FAQ

Can LocalAI and Ollama run together?

Yes. Use different ports (8080 and 11434).

How to secure LocalAI?

Place behind nginx reverse proxy.

Can I use OpenAI SDK?

Yes. Change the base_url to http://localhost:8080/v1.