Unsloth QLoRA Tutorial 2026: 4-Bit Fine-Tuning That Fits in 8 GB of VRAM

You have an 8 GB GPU and a 7B model to fine-tune. That used to be impossible. QLoRA with Unsloth makes it work - here is exactly how, with the memory math.

💡 What You Will Learn

You have an 8 GB GPU and a 7B model to fine-tune. That used to be impossible. QLoRA with Unsloth makes it work - here is exactly how, with the memory math.

📜 Table of Contents

• The Memory Math First
• The Unsloth Advantage
• Step 1: Load 4-Bit With Unsloth
• Step 2: The Adapter Config
• Step 3: Train Small
• Step 4: Watch the Real Budget
• Step 5: Export and Serve
• The Honest Expectation
• FAQ

The Memory Math First

A 7B model in full precision needs ~14 GB just for weights, plus optimizer states and gradients - 40 GB total territory. LoRA freezes the base and trains small adapters, dropping the trainable part to ~1-2%. QLoRA goes further: it quantizes the frozen base to 4-bit, cutting the base weights to ~4 GB. That is the entire trick: (stars fetched 2026-08-12, Unsloth at 70,038)

The Unsloth Advantage

Unsloth implements QLoRA's 4-bit path with hand-tuned kernels, so the quantized training is not just possible but fast - roughly 2x faster than standard QLoRA implementations on the same hardware, with ~70% less memory in the common 7B case.

Step 1: Load 4-Bit With Unsloth

from unsloth import FastLanguageModel
model, tokenizer = FastLanguageModel.from_pretrained(
    model_name="unsloth/Qwen2.5-7B-bnb-4bit",
    max_seq_length=1024,
    load_in_4bit=True,
)

max_seq_length 1024 keeps activation memory low - the second lever after quantization.

Step 2: The Adapter Config

model = FastLanguageModel.get_peft_model(
    model, r=16, lora_alpha=32, lora_dropout=0,
    target_modules=["q_proj", "k_proj", "v_proj", "o_proj", "gate_proj", "up_proj", "down_proj"],
)

More target modules = more capacity at slightly higher memory. For 8 GB, r=16 and the four attention projections is the safe baseline.

Step 3: Train Small

• Batch size 1 with gradient accumulation 8.
• max_steps 100-200 for a first run.
• Use unsloth's UnslothTrainer with UnslothTrainingArguments to get the memory-optimized loop:

from trl import SFTTrainer
from unsloth import is_bfloat16_supported

Step 4: Watch the Real Budget

On an 8 GB card: 7B-4bit weights (~4 GB) + LoRA adapters (tens of MB) + activations + optimizer. Unsloth's fused optimizer keeps peak under ~7.5 GB in the 1024-token case - uncomfortable but workable. If you hit OOM: drop max_seq_length to 512 or r to 8.

Step 5: Export and Serve

Save as merged 16-bit for quality serving, or keep the 4-bit adapters if you serve with a quantized runtime. Either way, inference does not need Unsloth.

The Honest Expectation

QLoRA 4-bit is a quality compromise you accept for hardware reasons: the tuned model is slightly weaker than full LoRA on complex reasoning, fine for style/format/domain adaptation. Budget accordingly.

FAQ

Can I fine-tune on an 8 GB laptop GPU? Yes - 7B 4-bit with 1024-token context is the standard recipe; expect slow but working.

QLoRA vs LoRA - what is the difference? QLoRA quantizes the frozen base to 4-bit (memory win, tiny quality cost); LoRA keeps the base in its original precision (better quality, more memory).

What if I still get out of memory? Reduce max_seq_length first, then r, then batch size. One lever at a time.

Related reads: Unsloth Tutorial 2026, Unsloth LoRA Tutorial 2026, LLM Quantization Types Explained 2026.