How LLMs Work in 2026: Transformer Architecture Explained with Transformers (163k Stars) and llama.cpp (123k Stars)

A plain-language explanation of how LLMs work - tokens, attention, transformers - grounded in the two most important open-source projects: Hugging Face Transformers (163,356 stars) and llama.cpp (122,773).

💡 What You Will Learn

A plain-language explanation of how LLMs work - tokens, attention, transformers - grounded in the two most important open-source projects: Hugging Face Transformers (163,356 stars) and llama.cpp (122,

## The short answer An LLM is a next-word predictor trained on trillions of tokens. The Transformer architecture (introduced in the 2017 paper "Attention Is All You Need") lets the model weigh every word against every other word in the context - that attention mechanism is the core idea behind ChatGPT, Claude and every modern LLM. ## The three layers of understanding **1. Tokens** - text is split into tokens (words or word parts). A model like llama3.1:8b has a ~128k-token vocabulary; typical English is ~1.3 tokens per word. **2. Attention** - for each token, the model computes how relevant every other token is. The phrase "bank by the river" vs "bank loan" is disambiguated by attention weights. **3. Parameters** - an 8B model has 8 billion weights. Training adjusts them to minimize next-word prediction error on internet-scale text. ## See it in code - Transformers ```python from transformers import AutoTokenizer, AutoModelForCausalLM tok = AutoTokenizer.from_pretrained("HuggingFaceTB/SmolLM2-135M") model = AutoModelForCausalLM.from_pretrained("HuggingFaceTB/SmolLM2-135M") inputs = tok("The capital of France is", return_tensors="pt") out = model.generate(**inputs, max_new_tokens=5) print(tok.decode(out[0])) # "The capital of France is Paris" - next-token prediction in action ``` ## See it in code - llama.cpp llama.cpp (122,773 stars, MIT) runs the same idea in C/C++ with quantized weights, so a 7-8B model runs on a laptop. Quantization (e.g., Q4_K_M) maps 16-bit weights to ~4 bits - 4x smaller, ~1-2% accuracy loss. ## Real numbers - GPT-class models use billions to trillions of parameters; open models range 0.1B (SmolLM) to 671B (DeepSeek-class). - Inference generates ~10-100 tokens/second depending on hardware and quantization. - Context windows have grown from 2k tokens (GPT-3 era) to 128k+ (llama3, GPT-4o class). ## FAQ **Q: Does the model "understand" language?** A: It models statistical patterns extremely well, which produces understanding-like behavior - but there is no consciousness or intent. **Q: Why do smaller models seem dumber?** A: Fewer parameters = less capacity to store world knowledge and follow complex instructions. Scale is the main driver of capability. **Q: What is fine-tuning?** A: Continued training on curated data (instructions, domain text) that adapts a base model to specific tasks - see LoRA for a cheap way to do it.