LLM Observability 2026: Trace Every Token With Langfuse (32k Stars) - Setup in 10 Minutes

Your LLM app is slow and expensive and you cannot tell which prompt or model call is the culprit. You need tracing - the LLM equivalent of logs and metrics.

## LLM Observability: You Cannot Fix What You Cannot See LLM applications fail in ways traditional monitoring misses: a prompt that returns empty, a tool call that loops, a context window that silently truncates. LLM observability platforms add tracing and evaluation on top of the usual logs and metrics. Langfuse is the leading open-source option - 32,279 GitHub stars as of August 2026, with a generous self-hosted community edition. ## What a Trace Shows You With Langfuse, every user request becomes a trace: the prompt template used, the model called, token counts, latency, cost, and the final output - plus any tool calls and retrievals in between. You can replay a single bad response and see exactly which stage produced it. This one capability changes debugging from guesswork to forensics. ## Setup in 10 Minutes ```python from langfuse import Langfuse langfuse = Langfuse() # reads LANGFUSE_PUBLIC_KEY / SECRET_KEY @langfuse.observe() async def generate(prompt): return await client.chat.completions.create(...) ``` The @observe decorator auto-captures inputs, outputs, latency, and cost. There are SDKs for Python, JS, and OpenAI/Anthropic/LangChain integrations that work with zero code changes. ## The Three Dashboards You Will Live In 1. **Traces** - per-request debugging with full prompt/output history. 2. **Metrics** - cost and latency trends per model, per prompt, per user. 3. **Evals** - run automated scoring (correctness, toxicity) on recorded traces, and turn regressions into tests. ## Cost Reality Self-hosted Langfuse is free software (MIT for the core); the cloud has a free tier of 50k observations per month. The infrastructure cost of self-hosting is modest - a small Postgres instance and a container. For teams spending more than a few hundred dollars a month on LLM APIs, observability pays for itself by finding one over-budget prompt. ## FAQ **Is Langfuse really free?** The core is MIT-licensed open source; self-hosting is free. **Does it work with local models?** Yes - it traces any call, including Ollama and vLLM endpoints. **Langfuse vs LangSmith?** LangSmith is LangChain's hosted product; Langfuse is the open-source alternative with a similar feature set. **Can it monitor costs?** Yes, it tracks cost per trace if you configure model pricing.