LLM可观测性2026：用Langfuse追踪每个token，10分钟上手
LLM应用又慢又贵，却查不出是哪个提示词或模型调用的问题。你需要链路追踪。
LLM Observability: You Cannot Fix What You Cannot See
LLM applications fail in ways traditional monitoring misses: a prompt that returns empty, a tool call that loops, a context window that silently truncates. LLM observability platforms add tracing and evaluation on top of the usual logs and metrics. Langfuse is the leading open-source option - 32,279 GitHub stars as of August 2026, with a generous self-hosted community edition.
What a Trace Shows You
With Langfuse, every user request becomes a trace: the prompt template used, the model called, token counts, latency, cost, and the final output - plus any tool calls and retrievals in between. You can replay a single bad response and see exactly which stage produced it. This one capability changes debugging from guesswork to forensics.
Setup in 10 Minutes
from langfuse import Langfuse
langfuse = Langfuse() # reads LANGFUSE_PUBLIC_KEY / SECRET_KEY
@langfuse.observe()
async def generate(prompt):
return await client.chat.completions.create(...)
The @observe decorator auto-captures inputs, outputs, latency, and cost. There are SDKs for Python, JS, and OpenAI/Anthropic/LangChain integrations that work with zero code changes.
The Three Dashboards You Will Live In
- Traces - per-request debugging with full prompt/output history.
- Metrics - cost and latency trends per model, per prompt, per user.
- Evals - run automated scoring (correctness, toxicity) on recorded traces, and turn regressions into tests.
Cost Reality
Self-hosted Langfuse is free software (MIT for the core); the cloud has a free tier of 50k observations per month. The infrastructure cost of self-hosting is modest - a small Postgres instance and a container. For teams spending more than a few hundred dollars a month on LLM APIs, observability pays for itself by finding one over-budget prompt.
FAQ
Is Langfuse really free? The core is MIT-licensed open source; self-hosting is free.
Does it work with local models? Yes - it traces any call, including Ollama and vLLM endpoints.
Langfuse vs LangSmith? LangSmith is LangChain's hosted product; Langfuse is the open-source alternative with a similar feature set.
Can it monitor costs? Yes, it tracks cost per trace if you configure model pricing.
