Arize Phoenix 2026：开源LLM可观测性与评估（10k星）
在生产环境发现的问题本地复现不了。Phoenix把生产链路抓下来变成评估数据集——闭环了。
Arize Phoenix: Observability That Feeds Back Into Your Evals
Arize Phoenix (10,850 GitHub stars, Elastic-style license with a permissive core) is the open-source LLM observability platform from Arize AI, the company behind the popular Arize production ML monitoring. Its distinctive idea: production traces are not just for debugging - they are raw material for evaluation. You capture real requests, and with one click turn bad ones into eval cases.
The Loop: Trace, Detect, Evaluate
Phoenix captures spans for every LLM call, retrieval, and tool execution - the same tracing model as Langfuse, but with a stronger evaluation story. When a production response is flagged (by a score, a user report, or a drift alert), you export that trace into an eval dataset, run it against a candidate fix, and verify the fix before redeploying. That loop is the difference between observability as a log viewer and observability as a quality engine.
Setup
pip install arize-phoenix
python -m phoenix.server.main serve
# or in-notebook: phoenix.launch_app()
Then instrument with the OpenInference auto-instrumentors (one decorator for LangChain, LlamaIndex, or OpenAI calls) and traces appear in the Phoenix UI. It also runs fully in a notebook, which makes it popular for offline eval sessions.
What You Get
- Traces with embeddings. Phoenix stores the actual embeddings from traces, so you can cluster and visualize where retrieval goes wrong.
- Evals built in. LLM-as-judge templates for hallucination, QA correctness, and toxicity - run them on captured traces directly.
- Dataset export. Bad traces become golden eval examples with one click.
- Local by default. Data stays on your machine; no cloud required.
Phoenix vs Langfuse vs Helicone
All three do tracing. The split: Langfuse has the richest product polish and prompt management; Helicone is the lightest and cheapest to run at scale (great for cost per trace); Phoenix has the deepest eval-to-production loop and the best notebook workflow. Many teams run Phoenix during development and one of the others in production - or Phoenix everywhere for simplicity.
FAQ
Is Phoenix free? The core is open source and self-hostable; Arize sells a hosted enterprise tier.
Does it work with any model? Yes - it instruments the framework layer, so OpenAI, Anthropic, and local models all trace.
Can it handle production scale? Yes - it supports Postgres-backed persistence and spans millions of traces.
Do I need to change my code much? No - one decorator or auto-instrumentor call per framework.
