AI Agent Monitoring 2026: Traces
Classic APM watches servers, but agents fail differently: loops, hallucinated tool calls, a single session burning double-digit dollars. Monitoring an agent means traces, session cost, loop detection and quality signals.
📜 Table of Contents
AI Agent Monitoring 2026: Traces, Token Costs and Failure Detection
Classic APM watches servers, but agents fail differently: loops, hallucinated tool calls, a single session burning double-digit dollars. Every service is green and users are still angry.
How Agents Fail
| Failure | Symptom | Visible to classic APM? |
|---|---|---|
| Loop | same tool call repeated, no progress | no - CPU and latency look fine |
| Tool misuse | right parameters, wrong tool, returns 200 | no |
| Cost blowup | dozens of calls per session | no - no per-session aggregation |
| Silent degradation | shorter, emptier answers, refusals | no error raised |
| State pollution | a wrong conclusion propagates to later turns | no |
All five need session-level tracing. Metric dashboards cannot see them.
Six Fields to Instrument
| Field | Why |
|---|---|
| session_id, trace_id | one tree per user request |
| step index and parent | reconstruct the chain |
| model name and version | regression comparison |
| input/output tokens | cost attribution |
| tool name, args, status | locate misuse and timeouts |
| latency and retries | separate slow from stuck |
Add one business field: did the session solve the problem.
Four Dashboards
- Session - success rate, turns, P95 duration, per-session cost distribution (watch the tail).
- Tools - call volume, failure and timeout rates, payload size.
- Cost - tokens by session, user and task type; find the few calls eating the budget.
- Quality - sampled scoring, user feedback, escalation rate. A sudden drop in escalations is not automatically good.
Setting Thresholds
- Observe for two weeks, alert nothing.
- Baseline at P95/P99 - alert when a session exceeds ~1.5x P99.
- Absolute rules for must-not-happen events: 5 identical tool calls in a row, session cost over budget, 3 consecutive tool failures.
- Review false positives monthly; delete noisy rules instead of loosening them forever.
Tooling
| Tool | Role | Fits |
|---|---|---|
| Langfuse | open-source LLM tracing and evals | default self-hosted start |
| Arize Phoenix | observability + evals | RAG and retrieval quality |
| Helicone | gateway with logs and caching | central recording at the gateway |
| OpenTelemetry GenAI conventions | neutral standard | avoiding vendor lock-in |
Trace first, evaluate second. Evaluation without traces is blind.
Instrumenting in an Evening
from langfuse import Langfuse
lf = Langfuse(public_key=..., secret_key=..., host=...)
trace = lf.trace(name="agent-run", session_id=session_id)
span = trace.span(name="tool:search", input=query)
span.end(output=result, metadata={"tokens": n})
Open a span per tool call, not one at the top. The tree shape is what makes it debuggable.
FAQ
Are logs enough? Logs say what happened; traces say where in the chain. With dozens of steps per session, logs alone rarely locate the fault.
Does sampling miss failures? Sample normal traffic only; keep 100% of errors and over-budget sessions.
Preventing cost blowups? Per-session and per-user budgets with forced human checkpoints.
Is this worth it for small teams? Start with session IDs, token counts, tool failure rate and a session cost alert. That covers most incidents.
Note: features and pricing follow official docs; set thresholds against your own baseline.
Related
Written by our editorial team; tools listed here are tested or verified against public sources. Links point to official sites or GitHub repos for reference only — no paid placements.