AI Agent Troubleshooting Playbook 2026
When an agent misbehaves, random trial and error wastes hours. Here is a repeatable order: locate the layer, make it reproducible, bisect the scope, read the full trace - plus a lookup table and a regression set.
💡 What You Will Learn
When an agent misbehaves, random trial and error wastes hours. Here is a repeatable order: locate the layer, make it reproducible, bisect the scope, read the full trace - plus a lookup table and a reg
📜 Table of Contents
Debugging is ordered narrowing, not trial and error
Agent bugs have a signature: the same input works today and fails tomorrow, because three sources of nondeterminism sit behind it - sampling, retrieval results and external services. People start by editing the prompt and three hours later discover an upstream API returned empty data.
The order below: identify the layer, make it reproducible, bisect the scope, read the full trace. With the right order, most issues are located in half an hour.
Step 1: Which layer is failing?
| Symptom | Likely layer | Check first |
|---|---|---|
| Irrelevant answers, invented tool names | Prompt/model | The exact prompt sent and model version |
| Tool not called when it should be | Prompt/tool declaration | Tool descriptions, argument schema, truncation |
| Tool called with wrong args | Model | Whether prompt examples cover the case |
| Tool errors out | Tool/dependency | Tool logs, auth, timeouts |
| Retrieved but unused | Context assembly | Whether context was truncated or reordered |
| Right answer, wrong format | Output parsing | Raw model output versus parsed object |
| Random failures | Infrastructure | Rate limits, timeouts, concurrency, retries |
The point of the table: fix the layer, not the symptom.
Step 2: Make it reproducible
Pin three things: the full input (including history and retrieved documents), the configuration (model version, temperature, system prompt hash, tool list), and the external state. Then turn on full tracing - every message, tool argument, tool return and final output into logs or an observability tool such as Langfuse or OpenTelemetry. With a trace, "why did it answer that" is usually obvious.
Step 3: Bisect with five switches
| Action | What it tells you |
|---|---|
| Disable all tools | Model error versus bad tool data |
| Swap in a stronger model | Capability limit versus prompt problem |
| Set temperature to 0 | Remove sampling randomness |
| Use a minimal prompt | Remove interference from the system prompt |
| Hard-code retrieval results | Is retrieval the culprit |
Two rounds usually localise it.
Step 4: Common failures lookup
| Symptom | Root cause | Fix |
|---|---|---|
| Calls a non-existent tool | Overlapping descriptions or truncated names | Fewer tools, clearer descriptions |
| Loops on one tool | No step cap | Max steps plus duplicate-call detection |
| JSON parse failure | Prose around the JSON | JSON mode, strip code fences |
| Context overflow | Unbounded history | Summarise and truncate old turns |
| 401/403 | Expired key or changed scope | Check key expiry and permissions |
| 429 | No rate limiting, high concurrency | Add the three-layer limiter |
| Timeout | No per-step timeout, slow upstream | Set timeouts, degrade to partial results |
| Inconsistent quality | Retrieval quality | Fix a dataset, measure hit rate |
Step 5: Prevent recurrence
Every fixed issue becomes a regression case: the exact input plus the expected behaviour. After twenty or thirty cases, run them all whenever you change prompts, models or tools. This is the highest-value investment in an agent project.
Anti-patterns
- Changing several things at once and learning nothing
- Reading only the final answer, never the tool calls
- Closing the ticket with "it works now" and no root cause
- Editing production prompts with no version control
- Load-testing against real traffic
FAQ
It only fails in production. Align four things first: model version, temperature, system prompt hash, retrieval index version. Then replay the raw production trace locally.
Capability or prompt? Re-run the same input on a stronger model. A clear improvement means a capability boundary; no change means prompt or data.
The agent intermittently skips a tool. Check overlapping tool descriptions, then whether the system prompt states when the tool is mandatory. Past a dozen tools, missed calls get more likely - group them by scenario.
When to stop prompt engineering? When the same class of bug survives three fixes, enforce it in code: schema validation for formats, precondition checks for critical actions.
Related
Written by our editorial team; tools listed here are tested or verified against public sources. Links point to official sites or GitHub repos for reference only โ no paid placements.
