AI Agent SLA设计：保证服务质量的几个关键指标

SLA指标详解与监控代码

核心指标与量化目标

Python埋点监控代码

import time
import json
from datetime import datetime
from collections import deque

class SLAMonitor:
    def __init__(self, window_minutes=60):
        self.window = window_minutes * 60
        self.latencies = deque()
        self.success_count = 0
        self.total_count = 0
        self.errors = {}

    def record(self, latency_ms: float, success: bool, error_type: str = None):
        now = time.time()
        self.latencies.append((now, latency_ms))
        self.total_count += 1
        if success:
            self.success_count += 1
        else:
            self.errors[error_type] = self.errors.get(error_type, 0) + 1
        while self.latencies and now - self.latencies[0][0] > self.window:
            self.latencies.popleft()

    def report(self) -> dict:
        latencies = [l[1] for _, l in self.latencies]
        latencies.sort()
        n = len(latencies)
        success_rate = self.success_count / max(self.total_count, 1)
        p50 = latencies[int(n * 0.5)] if n else 0
        p95 = latencies[int(n * 0.95)] if n else 0
        p99 = latencies[int(n * 0.99)] if n else 0
        return {
            "timestamp": datetime.now().isoformat(),
            "total_requests": self.total_count,
            "success_rate": round(success_rate * 100, 2),
            "latency_ms": {"p50": p50, "p95": p95, "p99": p99},
            "error_breakdown": dict(sorted(self.errors.items(), key=lambda x: -x[1])[:5])
        }

monitor = SLAMonitor(window_minutes=60)

def agent_call(user_input: str) -> str:
    start = time.time()
    try:
        result = agent.invoke(user_input)
        elapsed = (time.time() - start) * 1000
        monitor.record(elapsed, success=True)
        return result
    except Exception as e:
        elapsed = (time.time() - start) * 1000
        monitor.record(elapsed, success=False, error_type=type(e).__name__)
        raise

print(json.dumps(monitor.report(), ensure_ascii=False, indent=2))

低于SLA时的排查步骤

1. 响应慢：先看是P50高还是P95高。P50高说明模型本身慢，换模型或降量化；P95高说明偶发慢，看是不是某段时间流量峰值。

2. 成功率低：看错误类型分布。如果是RateLimitError，加流量控制和重试；如果是模型拒绝，改Prompt。

3. 准确率低：每天随机抽100条Agent回复，让人工标注。区分是RAG召回问题（知识库没相关内容）还是模型生成问题。

4. 可用性低：看是模型API挂了还是你自己的服务挂了。API挂加备选模型切换，服务挂加负载均衡。