Are agent → tool/model calls reliable — and is anything looping expensively?

Reframes raw availability and latency telemetry as a per-agent SLO. Engineering watches whether each agent's calls to its tools and model endpoints succeed fast — and flags any agent retry-storming a failing endpoint, which is both a reliability and a cost event.

Overall availability

99.2%

Successful calls across all agents.

p95 latency

1.8s

Agent → tool/model round-trip.

Slow calls

3.1%

Calls above the 3s latency budget.

Retry storms

data-pipeline-agent vs failing endpoint.

p95 latency (24h)

Seconds per agent call

p95 sits at 1.8s; a spike at hour 14 traces to data-pipeline-agent retries.

error rate (24h)

Failed calls / total

Error rate climbs from a 0.6% baseline as a downstream endpoint starts failing.

Per-Agent SLO

Reliability budget per agent — the unit Engineering pages on.

AgentAvailabilityp95 latencyError rateRetry stormsTrend

claude-code-prod99.8%1.1s0.4%none

data-pipeline-agent98.1%3.4s6.2%Retry storm

cursor-ide99.6%0.9s0.6%none

support-copilot99.0%2.2s1.8%none

marketing-gpt97.2%2.9s4.1%none

Retry storm · reliability + cost event

data-pipeline-agent is retry-storming against a failing endpoint — both a reliability and a cost event, because every retry burns more tokens. Triage the endpoint here, then confirm the dollar impact on Agent Spend.