Engineering · Performance
How fast each agent's model responds, what each call costs — and where speed and spend collide.
Find the tail-latency outliers users feel, then weigh them against $/call: slow AND expensive is the real target. But note the gap — this is response speed, not task success.
Bottom line
Real target: Codex CLI — 6.8k ms p95 AND $0.144/call (slow and costly). Latency is mostly the model's, not the agent's — re-modeling trades depth for speed. Task Outcomes & Quality — whether the agent succeeded — is still not measured.
p95 latency by agent
95th-percentile response time · lower is betterLatency, cost & variance by agent
percentiles in ms · $/call = agent cost ÷ calls · variance = (p99−p50)/avgRed rows are both high p95 (≥ 4.0k ms) and high $/call (≥ $0.100) — the agents worth re-modeling first. Amber variance (≥ 4×) flags erratic agents whose tail dwarfs their average.
Latency by model
grouped from performance.agents · the model-choice trade-offLatency is largely a property of the model, not the agent. Re-modeling a slow agent onto a faster family (e.g. gpt-5-codex → claude-haiku-4-5) trades quality/depth for tail latency — decide per workload, not per fleet.
Slowest responses
individual outlier calls · the tail you actually feelRead percentiles, not averages — and pair them with cost. A healthy average hides the tail; p95 and p99 expose the slow calls users wait on. But a slow call on a cheap model is a different decision than a slow call on an expensive one — that's why $/call sits next to p95. Codex CLI's 6.8s p95 / 11.2s p99 at $0.144/call is the real target: slow and costly.
This is call latency & cost — not task success. Speed and spend measure how fast and how expensively a model responds, not whether the agent accomplished the goal. Task Outcomes & Quality — success rate, goal completion, and output quality — is the named gap we do not yet measure.
That gap now has a home — Effectiveness — is it working? → tracks success, intervention & rework, with per-agent measured / inferred / none provenance.
Fields: performance.agents (name · model · calls · p50 · p95 · p99 · avg_ms · tput) · performance.slowest (ts · agent · model · duration_ms · tokens · session) · agents[].cost (for $/call). Derived: $/call · variance · latency-by-model — from app/data/c16-fleet.ts (qcontrol /api/performance shape).