Engineering · Performance
The real target is Codex CLI — 6.8k ms p95 and $0.144/call
The tail users feel and pay for lives in Codex CLI on gpt-5-codex — 6.8k ms p95 at $0.144/call. Slow and costly is the agent worth re-modeling first, not the merely slow one. And remember: speed is not success — a fast reply can still be the wrong answer, and that axis is not measured here.
Performance answers how fast & how expensive. Whether it worked — task success, intervention & rework — lives in Effectiveness, one level up.
Task success lives in → EffectivenessFleet p95 latency
call-weighted across 5 agents
Slow & costly target
Codex CLI · $0.144/call
Best throughput
data-pipeline-agent · fastest emitter
Most erratic tail
data-pipeline-agent · (p99−p50)/avg
p95 latency by agent
95th-percentile response time · lower is better · red = slow & costlyLatency, cost & variance by agent
percentiles in ms · $/call = cost ÷ calls · variance = (p99−p50)/avgAgent | Model | Calls | p50 | p95 | p99 | tok/s | $/call | Variance |
|---|---|---|---|---|---|---|---|---|
| Codex CLI slow & costly | gpt-5-codex | 540 | 2.4k | 6.8k | 11.2k | 31 | $0.144 | 2.8× |
| Claude Code slow & costly | claude-opus-4-8 | 1.2k | 1.8k | 4.2k | 7.8k | 42 | $0.173 | 2.9× |
| support-copilot | gpt-4o | 3.1k | 1.1k | 3.4k | 6.1k | 58 | $0.054 | 2.7× |
| Cursor | claude-sonnet-4-6 | 980 | 900 | 2.6k | 4.8k | 71 | $0.098 | 2.7× |
| data-pipeline-agent | claude-haiku-4-5 | 5.4k | 400 | 1.1k | 2.2k | 120 | $0.010 | 2.9× |
Orange rows are both high p95 (≥ 4.0k ms) and high $/call (≥ $0.100) — the agents worth re-modeling first. High variance (≥ 4×) flags erratic agents whose tail dwarfs their average.
Latency by model
grouped from performance.agents · the model-choice trade-offModel | Agents | Calls | Avg p95 | Worst p95 |
|---|---|---|---|---|
| gpt-5-codex | 1 | 540 | 6.8k ms | 6.8k ms |
| claude-opus-4-8 | 1 | 1.2k | 4.2k ms | 4.2k ms |
| gpt-4o | 1 | 3.1k | 3.4k ms | 3.4k ms |
| claude-sonnet-4-6 | 1 | 980 | 2.6k ms | 2.6k ms |
| claude-haiku-4-5 | 1 | 5.4k | 1.1k ms | 1.1k ms |
Latency is largely a property of the model, not the agent. Re-modeling a slow agent onto a faster family (e.g. gpt-5-codex → claude-haiku-4-5) trades quality/depth for tail latency — decide per workload, not per fleet.
Output throughput — tokens / second
how fast each agent emits tokens · higher is betterdata-pipeline-agent emits 120 tok/s — 3.9× the slowest (Codex CLI at 31 tok/s). Low throughput compounds tail latency: long answers on a slow emitter are where users wait the most.
Slowest responses
individual outlier calls · the tail you actually feelWhen | Agent | Model | Latency | Tokens | Session |
|---|---|---|---|---|---|
| 2026-06-02 19:28:00 | Codex CLI | gpt-5-codex | 11.2k ms | 28.4k | ee71aa90 |
| 2026-06-03 13:01:00 | Claude Code | claude-opus-4-8 | 7.8k ms | 19.8k | 1a1f867b |
| 2026-06-03 13:50:00 | support-copilot | gpt-4o | 6.1k ms | 14.2k | ab9920fd |
The slowest single call — 11.2k ms from Codex CLI (28.4k tokens) — is what a user actually waited on. Big-token answers on a slow model dominate the tail.
Latency ≠ task success — a fast wrong answer is still wrong
Every number on this page measures how fast and how expensively a model responds — not whether the agent accomplished the goal. A 400 ms reply that hallucinates an API, ignores a constraint, or needs a human to redo it is a failure that looks healthy here. Optimizing latency and cost without measuring outcomes optimizes for confident speed, not correctness.
That gap now has a home — Effectiveness — is it working? → tracks success rate, intervention & rework, with per-agent measured / inferred / none provenance. A slow-but-correct agent may beat a fast-but-wrong one.
Do next
Fields: performance.agents (name · model · calls · p50 · p95 · p99 · avg_ms · tput) · performance.slowest (ts · agent · model · duration_ms · tokens · session) · agents[].cost (for $/call). Derived: fleet p95 (call-weighted) · $/call · variance (p99−p50)/avg · latency-by-model · slow & costly flag — from app/data/c16-fleet.ts (qcontrol /api/performance shape).