Engineering · Effectiveness

Is it working?

run-outcome signal · last 30d

Fleet report card

task success vs 90% target

89%

just shy of target

0↑ 90% target

Across 1212 signal-bearing tasks the fleet completes goals 89% of the time — 1 pts below the 90% target, +10 pts over six weeks. Biggest single lift: fix support-copilot (~56 failed tasks). But only 63% of agents emit a measured outcome — read the rest as directional.

SuccessB

89%

1212 tasks

Human-freeB

92%

8% needed a human

First-tryB

87%

13% reworked

CoverageC

63%

5/8 measured

Outcome signal — known vs guessed

5/8 agents emit a measured run-outcome; 2 are inferred from retries / abandons / errors; 1 are dark (shadow). Numbers are projected where signal ≠ measured — treat inferred rates as directional.

measured inferred dark

Success rate — 6-week trend

projected · signal-bearing

Up +10 pts over 6 weeks (79% → 89%), still climbing this week.

target 90%

−6w: 79%

++10

now: 89%

Fix this first — failed-task leverage

tasks × (1 − success)

Where the most goals are actually missed. A high-volume agent at 82% can fail more tasks than a low-volume one at 48% — the top row moves the most outcomes.

1support-copilotmeasured~56

310 tasks82% success

2marketing-gptinferred~33

64 tasks48% success

3data-pipeline-agentmeasured~16

540 tasks97% success

4Codex CLIinferred~13

38 tasks66% success

5Claude Codemeasured~10

142 tasks93% success

6Cursormeasured~10

96 tasks90% success

7Claude Desktopmeasured~1

22 tasks95% success

By workflow — success vs target

sorted by success · dashed line = 90% target

content-gen and infra-ops are the laggards and both run on inferred signal — instrument them before trusting the rate.

data-syncmeasured

97%

540 tasks2% human

code-reviewmeasured

92%

260 tasks7% human

support-triagemeasured

82%

310 tasks12% human

infra-opsinferred

66%

38 tasks21% human

content-geninferred

48%

64 tasks34% human

By agent

ordered by failed-task volume · dark agents last

Agent	Tasks	Success	Intervene	Rework	Quality
support-copilotmeasured	310	82%	12%	18%	79
marketing-gptinferred	64	48%	34%	41%	—
data-pipeline-agentmeasured	540	97%	2%	5%	91
Codex CLIinferred	38	66%	21%	32%	—
Claude Codemeasured	142	93%	6%	11%	88
Cursormeasured	96	90%	8%	14%	84
Claude Desktopmeasured	22	95%	5%	9%	90
Zed Agentnone	—	no signal	—	—	—

This is task success, not speed. Latency and cost say how fast and cheap a model is; this asks whether the agent actually accomplished the goal. Cost & spend live in the Cost Center →.

Derived: effectivenessCoverage · fleetEffectiveness · lowestEffectiveness · outcomes — from app/data/c16-fleet.ts (proposed run-outcome signal; measured / inferred / none provenance per agent). Components marked [proposed] are m2 inventions, not yet in q-nuxt-layer.