Agent report

Devin failures

9 documented incidents, average severity 2.4/10. Most common failure mode: scope explosion. 1 study is listed but not counted.
n = 9
9
Incidents
2.4
Avg severity
0
Critical
9
Source checked

Severity breakdown

Stacked severity breakdown per agent. Longer bar = more incidents.

Failure modes

Distribution of failure modes across all documented incidents.

Worst documented incident

5.0medium

Devin replaced entire medical website with unrelated renal care site

All 9 incidents

5.0
Devin replaced entire medical website with unrelated renal care site
3.0
Devin CI workflow caused 836-comment spam storm on single PR
2.0
Devin built 13,600-line app with build failure instead of lean campaign dashboard
2.0
Devin PR broke ledger list API and created buckets on deleted resources
2.0
Devin repeatedly submitted identical docs PRs that kept getting rejected
2.0
Devin attempted to build entire Figma clone from scratch — 3 rejected attempts
2.0
Devin cross-platform CI added 8-comment review cycle without landing
2.0
Devin docs PR rejected by Prefect maintainers — documented behavior from removed feature
2.0
Devin added a pointless "Hello!" page to a disease prediction platform
1.0
In independent testing, Devin completed just 3 of 20 real-world tasks (15%)