Devin
AI Agent Reliability Report
According to StupidLLM's incident database, Devin has 12 documented failures with an average severity of 4.9/10 — including 3 critical and 0 high-severity incidents.
4.9
Avg Severity /10
12
Total Incidents
3
Critical
0
High
Failure Modes
Destructive Action2
Infinite Loop2
Scope Explosion2
Ignored Instructions2
Security Vulnerability1
Logic Error1
Hallucination1
Other1
Root Causes
Instruction Misunderstanding4
Scope Misunderstanding3
Training Data Gap2
Pattern Matching Error1
Tool Misuse1
Confidence Miscalibration1
Worst Documented Incident
STUPID-2026-000110.0criticalVerified
Devin deleted all migration files during auth refactor
All Devin Incidents
STUPID-2026-000110.0criticalVerified
Devin deleted all migration files during auth refactor
STUPID-2026-000610.0criticalVerified
Devin confidently shipped code that passed tests but had a SQL injection vulnerability
STUPID-2026-001710.0criticalVerified
Devin replaced entire medical website with unrelated renal care site
STUPID-2026-00145.8mediumVerified
Devin CI workflow caused 836-comment spam storm on single PR
STUPID-2026-00215.0mediumVerified
Devin built 13,600-line app with build failure instead of lean campaign dashboard
STUPID-2026-00113.4lowVerified
Devin PR broke ledger list API and created buckets on deleted resources
STUPID-2026-00133.4lowVerified
Devin attempted to build entire Figma clone from scratch — 3 rejected attempts
STUPID-2026-00393.3lowVerified
In independent testing, Devin completed just 3 of 20 real-world tasks (15%)
STUPID-2026-00122.2lowVerified
Devin repeatedly submitted identical docs PRs that kept getting rejected
STUPID-2026-00162.1lowVerified
Devin docs PR rejected by Prefect maintainers — documented behavior from removed feature
STUPID-2026-00152.0lowVerified
Devin cross-platform CI added 8-comment review cycle without landing
STUPID-2026-00181.4lowVerified