Root cause

Confidence Miscalibration

13 documented incidents where confidence miscalibration was the underlying cause.

What this cause produces

  • Other5
  • Hallucination3
  • Infinite Loop2
  • Logic Error2
  • Ignored Instructions1
8.4
Amazon AI coding agent mistake blamed on human employees
4.4
OpenAI's Operator scored 38% on real computer tasks — and panics instead of recovering from errors
4.1
An AI agent spun up duplicate CloudFormation stacks on every error and ran up a $6,531 AWS bill
4.1
Two AI agents ping-ponged for 11 days and ran up a $47,000 bill — neither noticed anything wrong
3.3
Salesforce Agentforce hit a 77% B2B failure rate — and Salesforce admitted it was 'more confident than we should have been'
3.3
Cyera study: 344 verified enterprise agent-damage cases, 188 with no attacker involved
3.3
In independent testing, Devin completed just 3 of 20 real-world tasks (15%)
2.9
Cursor's own support AI 'Sam' invented a one-device login policy, triggering subscription cancellations
2.5
AI 'CVE slop' is drowning open-source maintainers: 60-80% of HackerOne submissions now invalid
2.5
AI 'slop' vulnerability reports flooded curl until it killed its bug bounty
2.2
The runaway-cost pattern, quantified: agentic coding tools burn 10-100x more tokens and can rival developer pay
2.2
Uber burned its entire annual AI coding budget in ~4 months after rolling out Claude Code to 5,000 engineers
2.0
Sakana's 'AI Scientist' rewrote its own code to bypass its timeout and looped endlessly calling itself