Skip to content
StupidLLM
Incidents
Compare
Failure Modes
Methodology
Search
Home
/
Failure Modes
/
Training Data Gap
Root cause
Training Data Gap
8 documented incidents where training data gap was the underlying cause.
What this cause produces
Security Vulnerability
4
Hallucination
2
Logic Error
1
Other
1
10.0
AI vibe-coded Next.js app pinned vulnerable dependency — cryptominer compromised production server
Unknown Agent
10.0
GitHub Copilot suggested 2,702 valid secrets — 33% of extracted keys were real, live credentials
Github Copilot
10.0
Scan of 5,600 vibe-coded apps found 2,000+ high-impact vulns, 400+ exposed secrets, PII leaks
Multiple Agents
7.5
Claude Opus 4.5 leaked API key in console logs during YouTube scraper build
Claude Code
6.4
Slopsquatting: LLMs hallucinate package names attackers pre-register (react-codeshift, unused-imports)
Multiple Llms
4.4
The quiet correctness tax: 43% of AI code changes need production debugging, with up to 75% more logic errors
Multiple Agents
2.7
Anthropic found Claude Opus 4 would blackmail testers in up to 96% of simulated shutdown scenarios
Claude
2.1
Devin docs PR rejected by Prefect maintainers — documented behavior from removed feature
Devin