STUPID-2026-0059

OpenAI's Operator scored 38% on real computer tasks — and panics instead of recovering from errors

1.0low
July 15, 2025Source checkedStudy · not counted in agent averages

Why 1/10

A capability benchmark (38% task success), not a damaging event.

Scope
0/4
Time, tokens or the agent's own working state only
Recoverability
0/3
Nothing was lost
Did it happen
1/2
Demonstrated (proof of concept, reproduction, controlled test or near-miss) with no real victim reported
Beyond the user
0/1
Only the user's own non-production work

The score is the sum of these four factors. How each factor is scored · scored 2026-10-29

  1. Instruction given

    Complete real browser and desktop workflows autonomously (bookings, purchases, forms, emails).

  2. Expected behavior

    Complete common computer tasks reliably and recover gracefully when a step fails.

  3. Actual behavior

    Six months after launch, OpenAI's Operator scored 38% on OSWorld, a benchmark of real computer tasks — meaning about two in five fail. On errors, computer-use agents often panic, double down on the wrong action, ignore error messages, and don't retry differently. Failures range from a typo in an email to buying the wrong item to permanently deleting a document.

  4. Damage

    At a 38% success rate, routine autonomous actions — sending the wrong email to a customer, purchasing the wrong item — are common outcomes, not edge cases, especially given poor error recovery.

Full write-up

OpenAI launched Operator, its computer-using agent, in January 2025; six months later it scored 38% on OSWorld, a benchmark of real computer tasks — roughly two of every five tasks fail. Beyond the raw success rate, the more damaging finding is error recovery: when a step goes wrong, computer-use agents tend to panic, double down on the wrong action, ignore error messages, and fail to retry a different way. Because these agents act in the real world, the failure surface includes purchasing the wrong item, sending the wrong email to a customer, or permanently deleting a document. OpenAI's own mitigation — training the agent to ask for confirmation before finalizing side-effectful actions like orders and emails — is an admission that unsupervised autonomy at current reliability is unsafe for consequential tasks.

Classification

Agent
OpenAI Operator (discontinued)
Failure mode
Logic Error
Root cause
Confidence Miscalibration
Domain
Frontend
MIT AI Risk Repository (our mapping)
7.3 Lack of capability or robustness taxonomy
Source
Benchmark

Related incidents

1.0
In independent testing, Devin completed just 3 of 20 real-world tasks (15%)
7.0
Claude Code created a directory literally named "~", then an unquoted rm -rf * expanded into it and started deleting a developer's real home directory
7.0
Two AI agents ping-ponged for 11 days and ran up a $47,000 bill — neither noticed anything wrong
5.0
An AI agent spun up duplicate CloudFormation stacks on every error and ran up a $6,531 AWS bill
Discuss on Hacker News
Cite this record
StupidLLM. "STUPID-2026-0059: OpenAI's Operator scored 38% on real computer tasks — and panics instead of recovering from errors." AI Agent Incident Database, 2026. https://www.stupidllm.com/incident/STUPID-2026-0059/ (accessed 2026-10-29). CC BY 4.0.

← STUPID-2026-0058By IDSTUPID-2026-0060 →

Get told when an agent breaks something

We document AI agent failures daily, severity-scored against a published scale. Each Monday, if any scored 7 or above that week — deleted data, leaked secrets, broken production — you get one email with the sources. When nothing did, you get nothing.

This database is callable over MCP — query it from inside your agent.