Most AI’s Fail In A 500 Day Startup
Three AI models made money as CEOs of a fake software company for 500 simulated days. One rule-based non-AI algorithm also made money — and beat 11 of the 14 AI models. Claude Fable 5 came first, growing $1 million to $47.15 million.
A team of researchers at Princeton University gave control of a fake software company to 14 state-of-the-art artificial intelligence (AI) models. They let the AI run for 500 simulated days. Each AI began with $1 million and no customers. It had to interact with the company via an API, making decisions about pricing, advertising, product quality, customer service, and competitive actions. Its objective was to avoid bankruptcy. Many failed.
Three AIs ended up with more money than they began with. Claude Fable 5 did best, ending with $47.15 million after starting with $1 million. Claude Opus 4.8 ended with $27.8 million. GPT-5.5 ended with $21.3 million. Eleven AIs went bankrupt within the 500 days.
The result that stings
What’s interesting is not that Claude Fable 5 won, but that a rule-based non-AI algorithm ended up with $15.76 million — and beat 11 of the 14 AIs. This algorithm simply used fixed prices, fixed quotas, and targeted marketing. It didn’t use any reasoning, planning, or language models. It simply followed a deterministic series of if-then statements. Yet it outperformed almost all of the AIs.
This isn’t a trick question. It’s a diagnostic tool. The problem demands ‘steering intelligence’, according to the Princeton team. That means making decisions over months, taking into account how changing prices in week five will affect customer retention in week thirty. It means anticipating how competitors will react to your actions, which may be unpredictable. Today’s AIs are great at understanding a specific scenario, coming up with a targeted plan, or crafting a focused communication. But they struggle to maintain a consistent strategy over hundreds of decision iterations while circumstances change.
The successful AIs shared several characteristics. They tried different approaches instead of always cutting costs. They inferred missing data, such as customer satisfaction (which is hidden in the simulation and must be inferred from other factors). They forecast future cash flow trajectories rather than responding to crises. They rapidly adapted to competitor actions. All of these involve keeping track of the big picture over time, rather than just answering questions.
CEO-Bench is a benchmark that simulates a software company named NovaMind. It includes realistic lag times, such as billing intervals, long-term returns on R&D spending, and gradual changes in hidden customer satisfaction scores based on previous decisions. Its purpose is to diagnose a key deficiency in today’s AIs: the ability to maintain strategic consistency under stress.
If you're building with AI agents that run multi-step workflows, the CEO-Bench result is a useful frame. The failure mode isn't bad individual decisions — it's losing the strategic thread across many decisions under changing conditions. For anything consequential: build in regular moments where the agent explicitly checks whether the current action still makes sense given the original goal. And run your agents through longer tests before trusting them in production.
Source: the-decoder.com
Free course
Stop reading about AI — start building with it
The free Claude Code course: your first site, tool or game — no coding. No upsells, no cross-sells — nothing to buy here.
Start free →▌ Related guides

Author
Evgenii Arsentev
PhD · Founder & CEO, Dad4Kids
Articles · Latest articles