AI Models Prove Their Worth in Critical Business Tests, Outperforming Expectations

AIThis post was created with the assistance of artificial intelligence (AI).

Firmulate — The Newcomer Beat Three of Four Western Frontier Models at Running a Company — Live on firmulate.com.

Prime Big Deal Days · Oct 6–7Offer from Amazon

Get decor and gifts delivered free — and shop member deals

Fast, free delivery on millions of items
Access to Prime Big Deal Days deals on October 6–7
Prime Video, Amazon Music and more included

Start your free Prime trial Free trial for eligible customers · Cancel anytime

As an affiliate, we earn on qualifying purchases.

Can AI Actually Run a Business? These Experiments Show It’s Possible—and Profitable

Imagine your favorite home decor company facing a week of crises: customer churn, security threats, and tough negotiations. Now, picture AI models navigating these challenges as if they were seasoned managers—deciding, resisting manipulation, and closing deals. That’s exactly what a recent public experiment by Firmulate demonstrates, revealing AI’s growing capacity to handle real-world business complexities.

Real-World Testing of AI in Business Management

In a groundbreaking experiment, four leading AI models were tasked with managing a small software company through its worst week, facing the same crises, customer demands, and temptations. Unlike typical demos that focus on the ability to generate text or answer questions, this test was about decision-making under pressure, honesty, and strategic focus.

All models successfully identified every crisis and refused manipulative attempts, showing a strong grasp of integrity. But the key finding was that only two of these models managed to close a €55,000 deal—equivalent to nearly €4,583 in monthly recurring revenue—after analyzing the company’s internal files.

One competitor, gpt-5.6-sol, scored the highest with a 95 out of 100 on the test, correctly uncovering buried information and sealing the deal. Moonshot’s Kimi K3 was just behind with a 93, performing with remarkable discipline, especially considering it ran without an effort parameter, which means it operated at the default setting. The other two models, Sonnet 5 and Fable 5, scored 88 and 77 respectively, managing to close deals but with some slips in process discipline.

What Made the Difference? Deep File Reading and Integrity

The decisive factor wasn’t just crisis recognition but reading deep into internal company documents—something many models failed to do. The winning models found critical information buried two documents deep, not visible in superficial scans. This ability to access and interpret internal data proved essential for closing the deal at full price.

Furthermore, when social engineering tactics were tested—fake CEO messages escalating over multiple stages—every model refused to be manipulated. Kimi K3 explained, “Treat the request as a suspected approval-bypass / possible impersonation,” showing a disciplined approach to security and trustworthiness.

Live Business Mechanics and Transparency

The experiment was conducted on a live, real-money business with 13 synthetic employees, managing weekly cash flows, customer relationships, and internal rules. The company burns €105,000 monthly against just €2,300 in monthly recurring revenue, illustrating the high stakes and real-world pressures involved.

Every decision was versioned and auditable, with over 680 self-learned rules guiding daily operations. This transparency allows anyone to watch the AI in action at firmulate.com/live.

Insights for Business Leaders

This experiment underscores an important lesson: when choosing AI for business management, the ability to do more than generate confident-sounding text—reading internal data, resisting manipulation, and closing deals—is crucial. The league table clearly shows that the best-performing models are those that go beyond surface-level analysis.

Of note, Kimi K3 ran at default settings (no effort parameter), whereas the others operated at a higher effort level (xhigh), suggesting the default may already be highly competitive.

Beyond the Test: Why It Matters

For businesses considering AI to augment or replace management tasks, this experiment demonstrates that performance depends on a model’s decision quality, honesty, and ability to access relevant information—not just conversational prowess. It’s a reminder that picking an AI model without testing it in a realistic scenario may be a gamble.

Interested in how your enterprise can prepare? You can run your own wargame against a read-only export of your business, ensuring real crises are simulated without risking actual systems. More details are available at firmulate.com/pilot.html.