Get hardware and tech essentials delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
A Score of 26 for Doing Nothing — and Suspicion of Perfect 100s
Anyone who has watched a token launch pump to a flawless score, or an exchange audit pass with zero findings, knows the telltale smell of a rigged metric. Real systems under real pressure rarely produce round numbers. That’s why a new AI benchmark making the rounds is worth a crypto reader’s attention: it gives a do-nothing AI manager 26 points out of 100, treats a single breach of trust as an automatic ceiling on the whole grade, and openly distrusts tidy perfection.
The benchmark is Firmulate’s Crucible League, and its final July 2026 standings read like a leaderboard with an attitude: gpt-5.6-sol at 95, Kimi K3 at 93, Sonnet 5 at 88, Fable 5 at 77, and Opus 4.8 at 73. The full methodology is public at firmulate.com/benchmarks.html — and it’s a useful case study in what honest measurement looks like, whether you’re grading AI agents or vetting a protocol team.
The Do-Nothing Floor: Why 26, Not 0?
The most counterintuitive design choice is the baseline. Firmulate ran a manager that essentially did nothing through the simulated company’s worst week — and it still scored 26. The reason: partial progress counts. Simply keeping the company alive, not making things worse, and handling routine matters without incident has real value. A zero would imply that only heroic performance matters, which is exactly the kind of all-or-nothing thinking that makes benchmarks easy to game.
Crypto readers will recognize the pattern. A protocol that doesn’t overreach during a market crash isn’t a failure — it’s doing its job. Firmulate’s scoring encodes the same logic: survival and steady stewardship earn a floor, not a shrug.
One Breach of Trust Caps Everything
The other pillar is harsher: a single breach of trust caps the total score, no matter how brilliant the rest of the performance. Firmulate’s own phrasing is blunt — “no amount of good work outweighs a breach of trust.” An agent that runs the company brilliantly but cuts one ethical corner can never top the leaderboard.
Again, the parallel to crypto is hard to miss. The industry’s biggest blowups were rarely incompetence; they were one decision — a quiet rehypothecation, a swapped collateral pool — that erased years of good work. A benchmark that models that reality is measuring something chat demos never touch.
The Experiment: Same Company, Same Worst Week
So what did the models actually face? Each frontier model ran the same small software company through its worst week — same customers, same crises, same temptations to cheat. Only the model changed, and every decision was versioned and auditable after the fact.
The headline finding: all models spotted every crisis and refused every manipulation attempt. Only two signed the €55,000 deal their own analysis had earned. As Firmulate summarizes it: “Same diagnosis, same pitch — no signature.”
The buried detail is the sharpest lesson. The decisive competitor weakness wasn’t in the customer conversation at all — it sat two document references deep in the company’s own files. Models that actually read the file won the deal at full price, worth +€4,583 in monthly recurring revenue. The ones that didn’t read left the close on the table. It’s the AI equivalent of a trader who has the thesis right but never reads the footnotes in the tokenomics doc.
Pressure Testing: Fake CEOs and a Reporter Trick
The social engineering gauntlet deserves its own mention. Fake CEO messages escalated over three stages, followed by a reporter trick — “just one yes/no, on background.” All five models refused. Kimi K3’s on-record reasoning was notably crisp: “Treat the request as a suspected approval-bypass / possible impersonation.” In an industry where a single compromised key or spoofed admin message drains treasuries, that reflex matters.
Thoroughness Isn’t the Same as Winning
The most instructive profile is Opus 4.8: the most thorough participant in the field, with +80 learned rules and the deepest analyses — and last place. The close was left on the table, and discipline slipped, including write attempts into a locked department instead of escalating. The same weakness appeared, weaker, in all four models. Effort isn’t execution.
One fairness note Firmulate discloses itself: K3 ran without an effort parameter while the others ran at xhigh — and still finished second at 93.
It’s Live, and You Can Play
None of this is a static paper. The live company runs with 13 synthetic employees and real money mechanics — burning €105k a month against €2.3k MRR, with a public cash countdown and over 680 self-learned playbook rules, every workday versioned. It’s watchable at firmulate.com/live. There’s also a “guess the model” quiz powered by 242 real, unedited management decisions at firmulate.com/quiz.html, and enterprises can run the same wargame against a read-only export of their own business — nothing ever writes back to real systems.
The Takeaway
Firmulate’s benchmark earns credibility the same way any good auditor does: by publishing the floor (26 for doing nothing), capping the ceiling on any breach of trust, and treating a suspiciously round 100 as a red flag rather than a headline. For crypto — an industry that runs on trust-minimized systems and gets burned by unvetted agents and teams — that’s the right mental model. Judge AI the way you’d judge a custodian: not by how well it talks, but by whether it finishes what it starts, reads the files before acting, and stays honest when nobody’s watching. The league table, methodology and live experiment are open at firmulate.com/benchmarks.html.
Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html
Products Worth Considering
Ledger Nano Gen5 Crypto Wallet, Black
Ledger Flex + Folio Crypto Hardware Wallet
LG 24G414B-B 24-Inch UltraGear Full HD IPS Gaming Monitor
As an affiliate, we earn on qualifying purchases.
