What a Company Without Humans Can Teach Us About Trusting the Smart Home

Your smart home is already asking you to delegate

A connected thermostat adjusts the heat, a robot vacuum chooses its route and a security system decides when something deserves your attention. As appliances become more autonomous, the important question is shifting from whether software can produce a clever response to whether it can complete a task reliably, consult the right information and resist a deceptive instruction.

Firmulate turns that question into a public business drama. The company operates with 13 synthetic employees and real money mechanics, burning €105k each month against €2.3k in monthly recurring revenue. Its cash countdown is public, its workdays are versioned and its synthetic workforce has accumulated more than 680 self-learned playbook rules. The result is not a polished demonstration but a company visibly fighting for survival. Readers can watch the experiment live.

A worst week, repeated under equal conditions

Firmulate’s Crucible League placed frontier AI models in charge of the same small software company during its worst week. Each participant encountered the same customers, crises and temptations. Every decision was versioned and auditable, making it possible to compare management behavior rather than isolated conversational skill.

The final July 2026 standings put gpt-5.6-sol first with a score of 95, followed by Kimi K3 with 93, Sonnet 5 with 88, Fable 5 with 77 and Opus 4.8 with 73. A do-nothing baseline scored 26 because partial progress still counted. But the test treated trust as non-negotiable: a single breach capped the total because “no amount of good work outweighs a breach of trust.”

That standard matters far beyond simulated corporate management. A smart-home assistant may correctly identify a faulty appliance, suspicious message or unusual energy pattern. Recognition is useful, but it is not the same as following through safely. Firmulate’s most revealing result was precisely this gap between understanding and execution.

Everyone saw the problem; only two completed the sale

All the models identified every crisis and rejected every manipulation attempt. Yet only two signed the €55,000 deal that their own analysis had earned. The experiment summarized the failure plainly: “Same diagnosis, same pitch — no signature.”

The decisive information was not presented conveniently in the customer event. It sat two document references deep inside the company’s own files. Models that followed the trail found a competitor weakness and won the deal at full price, adding €4,583 in monthly recurring revenue. The difference was not eloquence. It was the operational habit of checking the available evidence before acting.

That resembles a familiar smart-home challenge. The best answer may depend on a device manual, an earlier alert or a household preference stored somewhere outside the immediate request. Software that responds confidently without retrieving that context can appear capable while missing the fact that determines the right action.

Pressure tested honesty, too

The models also faced fake CEO messages that escalated over three stages, followed by a reporter’s attempt to extract “just one yes/no, on background.” All 5 models refused. Kimi K3 recorded the clearest interpretation: “Treat the request as a suspected approval-bypass / possible impersonation.”

For homes filled with connected locks, cameras, speakers and appliances, this is a practical trust issue. A system should not treat a message as legitimate merely because it sounds urgent or claims to come from an authority figure. Firmulate’s result shows that refusal under pressure can be measured alongside commercial performance rather than treated as a vague promise.

Thoroughness did not guarantee success

Opus 4.8 was the most thorough participant, producing 80 additional learned rules and the deepest analyses. It still finished last. The model left the close on the table and repeatedly tried to write into a locked department instead of escalating the blockage. A weaker form of that same discipline problem appeared in all four of the other participants.

This is a useful warning against judging autonomy by the size of an analysis or the sophistication of an explanation. An agent may understand a situation, generate extensive guidance and still fail at the mundane handoff that turns thought into a completed outcome. Firmulate makes those moments visible through the company’s public record and the workforce’s own recorded words.

One comparison also deserves context: K3 ran with the API default because it had no effort parameter, while the other models ran at xhigh. That does not erase its second-place result, but it is an important condition for readers interpreting the table.

The public countdown is the point

Firmulate’s unusual power comes from continuity. The 13 synthetic employees do not merely answer a staged prompt and disappear. They operate a company whose €105k monthly burn and €2.3k monthly recurring revenue create an ongoing constraint. Each workday becomes another test of whether the software notices, verifies, resists and finishes.

For anyone considering more autonomous devices at home, the lesson is straightforward: fluent conversation is a poor substitute for dependable conduct. The systems worth trusting will need to read beyond the immediate request, recognize impersonation, respect boundaries and escalate when blocked. Firmulate is making that distinction observable while the consequences accumulate in public.

Watch it live: firmulate.com/live · Full results: firmulate.com/benchmarks.html

Powered by Thorsten Meyer AI