The Machine that Changed the World, vAI#

Tariq Shaukat (Sonar, CEO)

• Why: A useful synthesis of the external evidence on where agentic code production breaks down — worth the PM's time for the cited constraints, with the caveat that the back half is a vendor pitch.
• What: The five constraints on agentic software factories: (1) human review doesn't scale — a cited study found asynchronous agents drove +2,450% lines of code and +240% commits but only +30% releases, and "cognitive surrender" research shows humans agree with wrong AI answers four out of five times; (2) agents struggle in large codebases — cited code-translation accuracy collapses from 91% at 10K lines to much worse by 50K, and token use for the same task varies 30x by starting context; (3) errors compound — 95% per-step success over 20 steps is ~36% end-to-end, and METR's task-horizon curves flatten sharply at higher success thresholds; (4) LLM judges are unreliable — 7–17% of verdicts change on identical replication, and a simple "are you sure?" flips 7–34%; (5) time-to-exploit is near zero, so prevention must be designed in.
• How: Sonar's answer is a "code confidence formula" — intent × guide rails × verification × evidence (multiplied: any zero is a zero) — with verification as a quality gate inside the production line rather than a report beside it, mixing deterministic and agentic checks. Reported customer results: 36% lower token consumption and 92% fewer issues in committed code from context/rail product (Sonar Vortex); the closing survey comparisons (81% more likely to report positive security impact, etc.) are sentiment differences among 1,026 respondents, not outcome measurements.
• Where applicable: The constraint evidence is general and cited; the results are Sonar-reported for its own products and should be treated as vendor figures. The factory framing (borrowed from Toyota production-system thinking, via Nadella's containment argument) is a good mental model for engineering-organization design regardless of tooling.