NEW YORK — What happened: Software engineering researchers published a controlled productivity audit analyzing 2,000 professional developers using leading AI coding assistants over a six-month period. The study measured pull request velocity, bug frequency, and total time spent on code reviews.
Why it matters: Software development has been ground zero for commercial AI marketing, with vendors claiming developers can write code twice as fast. Understanding the real-world impact helps engineering leaders allocate resources without falling for inflated metrics.
Deep dive: While the raw line-of-code output increased by roughly 55%, the study revealed a paradoxical increase in maintenance overhead. AI-generated code frequently introduced subtle edge-case bugs that took longer to debug than writing the boilerplate manually. Furthermore, pull request review times increased because human reviewers had to meticulously check AI-suggested logic for hidden vulnerabilities.
Report check (claims vs what is verified vs still rumor): Vendor claims of 2x overall project completion speed are verified as false for complex enterprise software. What is verified is a significant speedup in boilerplate code generation and routine syntax translation. Rumors that AI tools would entirely replace junior developers were rated as completely unfounded by the engineering leads surveyed.
Open questions: Can next-generation models with integrated formal verification engines successfully mitigate the rise in subtle code regressions?