• SPONSORED BY CODERABBIT Companies are looking to harness agentic code generators to get software built faster. • But for every story of increased developer productivity or greater code base understanding, there’s a story about creating more bugs and the increased likelihood of production outages. • Here at CodeRabbit, we wanted to know if the problems people have been seeing are real and, if so, how bad they are. • We’ve seen data and studies about this same question, but many of them are just qualitative surveys sharing vibes about vibe coding. • This doesn’t show us a path to a solution, only a perception. • We wanted something a little more actionable with actual data.
Article Summaries:
- CodeRabbit’s “State of AI vs. Human Code Generation” report analyzed 470 open‑source GitHub pull requests to compare bugs introduced by AI agents and human developers. The study found that AI‑generated code produced 1.7 times more bugs overall, with 1.3‑1.7 times more critical and major issues. Logic and correctness errors were 75 % higher in AI PRs, totaling 194 incidents per 100 PRs, and included dependency, configuration, and control‑flow mistakes that are hard to spot in reviews. AI also introduced more security flaws (1.5‑2×), concurrency errors (≈2×), and readability problems (≈3×). While not all outages can be directly linked to AI, the surge in AI coding coincided with a spike in production incidents in 2025.
Sources: