Security
AI-generated code: a known flaw in 45% of tasks
Veracode's 2025 GenAI Code Security Report and a METR study are reminders that speed is not enough without a change in method.
According to Veracode's 2025 GenAI Code Security Report, in 45% of the code-generation tasks tested (80 tasks, 100+ models), the model introduced a known OWASP Top 10 security flaw. The code almost always compiles; it is secure in only slightly more than half of cases.
A year later, nothing has moved
The 2026 edition, published on 28 July, confirms the ceiling (Veracode). Across all models tracked since the series began, the security pass rate is 56%, against 55% in the first report: about 44% of tasks still produce a known flaw. The best model in the sample, GPT-5.5, reaches 68%, and six of the eleven newly tested models sit between 50% and 53%.
Three common assumptions do not survive the data:
- Coding-specialised models are not safer: 51% pass rate, against 52% for general-purpose models.
- Size barely matters: 53% for models above 100 billion parameters, 51% for the others.
- Reasoning helps a little: 56% for reasoning models, 51% without.
Where it breaks
The gaps mostly depend on the type of flaw. Models handle SQL injection well (83% pass rate) and pick sound cryptographic algorithms (87%). They fail on cross-site scripting (15%) and log injection (12%), two flaws that require tracking how data moves through the application. Java remains by far the riskiest language, with a mean pass rate of 30% in the 2026 edition, even though it is the only one clearly improving.
Speed, the other blind spot
A randomised controlled trial run by METR in early 2025 measured a 19% slowdown among 16 experienced open-source developers using AI on 246 real tasks (METR). Before the study, they expected a 24% speed-up; afterwards, they still believed they had gained 20%.
In February 2026, METR published results from a second study covering 57 developers and more than 800 tasks (METR). AI-allowed tasks took an estimated 18% less time for developers returning from 2025, and 4% less for new ones, with confidence intervals that still include a slowdown. METR itself calls the result unreliable: many developers refused to work without AI, which probably understates the real gain.
What it changes for developers
- Reviewing and testing generated code is still essential, starting with user input, HTML output and logs.
- Put analysis inside the agent loop: tools such as Sonar with Gitar check code while the agent writes it, not only at the end of the pipeline (see our article).
- Pick models on security too: the gap between a 50% and a 68% pass rate translates directly into remediation work.
- Measure before rolling out: gains come from how work is organised, not only from the tool.
The limits
Veracode tests raw models, without agents, guardrails or human review (press release): in production, those layers reduce the risk. Veracode also sells code analysis tools. METR's studies cover experienced open-source developers working on their own repositories, a setting that does not represent the whole profession.
Sources: Veracode, 2025 GenAI Code Security Report; Veracode, 2026 GenAI Code Security Report, 28 July 2026; METR, July 2025; METR, February 2026.
Spotted an error in this article? Report it · Our corrections policy
