The Models Got Smarter. The Code Did Not Get Safer. - VibeDoctor 
← All Articles 🤖 AI Comparison & Trending High

The Models Got Smarter. The Code Did Not Get Safer.

Veracode: a 56% security pass rate across 100+ AI models tested over four years, up from 55%, while syntax correctness sits near 100%. What that gap means.

SEC-002 SEC-003 SEC-014

Quick Answer

Veracode's 2026 GenAI Code Security Report puts the average security pass rate at 56% across more than 100 AI models tested over four years - up from 55% in its first report. Roughly 44% of code-generation tasks produced code containing a known vulnerability. Meanwhile the same models produce syntactically valid, compilable code close to 100% of the time. The models are getting better at writing code that runs, and not at writing code that is safe. Those are different skills, and only one of them improved.

What The Report Actually Measured

This is worth being precise about, because "AI code is insecure" is a claim people repeat without a number behind it.

Veracode has run standardised code-generation tasks against more than 100 AI models over four years, spanning multiple languages and vulnerability categories. The 2026 dataset added 11 new models across 80 tasks. Crucially, the prompts contain no security-specific instruction. The model is asked to build the thing, not to build the thing securely. That is the realistic condition: it is how you prompt Cursor at 1am, and it is how Bolt and Lovable prompt on your behalf when you describe a feature in plain English.

The headline result: a 56% average pass rate across that four-year corpus, against 55% in the first report. The best-performing model in the summer 2026 dataset, GPT-5.5, reached 68%. That is the ceiling, and it still means the best model available fails roughly one security task in three.

The Gap That Matters: Syntax Versus Security

The single most useful number in the report is not 56%. It is the distance between two figures.

DimensionModel performanceTrend
Produces compilable, syntactically valid code~100%Solved
Produces code free of a known vulnerability56%55% in the first report
Best single model (GPT-5.5, summer 2026)68%Ceiling, not average

Every feedback signal you get while building points at the first row. The code compiles. The dev server starts. The button works when you click it. Vercel deploys green. None of those signals touch the second row, and the second row is the one that has not moved.

This is why "it works" feels like such strong evidence and is such weak evidence. You are reading a metric that was already at 100% and treating it as a proxy for one that is at 56%.

Why The Flat Line Is The Real Story

Model capability improved substantially across the period the report covers. Reasoning benchmarks moved. Context windows grew. Agentic tooling matured to the point that models now open their own pull requests. Security pass rate did not follow.

That is not a mystery once you look at what training optimises for. A model is rewarded for output that satisfies the request. "Add login to my app" is satisfied by a login that logs people in. A missing rate limit does not make the login fail. Neither does a missing httpOnly flag, an over-broad CORS origin, or a query built by string concatenation. The insecure version and the secure version both pass the only test the model is being graded on.

The practical consequence: waiting for the next model release to fix this has been, for a year, a losing strategy. The gap is structural, not a temporary capability shortfall.

What This Looks Like In Your Repo

Here is the shape of it. Ask any current assistant for a search endpoint in a Next.js app and you will frequently get something along these lines.

// ❌ BAD - runs perfectly, ships a SQL injection
// app/api/search/route.ts
export async function GET(request) {
  const { searchParams } = new URL(request.url);
  const q = searchParams.get('q');

  // String interpolation straight into the query
  const results = await db.query(
    `SELECT id, title, body FROM posts WHERE title LIKE '%${q}%'`
  );

  return Response.json(results);
}

This endpoint works. It returns search results. It passes a manual click-through, and a coverage-driven test suite will happily exercise it. It is also the textbook case Veracode's SQL injection category measures, and ?q=' OR 1=1-- is the whole exploit.

The same pattern recurs across the categories the report tracks: unescaped output rendered through dangerouslySetInnerHTML, secrets pulled into a NEXT_PUBLIC_ variable so the client can read them, wildcard CORS because it made a local fetch stop failing.

How To Fix It

The fix at the code level is not exotic. Parameterise the query and let the driver handle escaping.

// ✅ GOOD - parameterised, and the input is validated first
// app/api/search/route.ts
import { z } from 'zod';

const QuerySchema = z.object({ q: z.string().min(1).max(100) });

export async function GET(request) {
  const { searchParams } = new URL(request.url);
  const parsed = QuerySchema.safeParse({ q: searchParams.get('q') });
  if (!parsed.success) {
    return Response.json({ error: 'Invalid query' }, { status: 400 });
  }

  // Parameterised - the driver escapes, the string is never concatenated
  const results = await db.query(
    'SELECT id, title, body FROM posts WHERE title LIKE $1',
    [`%${parsed.data.q}%`]
  );

  return Response.json(results);
}

The harder problem is not knowing the fix. It is knowing which of the several hundred files an assistant generated for you contain the first version. You did not write them, you have not read most of them, and the categories in question produce no runtime symptom until someone goes looking.

That is a scanning problem rather than a knowledge problem. Tools like VibeDoctor's Vibe Check (vibedoctor.io) automatically scan your codebase for injection, exposed secrets and missing input validation, and flag specific file paths and line numbers. Free to sign up.

Whatever you use, the point is to introduce a signal that measures the 56% row rather than the 100% row, because your existing build and deploy pipeline only measures the latter.

What To Do With This Number

Three things follow from the report that are worth acting on.

  1. Prompt for security explicitly. The 56% figure comes from prompts with no security instruction. Asking for parameterised queries, validated input and no client-exposed secrets measurably changes what you get back. It does not solve the problem, and it is free.
  2. Do not treat model choice as the control. The spread between an average model and the best one is 56% to 68%. That is real, and it is not the difference between unsafe and safe.
  3. Put a check between generation and production. The gap is structural, so the mitigation has to be a step in your process rather than a better prompt or a newer model.

FAQ

Does this mean I should stop using AI to write code?

No, and the report does not support that reading. It says AI-generated code carries a materially higher chance of containing a known vulnerability class than most people assume, and that the chance has not fallen as models improved. The response is a verification step, not abstinence.

Is 56% good or bad? It sounds like a coin flip.

It is close to one, which is the point. Read it as: for any given generation task involving a security-relevant decision, there is a little better than even odds the output is clean. Across a few hundred generated files, that compounds into near-certainty that something in your codebase is affected.

Do Bolt, Lovable and v0 do better because they are purpose-built?

They sit on the same underlying models the report tested. A product layer can add guardrails - some do - but the code-generation step inherits the base model's security behaviour. Treat platform output the same way you would treat raw model output.

My app is small and has no users yet. Does this apply?

The vulnerability categories measured here do not scale with user count. A hardcoded key in a public repo or an injectable query is exactly as exploitable on day one as at ten thousand users, and pre-launch is the cheapest possible time to fix either.

Will the next model generation close the gap?

Unknown, and the trend so far argues against assuming it. The pass rate stayed flat across a year in which general capability rose sharply, which suggests the two are not tightly coupled. Planning on the assumption that it resolves itself has not paid off for anyone yet.

Sources

Diagnose your codebase - free

VibeDoctor checks for SEC-002, SEC-003, SEC-014 and 148 other issues across 21 diagnostic areas - security, performance, code quality, and more.

SCAN MY APP →
← Back to all articles View all 149+ checks →