Claude, GPT, and Gemini: Which AI Writes the Safest Code?

Why Model Choice Affects Code Security

Not all AI models are trained equally on security-focused code. The training data, RLHF tuning, and system prompts each model uses determine how likely it is to produce secure patterns by default. CodeRabbit's State of AI vs Human Code Generation Report (December 2025) found security issues up to 2.74x higher in AI co-authored pull requests than in human-only ones, and model choice matters too: in Veracode's 2026 GenAI Code Security Report, security pass rates ranged from 50% to 68% across the 11 models on its leaderboard.

A Stanford study (Perry et al., ACM CCS 2023) found that developers using AI assistants wrote less secure code while believing it was more secure. This confidence gap exists regardless of model, but some models reinforce it more than others by producing code that looks clean and professional while hiding critical security gaps.

We tested each model by asking it to generate common web application patterns: authentication flows, API endpoints, database queries, file uploads, and payment integrations. Each output was scanned against 10 security checks that map to real-world vulnerabilities.

Head-to-Head: Security Check Results

Where Claude Excels

Claude consistently handles secret management and input validation better than the other models. When generating Next.js API routes, Claude uses server-side environment variables by default and adds Zod schema validation without being prompted. It also produces secure cookie configurations with httpOnly, Secure, and SameSite flags.

However, Claude still fails on authentication middleware and rate limiting. It generates fully functional endpoint logic but does not wrap routes in auth checks unless explicitly asked. This means a Claude-generated API is internally secure but externally wide open.

// Claude's typical output - validates input but no auth middleware
export async function POST(req: Request) {
  const body = await req.json();
  const parsed = createUserSchema.safeParse(body);
  if (!parsed.success) {
    return Response.json({ error: parsed.error.flatten() }, { status: 400 });
  }
  // Business logic with validated data...
}

Where GPT-4 Falls Short

GPT-4 produces the most complete and readable code, which makes its security gaps harder to spot. It generates clean architecture with proper typing and documentation, but skips security fundamentals. The most common GPT-4 pattern is exposing secrets via NEXT_PUBLIC_ prefixes and sending raw error.message in API responses.

GitGuardian's State of Secrets Sprawl 2024 found that 12.8 million new secrets were leaked publicly on GitHub in 2023, and a model that suggests public environment variable prefixes for server secrets adds one more way for keys to leak. Veracode's State of Software Security 2023 found insufficient input validation flaws in 41.6% of applications scanned with static analysis, and GPT-4 code that skips validation belongs in that group.

// ❌ GPT-4 typical output - clean code, no validation, leaks errors
export default async function handler(req, res) {
  try {
    const { email, password } = req.body;  // No validation
    const user = await prisma.user.create({
      data: { email, password }  // No hashing either
    });
    res.status(201).json(user);
  } catch (error) {
    res.status(500).json({ error: error.message });  // Leaks internal error details
  }
}

Gemini's Unique Weaknesses

Gemini produces functional code but has a distinct weakness with SQL query construction. While Claude and GPT-4 generally use parameterized queries, Gemini frequently falls back to string interpolation, especially in more complex query patterns like search filters and dynamic ordering.

Gemini also tends to use Math.random() for generating tokens and session IDs rather than crypto.randomUUID(). This is a critical vulnerability because Math.random() is not cryptographically secure and its output can be predicted.

// ❌ Gemini typical output - string interpolation in queries
const searchUsers = async (name, sort) => {
  const query = `SELECT * FROM users 
    WHERE name LIKE '%${name}%' 
    ORDER BY ${sort}`;
  return await db.query(query);
};

// ❌ Gemini uses Math.random for tokens
const generateToken = () => {
  return Math.random().toString(36).substring(2) + Date.now().toString(36);
};

// ✅ GOOD - Parameterized queries and secure randomness
const searchUsers = async (name, sort) => {
  const allowedSorts = ['name', 'created_at', 'email'];
  const sortColumn = allowedSorts.includes(sort) ? sort : 'created_at';
  return await db.query(
    `SELECT * FROM users WHERE name LIKE $1 ORDER BY ${sortColumn}`,
    [`%${name}%`]
  );
};

const generateToken = () => {
  return crypto.randomUUID();
};

How to Scan Regardless of Model

The model comparison matters less than whether you scan at all. Every model produces insecure code frequently enough that manual review alone is insufficient. Tools like VibeDoctor's Vibe Check automatically scan your codebase for most of the vulnerabilities in this comparison - SQL injection, client-side secrets, missing validation, unprotected Express, Fastify and NestJS routes, CORS wildcards, insecure cookies, unchecked file uploads and Math.random() tokens - and flag specific file paths and line numbers. Free to sign up.

The safest workflow is to use whichever model you prefer for productivity, then scan the output before committing. No model is secure enough to skip scanning.

FAQ