Claude, GPT, and Gemini: Which AI Writes the Safest Code?

Why Model Choice Affects Code Security

Not all AI models are trained equally on security-focused code. The training data, RLHF tuning, and system prompts each model uses determine how likely it is to produce secure patterns by default. Apiiro's 2025 research found that AI-generated code contains security vulnerabilities at 2.74x the rate of human-written code, but that rate varies significantly between models.

A 2024 Stanford study confirmed that developers using AI assistants wrote less secure code while believing it was more secure. This confidence gap exists regardless of model, but some models reinforce it more than others by producing code that looks clean and professional while hiding critical security gaps.

We tested each model by asking it to generate common web application patterns: authentication flows, API endpoints, database queries, file uploads, and payment integrations. Each output was scanned against 10 security checks that map to real-world vulnerabilities.

Head-to-Head: Security Check Results

Where Claude Excels

Claude consistently handles secret management and input validation better than the other models. When generating Next.js API routes, Claude uses server-side environment variables by default and adds Zod schema validation without being prompted. It also produces secure cookie configurations with httpOnly, Secure, and SameSite flags.

However, Claude still fails on authentication middleware and rate limiting. It generates fully functional endpoint logic but does not wrap routes in auth checks unless explicitly asked. This means a Claude-generated API is internally secure but externally wide open.

// Claude's typical output - validates input but no auth middleware
export async function POST(req: Request) {
  const body = await req.json();
  const parsed = createUserSchema.safeParse(body);
  if (!parsed.success) {
    return Response.json({ error: parsed.error.flatten() }, { status: 400 });
  }
  // Business logic with validated data...
}

Where GPT-4 Falls Short

GPT-4 produces the most complete and readable code, which makes its security gaps harder to spot. It generates clean architecture with proper typing and documentation, but skips security fundamentals. The most common GPT-4 pattern is exposing secrets via NEXT_PUBLIC_ prefixes and sending raw error.message in API responses.

GitGuardian's 2024 report found that 12.8 million new secrets were exposed in code repositories in 2023, and GPT-4's tendency to suggest public environment variable prefixes contributes to this. According to the Veracode 2024 State of Software Security report, input validation flaws are present in 63% of applications - GPT-4 generated code is firmly in that majority.

// ❌ GPT-4 typical output - clean code, no validation, leaks errors
export default async function handler(req, res) {
  try {
    const { email, password } = req.body;  // No validation
    const user = await prisma.user.create({
      data: { email, password }  // No hashing either
    });
    res.status(201).json(user);
  } catch (error) {
    res.status(500).json({ error: error.message });  // Stack trace leak
  }
}

Gemini's Unique Weaknesses

Gemini produces functional code but has a distinct weakness with SQL query construction. While Claude and GPT-4 generally use parameterized queries, Gemini frequently falls back to string interpolation, especially in more complex query patterns like search filters and dynamic ordering.

Gemini also tends to use Math.random() for generating tokens and session IDs rather than crypto.randomUUID(). This is a critical vulnerability because Math.random() is not cryptographically secure and its output can be predicted.

// ❌ Gemini typical output - string interpolation in queries
const searchUsers = async (name, sort) => {
  const query = `SELECT * FROM users 
    WHERE name LIKE '%${name}%' 
    ORDER BY ${sort}`;
  return await db.query(query);
};

// ❌ Gemini uses Math.random for tokens
const generateToken = () => {
  return Math.random().toString(36).substring(2) + Date.now().toString(36);
};

// ✅ GOOD - Parameterized queries and secure randomness
const searchUsers = async (name, sort) => {
  const allowedSorts = ['name', 'created_at', 'email'];
  const sortColumn = allowedSorts.includes(sort) ? sort : 'created_at';
  return await db.query(
    `SELECT * FROM users WHERE name LIKE $1 ORDER BY ${sortColumn}`,
    [`%${name}%`]
  );
};

const generateToken = () => {
  return crypto.randomUUID();
};

How to Scan Regardless of Model

The model comparison matters less than whether you scan at all. Every model produces insecure code frequently enough that manual review alone is insufficient. Tools like VibeDoctor (vibedoctor.io) automatically scan your codebase for all of the vulnerabilities in this comparison - SQL injection, secret exposure, missing validation, authentication gaps, and more - and flag specific file paths and line numbers. Free to sign up.

The safest workflow is to use whichever model you prefer for productivity, then scan the output before committing. No model is secure enough to skip scanning.

FAQ