Working AI Backends Are Still Exploitable: What BaxBench Found

What BaxBench Actually Tests

Most AI coding benchmarks ask one question: does the code work? BaxBench, built by researchers at ETH Zurich's SRI Lab and LogicStar.ai, asks two.

Each task gives a model a backend to build, for example a service that streams songs stored in /var/songs, or one that turns a zip file into a single text file. The model writes the whole backend. BaxBench runs it in an isolated Docker container and checks it twice:

• Functional tests call the API and check the answers. Pass them and the backend counts as correct.
• Security exploits then attack the same running backend through its endpoints. Some go further, such as reading the app's SQLite database to see whether secrets were stored safely.

A backend counts as secure only if it passes the tests and survives every exploit. That makes each failure stronger evidence than a linter warning: it is an attack that worked, not a pattern that looked risky.

The benchmark has 392 tasks: 28 scenarios, each built in 14 backend frameworks across 6 languages (Go, JavaScript, PHP, Python, Ruby and Rust). The paper tested 11 models in 2025. The public leaderboard now lists 38.

The Numbers

The leaderboard gives two figures per model: how often the backend works, and how often it works and survives the exploits. The gap between them is the share of working code an attacker could break. Here are four well-known models with no security advice in the prompt:

Read the Opus row again. It is the best result on the board, and in 44 of every 100 tasks its backend was either broken or exploitable. GPT-5 had the smallest exploitable share of the four, but it built fewer working backends, so it finished two points behind on working and secure.

The average covers the 37 models on the board that built at least one working backend. The 2025 paper reported the same thing: exploits succeeded on around half of the correct programs from each model. Newer models were not on the leaderboard when we checked, so treat these as the latest public numbers, not a ceiling.

What Got Exploited (Mostly Not SQL Injection)

The paper breaks its results down by task and by weakness type (CWE). Each figure below is the share of working backends that had the weakness, counting only tasks where it was possible. The prompts had no security advice, and the models are the ones the paper tested in 2025.

SQL injection, the bug most checklists start with, barely shows up. It was 0% in the Login, ProductCatalog and Wiki tasks, and 21% in Forum, its worst case. The models seem to have learned parameterized queries.

What they have not learned is everything around the query: escaping user text before it goes back out as HTML, keeping a file name inside its folder, protecting stored passwords and secrets, stripping line breaks before writing a log line, and limiting how much work one request can cause. Cross-site scripting hit between 79% and 99% of working backends in five of the tasks where it was possible. If your review stops at SQL injection, you are checking the one thing the models mostly get right.

What It Looks Like in Your Code

Take the SongDownloader task: stream songs stored in /var/songs to the user. In the paper, 73% of the working backends for it had path traversal. Here is the kind of Express route that passes every functional test:

// Works for every song, and for ../../etc/passwd
app.get('/songs', (req, res) => {
  const file = path.join('/var/songs', req.query.filename);
  fs.createReadStream(file).pipe(res);
});

Every real song streams, so the tests pass. Then someone sends this:

curl "http://localhost:3000/songs?filename=../../etc/passwd"

path.join resolves the ../ segments, and the route streams /etc/passwd. It builds paths, it does not keep them inside a folder. The route has a second bug: ask for a file that does not exist and the unhandled stream error crashes the whole Node process.

const SONGS_DIR = '/var/songs';

app.get('/songs', (req, res) => {
  const name = path.basename(String(req.query.filename ?? ''));
  if (!/^[\w-]+\.mp3$/.test(name)) {
    return res.status(400).json({ error: 'Invalid file name' });
  }
  const stream = fs.createReadStream(path.join(SONGS_DIR, name));
  stream.on('error', () => res.status(404).end());
  stream.pipe(res);
});

path.basename drops any folder parts, the pattern accepts only plain .mp3 names (adjust it to your own file names), and a missing file returns 404 instead of taking the server down.

Does Telling the Model to Be Secure Help?

BaxBench ran every task three ways: with no security advice, with a generic reminder to follow security best practices, and with an "oracle" prompt that names the exact weaknesses the task is exposed to.

Two lessons. First, naming the specific risk works far better than a general "be secure". You can only name the risk if you know which ones your app has, and the checklist below is for finding them.

Second, the reminder is not free. With the generic reminder, GPT-5 built fewer working backends (58% instead of 71%) and ended up with fewer that were working and secure (46% instead of 54%). The paper also found that reasoning models gained a lot from the generic reminder, while the other models gained little.

How to Check Your Own Backend

You can test your own app for the same weaknesses, whether Cursor, Claude Code, Bolt or Lovable wrote it. For every endpoint, ask five questions:

1. Does user input become a file path? Send ../../etc/passwd as the file name and see what comes back.
2. Does user text come back as HTML? Save a profile name or comment like <img src=x onerror=alert(1)>, then open the page that shows it. If an alert appears, it is exploitable.
3. How are passwords and secrets stored? Open the database. A password should be a bcrypt or argon2 hash, never the plain text or a reversible value. API keys you keep for users should be encrypted.
4. Does user input reach your logs? Put a line break in a username and check whether it creates a fake second log line.
5. Can one request tie up the server? Send a huge upload, a zip that expands to gigabytes, or a pattern the server runs as a regex, and check whether anything sets a limit.

Then put what you find into your prompts. "Keep file names inside /var/songs" beats "make it secure", as the oracle numbers show.

A scanner cannot replace those requests. It reads code, it does not attack a running app. What it can do is tell you which lines to try first. VibeDoctor (vibedoctor.io) traces request input into file paths and into HTML sinks such as innerHTML and dangerouslySetInnerHTML in JavaScript and TypeScript, and flags missing input validation and insecure file uploads. We ran its path traversal rule on the two routes above: it flagged the first on the createReadStream line and passed the fixed one. To have your agent check its backend code as it writes it, paste this into Claude Code, Cursor or Codex:

connect to vibedoctor using https://vibedoctor.io/mcp/start

Not using an agent? Connect your GitHub repository on vibedoctor.io for a free scan.

FAQ

Sources

• Vero et al., "BaxBench: Can LLMs Generate Correct and Secure Backends?" (arXiv 2502.11844, revised May 2025): 392 tasks, 28 scenarios, 14 frameworks, 6 languages; exploits succeed on around half of correct programs; per-task weakness tables (Table 6, no security instructions).
• BaxBench leaderboard: 38 models, three prompt settings; data file dated May 22, 2026. "Working but exploitable" is our calculation: 1 minus (correct and secure / correct).
• BaxBench on GitHub: benchmark code, scenarios and exploits.