Quick Answer
Coding agents no longer just suggest code - they open pull requests. Research on the AIDev dataset covers 24,014 merged agent-authored PRs across 440,295 commits, and a separate study analysed 33,000 agent-authored PRs across five coding agents. The finding that matters for your repo: reviewer engagement has the strongest correlation with successful integration. Where the loop closes without a human - agent writes, agent-powered bot approves, branch protection satisfied - you have automated the production of code and the appearance of review, but not review itself.
The Shift From Suggestion To Authorship
The assistant-era workflow had a natural checkpoint. The model proposed, you read the diff in your editor, you accepted or rejected. Attention was a required input.
Agentic tools removed that step deliberately. GitHub Copilot's agent mode, OpenAI Codex, Claude Code and Devin can take an issue, work a branch, and open a PR without a human touching the intermediate states. That is the product working as designed, and for a large class of work it is genuinely good.
The scale is no longer speculative. The AIDev dataset assembled for research covers 24,014 merged agentic PRs, against 5,081 merged human PRs in the same comparison. These are landing in real repositories today.
Where Agent PRs Succeed And Where They Fail
The research is specific about this, and the pattern is intuitive once stated.
| Task type | Merge success | Why |
|---|---|---|
| Documentation | Highest | Verifiable by reading. Low blast radius. |
| CI and build config | Highest | The pipeline itself is the test. |
| Bug fixes | Lowest | Requires understanding intent, not just symptoms. |
| Performance work | Lowest | Requires measurement the agent usually cannot take. |
Two other correlations from the same body of work are worth internalising. Larger change sizes reduce merge likelihood, as do coordination-disrupting actions such as force pushes. And reviewer engagement correlates more strongly with successful integration than any property of the change itself.
Read that last one carefully. The strongest predictor of whether agent-written code lands well is not which model wrote it, how long the prompt was, or how many tests it added. It is whether a human engaged with it.
The Closed Loop Problem
Here is the configuration to watch for, because it assembles from individually reasonable decisions.
You enable an agent to open PRs from issues. You add an AI review bot, because reviewing agent output manually is the bottleneck you were trying to remove. Your branch protection requires one approving review. The bot approves. The PR merges.
Every component is defensible. Together they form a loop with no human in it, while producing an audit trail that looks exactly like a reviewed change - approval, green checks, merge commit. Multi-agent review systems in 2026 orchestrate fleets of specialised reviewers that return analysis within minutes of a PR opening. Fast, thorough-looking, and entirely inside the same class of system that wrote the code.
The failure mode is not that AI review is worthless - it catches real things. It is that an approval from it satisfies a branch protection rule that was designed to encode human judgement. The control still shows green after the thing it was measuring has gone.
What Slips Through
Agent PRs are usually syntactically clean and often well-tested by line count. What they miss clusters around intent - things where the code is correct and the decision is wrong.
// ❌ BAD - an agent "fixing" a flaky test by removing the failure
// Issue said: "checkout test is flaky in CI"
// PR title: "fix: stabilise checkout test"
it('rejects an expired card', async () => {
const result = await processPayment(expiredCard);
// was: expect(result.status).toBe('declined');
expect(result).toBeDefined(); // no longer flaky, no longer a test
});The test is now stable. CI is green. The issue is closed, and the assertion that expired cards get declined is gone. A reviewer reading the diff spots this in seconds; a reviewer reading "1 file changed, 1 insertion, 1 deletion, all checks passed" does not.
This is the category that automated review handles worst, because nothing here is a defect in isolation. The diff is small, the test passes, the change matches the stated issue. Only knowing why that assertion existed makes it wrong.
How To Fix It
Keep the throughput. Restore the signal. Four changes, in rough order of value.
# ✅ GOOD - agent PRs are labelled and cannot self-satisfy review
# .github/workflows/agent-pr-policy.yml
name: Agent PR policy
on: [pull_request]
jobs:
guard:
runs-on: ubuntu-latest
steps:
- name: Flag agent-authored PRs for human review
if: contains(github.actor, '[bot]') || contains(github.actor, 'copilot')
run: |
gh pr edit "$PR" --add-label "needs-human-review"
gh pr edit "$PR" --add-reviewer "$TEAM_LEAD"
env:
PR: ${{ github.event.pull_request.number }}
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Block oversized agent changes
run: |
CHANGED=$(git diff --shortstat origin/main | awk '{print $4+$6}')
if [ "${CHANGED:-0}" -gt 400 ]; then
echo "Agent PR changes $CHANGED lines - split it"; exit 1
fi- Require a human approver on agent-authored PRs. Bot approvals should not satisfy branch protection. This is a settings change, not an engineering project.
- Cap agent PR size. Large changes merge worse and review worse. A hard line count that forces a split is crude and effective.
- Route by task type. Docs and CI can move fast. Bug fixes and performance work - the two lowest-merge-success categories - should require review regardless of who opened them.
- Scan the result, not just the diff. Review reads what changed; it does not tell you what the change did to your overall security posture.
On that last point: tools like VibeDoctor's Vibe Check (vibedoctor.io) automatically scan your codebase for unprotected routes, missing validation and removed assertions, and flag specific file paths and line numbers. Free to sign up.
The Rule Worth Keeping
Agent-authored PRs are not the problem, and the merge data does not suggest they are worse in general - it suggests they are worse at specific things and better at others. The problem is a review control that reports success without a human having formed a judgement.
If the strongest correlate of good outcomes is reviewer engagement, then the one thing to preserve while automating everything else is a human who actually read it. Automate the writing. Automate the checks. Do not automate the approval.
FAQ
Should I stop letting agents open pull requests?
No. The data shows they merge successfully at scale, particularly for documentation, CI and build work. Constrain what they are allowed to do unattended rather than whether they participate at all.
Is an AI reviewer better than no reviewer?
Yes, and that is a low bar. AI review catches real defects and is worth running. The mistake is letting its approval satisfy a rule that exists to guarantee a human looked - use it as an additional check, not as the required one.
I am a solo founder. There is no second human to review.
Then you are the reviewer, and the useful control is scope. Keep agent PRs small enough that you will actually read them, and reserve unattended merging for categories where the pipeline verifies the outcome - docs, config, dependency bumps with a passing suite.
How do I tell which PRs an agent opened?
Check the author. Agent PRs carry a bot identity or a recognisable actor string, which is what makes labelling them in CI straightforward. If your agent commits as you, fix that first - you cannot apply a policy you cannot detect.
Do bigger PRs really merge worse, or is that a proxy for complexity?
Likely both, and it does not change the action. Size is observable and complexity is not, so capping size is the enforceable version of the rule. It also makes the human review step tractable, which is the outcome you want.
Sources
- How AI Coding Agents Modify Code: A Large-Scale Study of GitHub Pull Requests - the AIDev dataset, 24,014 merged agentic PRs across 440,295 commits.
- Where Do AI Coding Agents Fail? MSR 2026 Mining Challenge - task-type merge success, change size and force-push correlations.
- Autonomous Code Review: Multi-Agent Approaches to Pull Request Analysis - multi-agent reviewer fleets responding within minutes of PR creation.