INSIGHT 25: Agentic PRs Have a Different Shape

Agent-written code is no longer a few anecdotes from people trying Claude Code on side projects. AIDev gives the phenomenon ecosystem scale. "How AI Coding Agents Modify Code" compares agentic and human merged PRs. The readability paper mines readability-intended agent commits.

This matters for the article because an agent-friendly codebase is not just a repo that helps an agent write code. It is also a repo that can review, constrain, and absorb agent-shaped changes. Agent patches have measurable patterns. Some are useful. Some are biased. The repo should be designed around those observed biases.

Plot-ready data lives in presentations/write-code-ai-agents-love/research/data/agentic_pr_change_shape.csv.

Source map

AIDev: agentic PRs are now observable at scale

AIDev is a dataset paper. Its importance is not a single performance number. Its importance is that agent-authored PRs are now numerous enough to study empirically. That changes the tone of the blogpost. We do not need to argue only from personal experience. We can say: agentic development has a measurable footprint in public repositories.

AIDev data copied from the paper

Source trace: R69, paper-text/aidev-agentic-prs-2602.09185.txt.

Graph sketch: AIDev data layers

The inference I want to make carefully: AIDev does not prove agentic PRs are good or bad. It proves there is enough agentic PR activity to study patterns. That supports using ecosystem data rather than relying only on "vibes."

How AI Coding Agents Modify Code: patch shape differs

This paper compares merged Agentic PRs and merged Human PRs. It reports statistical and practical differences across commit counts, files touched, additions, deletions, and line changes. The most useful number is Cliff's delta because it describes distributional separation, not just mean difference.

I need to be cautious about direction. The text summary says human PRs modify codebases more broadly and remove more code, while the table reports effect sizes for agentic vs human metrics. Before using a visual in the article, re-open the exact table and label the direction from the paper, not from memory. For this research note, the safe claim is that the distributions differ with small to large practical effects.

Dataset comparison copied from the paper

Effect-size data copied from the paper

Similarity data copied from the paper

Source trace: R70, paper-text/how-ai-coding-agents-modify-code-2601.17581.txt.

Chart sketch: practical effect sizes

The blog inference: code review systems should not treat an agent PR as "just a normal PR with an AI author." The patch shape, similarity profile, and failure modes may differ. Review gates should watch the patterns that matter:

• more or fewer commits than expected for task size;
• broad file touches;
• low deletion or suspicious compatibility shims;
• generated files edited by hand;
• tests changed without implementation;
• implementation changed without tests;
• public API changed without generated clients/docs;
• architecture boundaries crossed.

Readability paper: even "readability" intent can degrade metrics

The readability paper is a useful corrective because it looks at commits that explicitly claim readability/understandability/clarity-like intent. If even those commits often increase size or reduce maintainability metrics, then "the agent cleaned it up" is not enough.

The paper mines AIDev for readability-related commits, classifies what smells they address, and computes static metric deltas before and after.

Readability-paper data copied from the paper

What readability commits addressed

The full table has more categories, but these are the numbers most relevant to the blog point: agents emphasize logic complexity and documentation more than surface-level naming/formatting.

Metric deltas copied from the paper

Directional outcome shares copied from the paper

Source trace: R76, paper-text/readability-agents-2603.13723.txt.

Chart sketch: readability-intended does not mean structurally simpler

This is not a claim that the agent changes were all bad. Static metrics are imperfect. Sometimes a readable change legitimately adds lines or comments. But the result is strong enough to justify review gates: readability claims should be checked against complexity, size, duplication, and actual human clarity, not accepted as self-evident.

Link to maintainability benchmarks

SWE-CI and Needle in the Repo broaden the same concern. SWE-CI finds that many models can look good on style metrics while underperforming on Maintainability Index. Needle in the Repo finds functional test passes can still violate structural oracles.

Supporting maintainability data from earlier insights

Source traces: R09 and R46.

Inference for codebase design

Agent-friendly codebases should assume agent PRs are productive but biased. They should make the desired patch shape visible and enforce the unacceptable patch shapes.

How this changes the article

The article should not sound like "make the repo nice so agents can move faster." It should say:

1. Agents are a new contributor class.
2. They have observable patch-shape and quality tendencies.
3. The repo needs affordances for generation and guardrails for review.
4. The same artifacts help both: types, tests, generated clients, lint rules, package boundaries, setup commands, and explicit task specs.

This gives the talk a more serious engineering tone. It is not "AI agents love clean code." It is "AI agents need operational constraints because their changes have different statistical failure surfaces."

What I should not claim

I should not claim all agent PRs are worse. AIDev is a dataset; it does not score quality by itself. The agent-vs-human PR paper compares merged PRs, which have already passed some project review process. The readability paper is focused on readability-keyword commits and Python metrics.

I should not overclaim static readability metrics. They are signals, not truth. A larger patch can be more readable. A lower Maintainability Index can sometimes be an artifact. The safe claim is that readability intent does not reliably imply structural quality improvement.

Blog visual candidates

1. AIDev scale graph: PRs -> repos -> users -> curated subset.
2. Agentic-vs-human Cliff's delta bar chart.
3. Readability commit outcomes bar chart.
4. Review-gate matrix: observed agent PR risk -> codebase guardrail.
5. Patch-shape dashboard mock: files touched, generated files, tests changed, boundary violations.

References

• R09: SWE-CI, paper-text/swe-ci-2603.03823.txt
• R46: Needle in the Repo, paper-text/needle-in-the-repo-2603.27745.txt
• R69: AIDev, paper-text/aidev-agentic-prs-2602.09185.txt
• R70: How AI Coding Agents Modify Code, paper-text/how-ai-coding-agents-modify-code-2601.17581.txt
• R76: Do AI Agents Really Improve Code Readability?, paper-text/readability-agents-2603.13723.txt