What gets measured as engineering productivity now that lines-of-code and PR count are gamed by agents?

The shift

Producing plausible, syntactically valid code, tests, and PR descriptions goes from a slow, effortful human act to something an agent emits continuously and near-free — so any metric built on counting output artifacts (lines, commits, PRs, tickets closed) stops correlating with the effort or scarcity it used to proxy for.

The axioms

• Lines of code and PR count correlate with engineering effort and output. Rests on: writing code is slow and effortful, so more of it signals more work done.
• A metric is hard to game if gaming it takes as much effort as doing the real work. Rests on: producing plausible-looking artifacts (code, tests, commits) is itself scarce and effortful.
• Individual output is a fair unit to measure, rank, and compensate against. Rests on: output is bottlenecked by the individual's own scarce drafting speed and judgment, so it reflects them specifically.
• Code review and CI catch what a raw output metric can't, so counting output alongside review is enough oversight. Rests on: review capacity is sized correctly relative to how much output humans could plausibly generate.
• Manager-legible proxies (velocity, throughput, cycle time) stand in for the harder-to-observe thing — whether the software works, ships value, and stays maintainable. Rests on: directly observing "did this help the business" is expensive and slow, so cheap proxies are the practical substitute.
• Story points and sprint velocity measure a team's real capacity and improve forecasting over time. Rests on: the unit of work (a ticket, a point) maps to a roughly stable amount of scarce human effort.

Invalid axioms

1. Lines of code, commits, and PR count measure output or effort. Generating large volumes of plausible code, splitting it into many small commits, and drafting PR descriptions is exactly what agents do fastest and cheapest — the artifact volume that used to require scarce human drafting time is now abundant and can be inflated on demand. The habit-trap: dashboards, perf cycles, and "who shipped the most" leaderboards still tally these counts as if they still cost what they used to cost to produce.
2. A metric resists gaming because faking it is roughly as much work as doing the real thing. That assumption held when producing a plausible-but-hollow PR took nearly as long as producing a real one. An agent can generate ten superficially reasonable PRs, each individually low-effort to produce, faster than a reviewer can meaningfully assess one. The habit-trap: teams keep trusting self-reported or automatically-logged activity metrics because gaming them "used to be too much effort to bother," a floor that no longer holds.
3. Individual output volume is a fair unit for ranking and comparing engineers. Once an agent does most of the drafting, output volume increasingly reflects how aggressively someone runs agents and how loose their bar for merging is, not their own scarce skill. The habit-trap: perf calibration and stack-ranking still anchor on tickets-closed or PRs-merged as an individual signal, rewarding agent-prompting volume over judgment.
4. Sprint velocity (points completed per sprint) is a stable capacity signal for forecasting. Point estimates assumed the effort to build a ticket was bounded by scarce human drafting speed; when a chunk of that drafting is agent-generated, the same point value can now be delivered in a fraction of the time, or inflated by generating more, smaller tickets. The habit-trap: planning still treats historical velocity as a physical capacity constant instead of re-baselining it against how much of the work is now agent-assisted.

Unchanged axioms

1. Someone is accountable when the shipped change breaks production, leaks data, or causes an incident. No output metric, gamed or not, changes who takes the page, who explains the outage to a customer, or whose name is on the postmortem. Accountability is not a counting problem and doesn't move with generation volume.
2. Whether the software actually works, holds up under load, and solves the customer's problem still requires verification against ground truth. A metric can be gamed; whether the checkout flow actually processes payments correctly cannot be faked at scale without eventually surfacing in outcomes — incidents, churn, support load. Outcome-based signals (uptime, defect escape rate, customer-reported bugs) stay honest even as activity metrics stop being trustworthy, because they're anchored to reality, not self-reported effort.
3. Judging whether a specific piece of work was hard, risky, or high-leverage requires a human who understands the system, not a formula. Distinguishing a trivial agent-generated config change from a gnarly concurrency fix that happened to be small in diff size takes contextual judgment that doesn't reduce to any single number, gamed or not. This is qualitatively the same problem good engineering managers always had — it just gets more load-bearing once the easy proxies fail.
4. Trust that a given engineer's approval on a risky change means something is earned over time, not generated per-review. As review volume increases, orgs still rely on specific people's track record and standing to gate the riskiest changes — that reputation is scarce and can't be manufactured by an agent or backdated.

New axioms

1. When every cheap-to-produce metric is gameable, what replaces it hasn't been agreed on, and premature convergence on the wrong replacement (e.g., "time saved" self-reports, or agent-usage counts) risks becoming the next gamed number. Candidates like deployment frequency, change-failure rate, incident-linked defect rates, or customer-facing outcome metrics are harder to fake, but they're laggier, noisier, and easier to blame on factors outside an individual's control — nobody has settled on a set that's both hard to game and fast enough to act on sprint-to-sprint.
2. Measuring "how much review and verification happened," not just "how much was shipped," is a metric category that barely existed before and now has to scale with generation volume. If review is the new bottleneck, someone has to instrument review depth, override rates, and post-merge defect attribution without that instrumentation itself becoming a box-ticking exercise agents or humans learn to satisfy superficially.
3. Attributing outcomes to a specific person gets harder exactly when performance management still wants an individual number. When a PR is agent-drafted, human-directed, and reviewer-modified, deciding whose judgment actually produced the good or bad outcome is an open problem — perf and comp systems built for single-author attribution don't have a clean answer yet.
4. Cross-company and industry benchmarking (used for hiring bars, comp bands, and "how productive should a team of N be") loses its baseline the moment the underlying unit — a line, a PR, a story point — stops meaning the same thing everywhere, and no shared replacement metric exists yet to recalibrate against.

Where it breaks

"PR count and velocity measure output" (invalid) collides with "outcome-based signals are laggy and noisy" (new): teams that drop gamed activity metrics without a fast substitute go metric-blind for weeks or months, and in that gap either revert to counting artifacts again out of habit, or lose the ability to have any data-informed conversation about performance at all.

"Individual output volume is a fair ranking unit" (invalid) collides with "attributing outcomes to a specific person gets harder" (new): perf cycles still want one number per engineer, but once drafting is agent-assisted and review is collective, the honest answer is that good outcomes are increasingly a property of the human-agent-review system, not the individual — and comp and promotion processes haven't built a model for rewarding a system's output instead of a person's.

ai-axioms.vercel.app/axioms/engineering/swe-productivity-metrics-ai-axioms