Measuring Developer Productivity in the Age of Agents

The problem

Why your dashboard is lying to you

One developer thinks for three days, then writes 50 lines that delete a system bottleneck. Another ships 500 lines of plumbing every single day. Most dashboards crown the second one, though every experienced engineer knows the first did the harder work. Volume was never the same thing as value. It just photographed better.

AI made this worse. Agents inflate code volume, PR frequency, and test coverage on command, so the metrics that were merely incomplete now actively mislead.

The trap

Why token usage kills real measurement

Token consumption gets tracked because it is easy to pull from an API, but it is an input wearing an output’s clothes. It shows consumption, not creation. A developer burning through tokens in hallucination loops scores higher than someone writing tight prompts that ship features. That rewards verbosity and punishes the precision that actually signals expertise.

Key insight: track tokens for cost management only. Never for performance.

The shift

When agents write code, judgment becomes the work

Developers now architect systems, write prompts, supervise agents, review output, and ship features, doing everything that matters except the typing. Their real skill shows up in three places: debugging generated code, explaining generated code, and articulating the tradeoffs an agent made on their behalf.

The toolkit

Seven measurable signals

Every one of these clusters around the same act: debugging, explaining, and understanding code a human did not write. That act is the job now, and it has a price. I gave that price a name in The Verification Tax. These seven are how you spot it on your own team.

1. Time to resolution on AI generated bugs — how quickly developers find and fix broken AI code in production or QA

2. Hallucination catch rate in code review — how often invented APIs, fake imports, and functions that do not exist get caught before merge

3. Code explanation depth, scored 1 to 3 — quality of the walkthrough in a 1:1 or design review

4. Code review comment depth, scored 1 to 3 — average quarterly score on surface versus structural versus systemic feedback

5. Architecture decision records authored — count of design docs that name the chosen approach, the rejected alternatives, and the explicit tradeoffs

6. Prompt iteration count per shipped feature — how much back and forth an agent needed before its output was acceptable

7. Code churn rate on AI generated code — percentage of AI lines refactored within 30 days of merge

A quick note on names, because I use two that sound alike across these posts. Code churn rate here is a post merge signal: work that shipped and then had to be redone. In The Verification Tax I track rework rate, which is a pre merge signal: how many times a human bounced the work back before signing off at all. Different moments, different instruments, both worth watching.

The recommendation

Pick three or four that fit your team

If you only pick three, make them time to resolution on AI bugs, code review depth, and code churn rate. Together they tell you whether your developers actually understand what they are shipping.

If you want the shortest possible starting set, and the argument for why all this checking is a bill you are already paying, that is in The Verification Tax: rework rate, discard rate, and context miss rate, three numbers you can track by Monday.

Stop measuring: PR count, token usage, lines of code.

Start measuring: outcomes.