HACKER Q&A
📣 obilgic

How can we measure AI's impact on global developer productivity?


I’m curious whether there’s a meaningful way to quantify how much AI tools (ChatGPT, Claude, Copilot, etc.) are affecting software development productivity worldwide.

Obviously, metrics like “number of repos created,” “number of commits,” or “lines of code changed” are not quality indicators — but could they still reveal directional trends? For example:

Are new repos being created at a faster rate than previous years?

Are commits per developer increasing?

Is the total volume of code changes growing noticeably on platforms like GitHub or GitLab?

Has anyone seen research, datasets, or attempts to measure this kind of global shift? Are there better proxies for AI-driven productivity gains (or losses) in software engineering?

  👤 efortis Accepted Answer ✓
Not an answer, but here’s an interesting talk between Scott Hanselman and Mark Russinovich from Microsoft discussing the senior boost vs. the junior 1.5X slowdown.

https://www.youtube.com/watch?v=A6l23fVG-UE

👤 gregjor
As you note, the software industry lacks meaningful definitions of "productivity." We talk about it as if we agree on what it means, but can't measure it or compare it across teams, projects, or business domains.

Studies I've seen thrown around come from companies such as Microsoft and OpenAI and Anthropic, all of which have deep vested interests in pushing the AI productivity narrative, so I hesitate to trust them. Digging in, their data seems either anecdotal or based on the kinds of mostly-useless metrics you listed above.

We won't know for years, maybe decades, whether AI tools benefit or hinder software development. Probably some combination. I think the most serious follow-on effect of AI in software development will come from the short-sighted decision to hire fewer junior programmers, which will lead to a shortage of experienced senior programmers. Executives and investors won't feel that pain for a while, they can cash in their shares and options today. Supply and demand will raise salaries for senior programmers, and the shortage will impose delays and opportunity costs. Those costs may outweigh whatever short-term and I think superficial gains AI tools may offer now.