How do you assess developer productivity?

It’s the ultimate mistake people keep making about programmers’ work.

That it’s linear or at least can be approximated with some linear formula, even if multidimensional.

The question itself implies that there’s some linear “productivity” scale where you can put all your developers and see who’s more and who’s less productive.

Because of this fallacy still dominating management minds - we developers can for example hardly justify 2x, 3x salary raise, even though we realize sometimes that our impact and value is as much higher relative to some peers.

The brutal truth is that the contribution of a developer to any complex and big enough project is rarely proportional to any measurable metrics. And barely ever can be reliably connected to such a metrics.

developer A can just sit around few days and invent simple yet scalable decision with just 1 PR, while developer B would be writing thousands of lines of a good looking code.

But this great looking and even well documented code might kill the whole product at some point in near future, even because of some new requirements that nobody could expect (except that developer A, who wasnt promoted for bad “productivity” and quit).

Add to this the fact that often you can’t really know who was good and who was the bad developer - at the moment both had good arguments.

So, my point is that big software products are essentially non linear, dynamic and poorly predictable beasts.

I don’t have ready answers on “how to decide who’s gonna be promoted”.

But i feel the farther you go from trying to “engineer” and “deeplearn” this issue and closer to trusting intuitions of experienced dev leaders, to treating developers as people not machines, considering each project as a unique thing, with unique approaches — the closer you to the truth and to real improvements.

It's best to give up the idea of measuring software productivity entirely. Almost no one would seriously wonder how to measure artistic productivity and software engineering is a lot more like art than most people realize because it is fundamentally a creative activity.

Software is also too abstract to be measured the same way assembly line productivity is measured so there is no point in trying to impose a metric based productivity framework on top of software engineering. It's unlikely any such effort will be successful or will lead to any positive outcomes.

Can you define developer productivity? Before you can measure something you need to have a definition. Measuring PRs and commits is like measuring a writer’s productivity by how many sheets of paper they use.

A useful metric might be derived from business value added vs. cost of programming effort, but then how well can the organization measure business value? Or the cost of programmer effort?

I only glance at quantitative things like once a year come review time. Talking with people is a much better way to assess their performance. Usually people are hardest on themselves. Everyone is different, has different life circumstances and performs differently.

In general I think the premise of the question is wrong. It's taking a factory building widgets approach to a creative process. You'd have to have standardized measurable inputs like product requirements that can be worked by anyone on the team. Everyone on the team has to have the same level of experience with the technology used so any learning curve is the same for everyone. Proper standardized estimation needs to be done on every piece of work and then measure the time used to do the work. This is like CMM Level 5 type of utter nonsense, in my opinion.

Also, its more often the case that the inputs (requirements) shift and change as they get worked on which will not be reflected in any quantitative way.

There is no quantitative way to do it as any measure will be gamed or will not be representative of all the dimensions of what makes a successful project.

High performing teams that I've been on rely on developers to meet their own given estimates and effectively communicate. It is pretty obvious when someone is repeatedly dodging as they will be unable to convincingly explain their status to their own team members.

Basically, each team member holds each other accountable. This only works if they have strong, shared incentives.

Lines of code. Yes, it's easy to game. Don't use it for bonuses.

Yes, it varies by person and how they close their brackets, how they write comments, whether they use functional, or how they break up formulas. Compare a person to themselves.

Coding is an iterative process. Write stuff. Stick placeholders everywhere. Remove stuff and replace it with better stuff. LOC measures the stuff.

Does it measure architecture? The biggest argument for architecture is that it slows things down at first, but speeds it later. Great, measure LOC for both stages.

If you see people writing 1000 LOC/day early then slowing to 50, that's probably underengineering or overengineering. It should be more or less steady, slowing a bit when they get to the edges of the familiar code and need more R&D, then speeding up again.

Your devs spent some time documenting and refactoring? How's LOC rate after that? You added 3 newbies to the team? How does the LOC change for the seniors training them? Do the meetings discussing code increase or decrease LOC? How much does No Meeting Day or remote work change productivity? How effective are 4 day weeks or days when everyone goes home at 5?

It's not perfect, but it works for many of the things you'd like to measure, as long as you manage the downsides. If you're trying to reward productivity, it throws it all off.

I'm putting feelers out for a tool I'm building in the same vein: https://bldbtr.tech/.

What makes engineering metrics hard is that every team is different. We can talk about cycle time, deploy frequency, bug counts, but not every team can pull the same levers and those levers don't always yield the same results. So the teams and the manager need to know how to create metrics that matter and metrics that the team has the ability to affect.

You also need to take into account what is organizationally important. I've been in startups that valued deployments more than quality and enterprises in compliance heavy industries that valued quality over deployments. If your metrics don't align with your culture, it never feels right.

Then the biggest thing is to not compare teams. This is a game of "better than yesterday", not a competition. As a team, you want to find what equilibrium is and use your retro to find ways to iterate. As you have more discussions around performance, you'll find more things to measure, you just need to decide if it's worth it.

Either you end up counting things that are poor indicators of performance (points, lines of code, commits, etc), or it's completely subjective.

My company basically uses the subjective, using the opinions of 3-4 people. Although some break policy and do things like point counting anyways.

Doesn't seem to me that you can really measure productivity at all. How do you compare : (a) dev that writes tons of code and makes things over complex and not maintainable (b) dev that writes hardly any code, procrastinates, but their code is v high quality, simple, has no problems (c) dev that cranks out a lot of code but is horrible to work with and very good teammates quit (d) dev whose code output isn't high but that's cos they do tons of useful mentoring leading to other people's output being good

Most methods are poor. PRs and commits can easily be gamed and generally the best developers are the ones with succinct small changes that deliver high value.

Two ways I have found to measure a developer's productivity are...

1. Based on the feedback of their peers. It helps to know the team members and the team's dynamics though.

2. Based on the $$$ the team bring in. Depending on how the team operate this could be far from their control though, a lot of decisions about the product will be out of the developer's hands.

Look for track record of deliverables and adjust for human energy. If you see a good track record of things getting done, and then see a drop off, adjust for human energy. Humans can’t be at 100% all the time.

If you measure clinically, then you disrespect what a human being is. You can’t say ‘well you never missed a deadline and now you are, so you must suck or have given up’, this is a really bad thing to do. Empathy is key here.

In other words, you cannot put a number on this, I’m sorry. The eye test is crucial here versus the data.

If you're more interested in team performance metrics, read Accelerate. The primary metrics from Accelerate that I use are Deployment Frequency and Lead Time.

> As an experiment we’ve starred building an app that measures things like PRs and commits for each developer.

Ever heard of perverse incentives? This repeats errors made in the early 80s.

How often do goals get met, on time? If you can rely on estimates, and get good results, you've got a productive developer. The one time I managed someone as a programmer, I really, really lucked out, and he was great at it. We would discuss something that had to be done on Tuesday, I might get a question or two, otherwise nothing until he showed it to me done on Friday, and it was always up to snuff.

my general metric is, per task:

  ( complexity_points - bugs_reported ) / total_time

over time, the total productivity is the sum of each task.

( for complexity_points i use https://ciandt.com/us/en-us/complexitypoints )