Manage to Test

I hold a weekly meeting to discuss all things quality and iterative development. This week, we tried to define Productivity. We came to the conclusion that the way we want to measure productivity in the software world is different than in the manufacturing world. People aren’t machines or even assembly lines. Simply being “up and running” doesn’t mean we are being productive. One common example is the manager walking around to see if people are surfing the Internet instead of being “productive.” Someone shared an anecdote of a dishwasher that finished early, but rather than incur the wrath of the boss for sitting around doing nothing decided to dirty up the dishes and wash them again. That’s not the kind of behavior we want to encourage.

We discussed why using # of Story Points/Ideal Days (i.e. Velocity) to measure productivity is a bad idea:

1. It’s not fungible , i.e. points aren’t transferrable between teams or across projects. So if team A has a 20 point velocity and team B has a 40 point velocity it doesn’t mean B is 2x productive as A. In fact, it doesn’t even mean they are *more* productive.
2. Once you start rating a team on productivity based on points, guess what? The team will start showing a dramatic increase in Velocity!

So what’s a better measure? You could simple count the number of actual stories you output over time. That could work, but it’s also susceptible to gaming. (Although, something that encourages lowering story size is probably a good feedback loop to employ!) Cycle time is also a metric you might use. This measures the time from when a story gets into the iteration to when it is marked Done. If your average cycle time is longer than a normal iteration, well that’s an indication of low productivity!

None of these are perfect. That’s why it’s also a bad idea to rely on one metric. Use a combination of tools to focus on areas of concern and improve them. If velocity is going up, but cycle time is staying the same, there may be some problems with story estimation – maybe one voice has started to dominate the process. If output is up, but velocity is down, there may be a problem with prioritization (we’re only working on easy stories).

We also discussed Throughput as a concept, being the result of Sales – Cost of Raw Materials and then Productivity = Throughput/Operating Costs. This comes from Throughput Accounting and it isn’t often used in a software context. I think it’s fine to couch Productivity in terms of $; it’s certainly fungible. However, again, software generally isn’t sold in units that can be created many times a day! You’ll need to figure out productivity during the project in some fashion before it even gets put into active use. Still, this is all the more reason to have small features that can be put into use as soon as possible.