Archive for February, 2012

How to Measure Productivity

The Productive Dishwasher

I hold a weekly meeting to discuss all things quality and iterative development. This week, we tried to define Productivity. We came to the conclusion that the way we want to measure productivity in the software world is different than in the manufacturing world. People aren’t machines or even assembly lines. Simply being “up and running” doesn’t mean we are being productive. One common example is the manager walking around to see if people are surfing the Internet instead of being “productive.” Someone shared an anecdote of a dishwasher that finished early, but rather than incur the wrath of the boss for sitting around doing nothing decided to dirty up the dishes and wash them again. That’s not the kind of behavior we want to encourage.

We discussed why using # of Story Points/Ideal Days (i.e. Velocity) to measure productivity is a bad idea:

  1. It’s not fungible , i.e. points aren’t transferrable between teams or across projects. So if team A has a 20 point velocity and team B has a 40 point velocity it doesn’t mean B is 2x productive as A. In fact, it doesn’t even mean they are *more* productive.
  2. Once you start rating a team on productivity based on points, guess what? The team will start showing a dramatic increase in Velocity!

So what’s a better measure? You could simple count the number of actual stories you output over time. That could work, but it’s also susceptible to gaming. (Although, something that encourages lowering story size is probably a good feedback loop to employ!) Cycle time is also a metric you might use. This measures the time from when a story gets into the iteration to when it is marked Done. If your average cycle time is longer than a normal iteration, well that’s an indication of low productivity!

None of these are perfect. That’s why it’s also a bad idea to rely on one metric. Use a combination of tools to focus on areas of concern and improve them. If velocity is going up, but cycle time is staying the same, there may be some problems with story estimation – maybe one voice has started to dominate the process. If output is up, but velocity is down, there may be a problem with prioritization (we’re only working on easy stories).

We also discussed Throughput as a concept, being the result of Sales – Cost of Raw Materials and then Productivity = Throughput/Operating Costs. This comes from Throughput Accounting and it isn’t often used in a software context. I think it’s fine to couch Productivity in terms of $; it’s certainly fungible. However, again, software generally isn’t sold in units that can be created many times a day! You’ll need to figure out productivity during the project in some fashion before it even gets put into active use. Still, this is all the more reason to have small features that can be put into use as soon as possible.


1 Comment

Contract and Integration Service Testing in a Componentized Enterprise Model

Contract Testing

(Presented at Agile Atlanta on May 1st, 2012 – Prezi)

I’ve been working a lot over the past six months on creating an automation framework for our service development teams. In our relatively large organization, we have have teams devoted to different horizontal strata of software development. So, for good or ill, we have database teams, mainframe/API teams, service layer teams, and UI teams. If you’re at that service layer, and that’s all you do, you should be taking a different approach to testing than if you had to do all of the testing for vertical slices of product functionality. With that in mind, here’s a proposal for a presentation on the strategies and the automation framework we’ve come up with. I welcome any feedback or thoughts!


How do we ensure appropriate testing and feedback for large Agile Enterprise projects where the value delivered may not be end-user facing features? Such is the case for many enterprises working to establish a Service Oriented Architecture. A lot of knowledge exists for agile teams delivering customer-facing products. Much of that work focuses on acceptance tests being written in clear business language that is understandable by the whole team and stakeholders. This approach becomes more complicated for component delivery teams developing Shared Services that are often seen in large Enterprise development organizations.

Services can be tested in a number of ways and the approach that’s best can vary greatly based on the context of the application and environment. The presentation discusses common patterns of testing services and focuses on an acceptance test solution appropriate for teams working on the shared service layer within a componentized Enterprise model. The solution is based on a “universal” fixture that works for any type of object-based service and is accessed via table-based acceptance tests in FitNesse.

The talk will be of a standard presentation format with slides and discussion. The first part of the presentation provides background on testing in an Agile Enterprise model and the various approaches to Service testing. (15) The second part of the presentation discusses the creation of specific framework to aid in service contract verification that is helpful for specialized teams working in a componentized Enterprise development model. (15) The last part gives examples on basic usage and on how the framework is used in a real-world environment. (30)

Learning outcomes
Knowledge of how Agile in the Enterprise affects our approaches to testing and feedback
Understanding of various methods of service testing and how to choose what’s right for your team
Details and examples of how to employ a framework for testing services appropriate for the componentized Enterprise model.

, , ,

Leave a comment