Many people really believe in metrics. In theory they tell you about the heath of a project in a scientific, non-subjective way – whether it is running to time, which features are likely to miss the cut, whether there are serious problems which need action. Agile methods are as good as any for generating clever new metrics – burn-up, burn-down, cumulative flow diagrams, etc.
Unfortunately my experience is that these metrics often fail to achieve what they intend. People spend a lot of time creating spreadsheets which draw impressive charts, but whenever these are presented at a planning meeting they are met with a sea of glazed-over eyes. The numbers don’t tell anybody anything.
I’ve often wondered why this is. I find that I can subjectively judge progress on a project quite accurately, so why is it so difficult to quantify objectively? Indeed I once got badly tripped up in a job interview over this very question.
Going back to first principles, we should perhaps define ‘progress on a project’. From my perspective it’s a question of how much you have done versus how much you still have to do to fulfil the vision, and how that looks in terms of the available time and effort. In a traditional waterfall project the target is quite clear, to deliver a fixed set of requirements by a certain date. Unfortunately you never know what you’ve actually achieved, nothing is built, tested and integrated, all you often have are a load of obscure tasks to which inane ‘percentage complete’ guesses are attached. In an agile project the situation is reversed – you know exactly what you have achieved, you have working and tested software in the bank. What you don’t understand so clearly is what you still have to do, as by definition the scope of the project is flexible.
In theory the agile answer to this conundrum is release planning – you estimate items on the backlog to arrive at an achievable target list of features to deliver in a given number of iterations, and thus obtain the information to create the classic burn-down chart. The problem is that given the level of story information available at this stage effort estimation is very difficult, so it is hard to arrive at a reasonable starting ‘height’ for the burn-down.
However when I’m asked to judge progress subjectively I can do it because, as the person in charge of the backlog, I know in my own mind what we still have to do, what the priorities are and what feature trade-offs I will be prepared to make to hit the target date. Now clearly, many heads are better than one and it would be preferred if the whole team had this appreciation. Which suggests to me that the aim of release planning should not be to generate metric targets, but to enable everybody to understand what lies ahead and what the finished product needs to look like. It seems like a good illustration of process not being a substitute for understanding.
Posted on March 17, 2011
0