Resource Center

Metrics management and Project Management Author: Chris Vandersluis © 2003 Chris Vandersluis

Project management is rooted in measuring. We make a plan based on our estimation of what effort we expect is required to accomplish a particular result. It doesn't matter much what currency you've made your plan in - be it time or labour hours or cost or units complete or whatever else. Regardless of how you made the plan, once the project starts, all analytical work is about measuring how you've done against the plan and adjusting your future work to complete the project in the shortest possible time.

It's no wonder perhaps, that project management teams are turning more and more to measurement systems.

We've always done measurements and comparisons against baseline expectations but the whole metrics movement has taken this thinking further down the road. Older thinking in the project management world would have been centered around the variance report notion. The earned value era of US defense fame (Can you spell "7000.2 Standards"?) would end up with reports showing highly detailed results, sometimes in dollars, sometimes in hours, sometimes in duration. While proponents of these systems are still hugely in favor of them, the biggest drawback to widespread acceptance of this kind of analysis was the large learning curve required to get to a basic understanding of the results. It's true that the earned value formats have since been simplified but when you look at metrics from a scorecarding perspective, life seems infinitely simpler.

One of the fallouts of the variance analysis period was the emergence of 'Traffic Light" reports. These reports would give a red, green or yellow symbol like a traffic light to certain variance criteria. Have the schedule within 5 days of the baseline and get a green light. Have it between 6 and 10 days, get a yellow light, be later than 10 days and get a red light next to your project in a one-line-per-project report for management.

The symbology was so simple that executives found it hard to resists. Immediately, the over-stimulated managers could synthesize a highly complex situation into a short list of only those projects with a red symbol thereby focusing executive attention on those projects which were too variant. Additional details could always be asked of the project managers of course and the elusive executive one-page report could now be delivered. Similar reports showing variance for actual costs vs. budget, for completed units vs. planned or for any other measurable baseline could be implemented in similar fashion.

The problem with the simple report is that it showed which projects had already gone off the wire. Also, the report format had become so popular, it was now being asked for in other lines of business. Simple benchmarking can be applied anywhere and the notion of Scorecarding was born. Scorecarding can be imagined as the variance report without the analytics. Management sets up a series of goals much in the same manner as a baseline project is stored. Once we know the goals, we can start measuring the success or failure of reaching those goals on a regular basis. Results can be aggregated by category and the summary can be again a simple indicator of whether the overall category is in a green, yellow or red situation. Goals can be the simplest of measures from a "met target yes/no" type to a much more complex formula type. Either way, the variance is what is sought, and the resulting report is designed to be so simple that mis-interpretation is basically impossible.

If you're keeping up on such developments you may have also come across the notion of "balanced" scorecarding. We could talk about the balancing concept for hours but it's sufficient to say that the idea here is to create budget and actual variances in 4 separate sections which should each have a certain weighting. The theory goes that it's not enough to be super-strong in one area of the organization if the rest of the corporate structure is weak.

Well, it's all fascinating but what does this have to do with us down here in the lowly project management role? Aside from the obvious, that some metrics should be project metrics, the answer seems to be due to market demand. The pressure from the executive suite in so many organizations to have a simple measurement reporting system is so severe that the project management group finds itself having to adjust to reply to it.

Project Management systems are ideally suited to feed proper results to a scorecarding system. Like me, you've no doubt seen lately a number of vendors purporting to have scorecarding functionality. This has almost become a checkbox for IT personnel trying to determine which project management products should be evaluated or make the short list. Vendors promises have to be taken with a grain of salt of course, and in this case perhaps with a case of salt on the side.

For a project management system to promise scorecarding results is not a huge technical challenge. After all, as I've been saying, a project system is by its very nature a scorecarding system. No, if the goal is to implement proper scorecarding across the enterprise, then project management would be better as a supplier of data to such a system rather than the interface. The data is, after all, perfectly formatted for such a use.

The caution that almost every scorecarding implementer will give you will be blindingly obvious the second time you implement a scorecarding system. It's possible to measure ten thousand items and to have a score for each. It's possible to roll-up or summarize those scores by whatever weighting you invent, the challenge is determining what to measure. It's an easy pitfall to drop into. The implementer starts doing interviews to find out what the organization would like to measure. The executives have one perspective, the middle-management another, each supervisor has their own list and the resulting list becomes a phenomenal list of measurements to manage each month. Some of the measures will be objective data but most will be subjective judgment calls and the resulting effort can bring an organization to its knees. It's far to easy to spend so much time measuring performance that you leave no time to perform at all.

If scoring performance is in the cards for you, then start by going slow. Excel can be a great scorecard reporter for first-timers and the integration with your project management data can be tied together effectively by hand with a minimal amount of effort. If the exercise proves fruitful, you can expand to enterprise scorecarding by using an experienced expert to fine-tune what you should measure and how.

For more information on this article or any aspect of project management software or systems contact the author, Chris Vandersluis or HMS Software at info@hmssoftware.ca or by phone at 514-695-8122.