Takeaway: Accurate metrics are essential for assessing performance and making informed decisions, but companies often rely on flawed information that paints a misleading picture. Alan Norton explains the most efficient strategies for obtaining valid, useful metrics.
The one constant in my career has been the collection and distribution of company information. I didn’t know the cost of the metrics I was producing. Those costs were never measured. But I did know my labor costs and the costs of the supplies I was purchasing. Multiply that by each department in the company where I worked and I knew that the numbers had to be getting real big, real fast. I also knew that there were opportunities there for big cost savings.
It is not difficult to make a case for more efficient metrics. Significant cost savings can be realized by reviewing and improving how and what metrics your company produces. Metrics are expensive. Add up the true total costs and they can be quite considerable:
•Printers
•Paper
•Printing supplies
•Labor
•Development
•Production
•Distribution
•Maintenance
And the costs don’t end at document creation. Metrics are shared multiple times, either electronically or physically. The security of company sensitive metrics is an additional responsibility and cost. The metrics must be managed, stored, and retrieved –all activities more costly than you might guess.
I won’t try to argue that metrics are in any way fun to discuss. However, their importance to the health of a company is undeniable. Get the metrics wrong and a company’s full potential will not be realized. I have seen enough to know that the metrics used by companies both large and small can be improved. Here are 10 ways to do just that.
Note: This article is also available as a PDF download.
1: Measure performance at aggregate levels and exceptions at lower levels
I have produced a lot of charts in my time. In retrospect, I realize that I produced too many charts. Many of those charts provided no value at all — they showed performance on schedule or close to on schedule for a number of subassemblies. Printing these charts might have given managers a reassuring sign that all was well, but little else. The better approach is to use exception reporting and set upper and lower limits. Sure, go ahead and measure performance at higher levels. But report only exceptional performance, good or bad, at the more granular levels.
2: Eliminate metrics with little or no value
Know the cost of each metric. Are all your metrics worth reporting? A cost benefit analysis can help identify reports that are costing more than they are worth. A best guess estimate of potential savings must be included in the analysis, so it is not as simple a process as it might first appear.
It can be difficult to discard those old, comfortable shoes when you’re rummaging through your closet. But if they’re no longer useful, they should go. The same applies to those charts and reports that managers have become comfortable seeing. Nevertheless, if they are of little or no value, they should go.
3: Avoid tying incentives to lower level metrics
Meeting goals at the department level can be detrimental to the company’s overall performance. For example, a help desk manager might be paid a bonus for increasing completed tickets at tier one. But this is counterproductive if the overall costs rise due to the increased workload at tier two and lower. Bonuses and other incentives should be tied to the corporate bottom line and not departmental performance.
4: Avoid metrics that are inaccurate, misleading, or ambiguous
Producing metrics that are inaccurate or ambiguous can be costly to correct if they lead to bad decision making. There are a number of reasons why metrics can be inaccurate, misleading, or ambiguous:
•Invalid assumptions
•Errors in data collection
•Incomplete data
•Outdated data
•Questionable/unverified data
•Difficult to measure data
•Subjective (not objective) data
•Complex and multifaceted data
Some metrics are difficult if not impossible to measure. Managers are nevertheless tasked with boiling down metrics like software development into one or two charts. Similarly, other metrics, like employee happiness, just don’t translate well into numbers and should be avoided. I’m not suggesting that measuring this type of information should be avoided altogether, just not presented as numbers in chart format.
5: Avoid skewing the data
One simple example of data skewing is reporting costs over a long period of time. 2010 dollars are not the same as 2005 dollars. If you are not compensating for inflation, you are delivering misleading information. Obtaining too small a sample or a non-random sample are other common ways that data can be skewed.
6: Standardize and consolidate
Reduce the amount of information generated by eliminating redundancies and standardizing where possible. Financial reports, like manpower metrics, can usually be centralized and standardized for all departments. There are those unique metrics that can’t be standardized for all departments, but many of these can be standardized across divisions.
7: Use unbiased personnel or outsource connect
Being a reporting guy my entire career, it isn’t surprising that I ran across a manager or two who wanted to game the system. There are countless ways to lie with numbers or, at a minimum, hide poor performance by the appropriate (or should I say inappropriate) manipulation of the data. One example I saw was to use fiscal YTD values instead of a 12-month moving window. The obvious problem with this is that when a new fiscal YTD starts you have no previous data points to show a trend. And that is exactly the goal. It’s a great way to hide bad numbers. I reported directly to the manager asking for the fiscal YTD format, so I was in no position to question his motives. The inherent problem created with this type of working relationship is another reason why metrics production should be centralized and reported directly to upper management or outsourced.
8: Automate whenever possible
Gathering data by hand and entering numbers into a charting program is a labor intensive and costly endeavor. Since charts are often generated weekly or even daily, one or more people can spend most of their time preparing the charts. If the data is available online, the charts should be automated. If the data is not available online and must be collected by hand, consider developing systems that can collect the data at a reasonable cost.
9: Consider going paperless
If your company hasn’t already done so, going paperless can save a lot of money. However, paper should still be an option and care should be taken before converting a report to online only. If it takes more time to look up a metric online, the additional labor costs can far outweigh the savings.
10: Be careful what you measure
All too often, overly simplistic metrics focus on only part of the whole picture. A metric like “IT spending as a percentage of revenue” treats IT services as a cost. But such incomplete information can lead to poor decision making. The challenge for IT managers is to develop metrics that show the value added by IT and get them under the CEO’s nose.
The very act of measurement determines where managers will place time and resources — often at the expense of other unmeasured metrics or other departments’ metrics. Measure too little and important performance metrics can be missed. Measure too much and the focus can be lost from the mission-critical metrics. The costs of measuring the metrics can then quickly exceed the benefits. The metrics package should be carefully selected to balance these tradeoffs.
The bottom line
Metrics are like laws. They are born but never seem to die. An annual review of the company’s metrics package provides a good opportunity to add new metrics, update out-of-date metrics, and discontinue those metrics that are no longer needed. Select your metrics package carefully. What you measure is what you get.
For further information, check out the whitepaper Efficient Metrics: Be Careful What You Measure.
http://www.techrepublic.com/blog/10things/10-efficient-ways-to-produce-useful-metrics
No comments:
Post a Comment