16 Sep How to Use Performance Metrics Effectively
Purpose of Performance Metrics
How do you know a vendor or contractor is successful? How do you know a project or service is going well? What value is your project generating?
Performance metrics were originally created to answer these questions. We use metrics because numbers can be the clearest and simplest way to explain things. The purpose of performance metrics should be to create transparency. Transparency helps the client know what they can expect to receive or what they are currently receiving from a vendor. Transparency can also improve a vendor’s ability to explain potential risks and ways they can improve their performance to the client.
Unfortunately, many people have not realized this, and as a result it has diminished the value of performance metrics. Now, many organizations do not bother tracking performance very seriously. The misuse of performance metrics has created a system which minimizes the need to do high quality work. When there is a lack of transparency, buyers are unable to differentiate vendors by their level of performance. Instead, vendors are awarded based on their relationships with the buyer and the quality of their marketing material.
Nowadays, many companies spend more money and effort on their public image and popularity rather than their ability to deliver high quality services. This is evident as the performance in multiple industries has continued to struggle over the last 20 years. Yet, many of the vendors responsible for the resulting low performance continue to get more work. This is why buyers disregard performance metrics and focus more on relationships, marketing, and technical information.
This article will review five common misuses of performance metrics that make them less valuable in any industry.
- When the buyer/client tries to collect and be responsible for the metrics
- When the client tries to create minimum requirements/standards from metrics
- When the metrics are too complicated
- When too many metrics are being collected
- When metrics are collected without having a baseline plan
The Wrong Party is Responsible for the Metrics
Due to the importance of performance metrics, buyers started trying to collect metrics on their vendors and services. There are two issues with this. First, it takes too much effort and resources from the client to do this correctly. While at Arizona State University, my research group helped multiple State and Federal government organizations do this. In all cases, organizations stopped collecting performance metrics on their projects after a few years. The task was too difficult for their own employees, and they couldn’t justify the cost of hiring our research group to do it for them.
The second issue is that the only way for vendors to minimize risk and be accountable for their performance is if they are the ones tracking it. When the client collects performance metrics, the vendors are no longer accountable for their performance. Even worse, vendors no longer have the information they need to minimize risk and improve their performance.
The end result is that no one is collecting performance metrics, no one knows the performance of each vendor and project, and the vendors are not accountable for their performance or improvement.
The Client Enforces Minimum Standards
In their quest to try and lower prices and ensure high performance, buyers decided to accept a system of minimum standards, using metrics. The idea was if all the vendors met a minimum standard, then the client would always get the same or higher performance and they could pick the vendor with the lowest price (See Figure 1).
Figure 1: Minimum Standards Reduce Performance
The biggest issue with this system was that, from the vendor’s perspective, the client was telling them performance didn’t matter. Figure X shows that all vendors have different performance capability and different abilities to minimize risk. By creating a minimum standard, the highest performing vendor doesn’t have any advantage. The only thing that will help them get work is by having a lower price. Thus, even if it lowers the vendor’s performance the vendor will make adjustments to lower their price. In fact, all vendors are trying to lower their performance as close to the standard as they can get and lower their price—they must do this to stay competitive in this system. In doing this, many of them are eliminating their ability to minimize risk for the client. In the end, the cost of the project might be seem lower, but the risk on the project is high and it ends up costing more in the end.
To make matters worse, this system creates an adversarial relationship between the vendors the buyers. Minimum standards encourage the vendor to think in their own best interest and focus on reducing cost instead of focusing on increasing quality. In the end, the client gets less but pays more.
The other issue with minimum standards is that the metrics aren’t always reliable. This leads us into our next misuse of performance metrics that occurs.
The Metrics are Too Complicated
Many times, buyers understand the importance of metrics, however, the metrics are too technical and are not understood by the majority of people. Metrics are supposed to simplify and create transparency, but when metrics are too technical it creates complexity and confusion. The most common type of technical metrics are industrywide minimum standards and key performance indicators (KPI). KPIs are very popular. They are used like minimum standards—a tool to try and ensure a minimum performance of the vendor’s technical work. Unfortunately, there is little research showing that minimum standards and KPIs will ensure client satisfaction and high performance.
Although technically skilled craftspeople and technicians may understand the literal meaning of technical metrics, most of the time they do not know how these metrics affect overall performance. In many instances, technical metrics might not impact customer satisfaction, cost, or quality.
Tables 1 and 2 show metrics for a roofing system. Table 1 shows technical metrics (metrics you would see for minimum standards and KPIs) and Table 2, you have simple performance metrics. When looking at Table 1, you see a metric called tensile strength. This metrics is too complicated for three reasons:
- The definition of tensile strength is complicated. Tensile strength is the amount of stress or load a material can take under tension before it fractures or breaks. It is measured in terms of pound per square inch (PSI). In other words, if you have a tensile strength of 900 psi, then it means your material can take 900 pounds of force applied to one square inch of it, before it breaks.
- There are multiple ways to measure tensile strength, and even when there is a standard for measuring it, you cannot ensure the testing was performed exactly to the standard.
- No one knows what a good tensile strength is. Is a material that has a tensile strength of 900 psi going to prevent your roof from having leaks or getting damaged? What about a material with 4,000 psi? What should be the minimum acceptable level for this? The answer is that no one knows. The industry standard is just an average of various laboratory tests.
Table 1: Industry Standards
Table 2: Simple Performance Metrics
In contrast, when you look at Table 2, you find metrics that are easily understandable. Without thinking, these metrics help you determine if the roofing system will be a high-performing one. The way you know if performance metrics are being used correctly is if no one has to explain what they mean. Less explaining means more transparent metrics.
When metrics are too complicated there is still no transparency. When no one can understand the metrics, no one knows what the performance is, and no one can be held accountable for the performance of the project. Complexity ruins the value of metrics.
There Are Too Many Metrics
Complicated metrics can cause confusion, but the other mistake that goes hand in hand with it, is people making too many metrics. When you have less, there is no hiding anything. When you have a million metrics to look at, people don’t know what is most important, and information gets overlooked.
There usually are not many performance metrics that bring transparency and help identify how well a project is going. The simplest metrics are usually time, cost, customer satisfaction, and number of end deliverables. Thus, when you have too many metrics, the metrics begin to be overly technical and complex. Usually, simplifying metrics means minimizing the number of metrics as well.
There Is no Baseline
Depending on the situation, this mistake might not have as great an impact on the value of the performance metrics compared to the other mistakes mentioned. However, the only way to bring performance metrics into perspective is to compare them to previous similar instances. When looking at an athletes’ performance metrics, the best way to know if a player is a higher performer is by comparing their performance metrics with other players in the same position and sport. Knowing a quarterback threw 50 touchdown passes in a season is only helpful if we know that the majority of quarterbacks don’t ever throw 50 touchdown passes in one season and only three have ever achieved such a feat. Without a baseline, it allows people to have unrealistic expectations of what the performance metrics should be. This can also decrease the impact and importance of the metrics.
Conclusion
Performance metrics are key to creating transparency, improving performance, and saving money on any project in every industry. These days, too many companies are getting lost in the details of big data and industry standards. Buyers are neglecting the importance of tracking and comparing simple performance metrics.
If you want to know more about how to best use performance metrics in your company or you would like to see case studies, take a look at these resources:
- Free membership for latest tips and news: https://pbsrg.com/membership/
- For latest books, events, and licensed partners: https://pbsrg.com/resources/
- Latest BVA journal publications: www.cibw117.org/
- Annual Best Value Conference in January: https://bestvalueconference.ksm-inc.com/
- Latest presentations and videos: https://www.youtube.com/channel/UCxBi26nXLDTqG4ZRV6p0iiQ