Table of Contents
What are good metrics?
Eli Goldratt, the creator of the Theory of Constraints, once wrote “tell me how you measure me, and I will tell you how I will behave.” This simple statement succinctly explains the driver behind performance—or a lack of performance—in any organization. When you as a business leader use a concrete measurement to assess your team, you are sending a message. “Make this number move in the right direction.” You ultimately leave how the team moves the number up to them. You may try to institute processes to guide their behavior, but ultimately what you measure a team buy will drive what they do.
Why do metrics matter?
As stated earlier, what you as a business leader measure is what will drive the behavior of your team. If you hold a team responsible for improving a metric, they will do whatever they can to improve the metric. Holding a team responsible for improving a metric is one of the simplest ways to change their behavior. However, you must consider the undesirable effects of this behavior change. Did you identify good metrics that will encourage the behavior you want, or did you choose a metric that incentivizes a behavior that actually hurts your company? Either way, as Goldratt tells us, you will cause employees to behave in a way that improves the metric.
Examples of Bad Metrics
As an example of a bad metric, let’s consider the behavior of Wells Fargo employees. In 2016, a massive scandal was uncovered: Wells Fargo staff secretly opened potentially millions of dual checking accounts on behalf of customers who neither wanted nor needed them. Reportedly, employees took these illegal actions as a result of their sales organization having unrealistic and otherwise unattainable metrics.
As a mid-level employee in this organization, what are you to do if your boss is breathing down your neck to open more accounts? Well, you’d find ways to open more accounts. You want to keep your job, after all, and if opening accounts is the most important metric, you’re going to bypass the rules and do exactly what your boss is demanding. Open accounts at all costs. Improving this metric ultimately hurt the organization. Wells Fargo faced intense public scrutiny, fired thousands of complicit employees, underwent federal investigation, and remunerated all customers victimized by the scheme.
A more theoretical example of a bad metric comes from debates about the existential threat of artificial general intelligence (AGI). Unlike the typical artificial intelligence (AI) applications we see today that are very good at a small subset of things, AGI is the term used for a theoretically sentient artificial mind. Philosophers argue that even a seemingly innocuous goal or metric given to an AGI could result in the termination of all other life in the universe. Consider an AGI whose goal is to create as many toothpicks as possible. As the metric of “number of toothpicks created” increases, the AGI will work to obtain more resources. Eventually, perhaps, the AGI consumes entire planets or galaxies, all for the sake of producing more toothpicks.
Outcomes of Bad Metrics
In either of the above bad-metric examples, you can likely see how, in theory, the metrics are good. Wells Fargo needs more accounts to drive an increase in revenue. However, without consideration for regulatory or ethical constraints the metric led to illegal activity that hurt millions of people. The AGI may believe toothpicks are the most important creation in the universe. However, if it eliminates all life on earth to produce more toothpicks, no one will benefit from its beautiful mouth-cleaning creations.
In more realistic scenarios, such as your business, your team looks to you for an understanding of what they should value. If you value something without appropriate consideration for its impact on other parts of the business, you may get the outcome you asked for but not the one you really wanted.
Outcomes of Good Metrics
Let’s imagine if Wells Fargo changed their incentives. What Wells Fargo really wanted was not more accounts. They likely believed more accounts would increase revenue. At a glance, this makes sense. If the average value of an account is $X, and you have Y accounts, Y * 2 = $X * 2. Therefore, double the number of accounts and you’ll double your revenue!
But, the sales organization doesn’t necessarily care about revenue. If an employee is told to double their number of accounts, well, they may not care how much cash is held in those accounts. In this instance, extra accounts were opened but—since they were opened in secret without customer authorization—they did not hold more cash. Certainly, Wells Fargo made money off of this scheme through fees and penalties, but it ultimately cost them more money to push this bad metric.
Alternatively, what if they focused on increasing the average amount of cash held in an account? Or the overall cash managed under a sales team member? Either of these metrics would incentivize completely different behavior. They might consolidate accounts—or at a minimum, not secretly open unneeded accounts—and regularly engage higher-net-worth customers to entice them to put more money into existing accounts. They might open up more investment vehicles that make sense for existing customers. Alternatively, they could push for reducing the friction in investing more money in Wells Fargo managed accounts.
These good metrics incentivize the right thing for Wells Fargo: making more money for the company.
Ways to identify good metrics
A good metric should get you as close to measuring your ultimate goal as possible. In the Wells Fargo example, we analyzed how bank leadership likely decided to incentivize increasing their number of accounts. However, you’ll see there’s a big assumption we’ve identified that must be true for the metric to work. The assumption is this: if we double the number of accounts held by a person, the average value of each account will not change. Double the accounts, double the cash you hold. In practice, the sales team did not achieve this. They instead met the metric: “open more accounts and you will get a bonus.”
We can take this same assumption-based approach to identify good metrics. If moving a metric requires an assumption to be true to meet your goal, then either:
- your assumption absolutely must be true, even after the measurement increases; or
- your metric must causally attain your goal as the measurement increases.
If neither of those scenarios are true—if you have an invalid assumption and your metric doesn’t have a causal relationship with your goal—you have a bad metric on your hands.
Identify your own goal-based metrics
Are you struggling with identifying appropriate metrics for your goal? If you would like to talk through possibilities, please contact us to start a discussion.