As a consultant, I find it rare to spend fifteen minutes with a capable executive before she asks me to assess how they’re doing compared with other companies. It’s a fair question: how can you know if you’re best in class unless you know how well other organizations are doing? And if you’re not yet best in class, how can you know how to get there without data?
Benchmarking works best on truly objective outcome goals: revenue per headcount or per square foot of retail space. Gross margin. Hotel occupancy. Share of wallet.
Unfortunately, support organizations own few of these. Arguably, for some organizations, maintenance renewal rates are a clear outcome, although even then, this depends on many cross-functional groups.
No, support leaders tend to want to benchmark other, more operational metrics: first contact resolution (FCR), customer satisfaction (CSAT), website “did this help” ratings, contact avoidance, and the like. And here’s where cross-company comparisons can confuse as much as enlighten.
First, on a very practical level, everyone measures these things differently. You might think that there would be an industry-standard way of measuring FCR…but you’d be wrong. (I was brutally disabused of this notion during a—I am not making this up—two hour conversation among ten service desk managers comparing and contrasting the details of their FCR measurement techniques. These were among the longest, yet most enlightening, two hours of my life.)
CSAT, in particular, sounds like it should be straightforward to compare between companies, and yet it isn’t. Survey results are notoriously sensitive to all kinds of details: how the question is worded and what scale is used for the answer can both substantially change the results. More subtle forms of bias can creep in, too. In our business, the most common is non-response bias—in other words, the small percentage of people who answer our surveys aren’t necessarily representative of all the people we want to reach. Also, some companies exclude certain cases or customers from surveys, while others mix up transactional (post-case) surveys with relationship (periodic, overall experience) surveys. Comparing your “4.3 CSAT” against my “4.1 CSAT” really doesn’t necessarily teach me anything unless I really, really understand the details of how you measure CSAT. If I’m simply using data from an industry benchmarking database, I have absolutely no idea how that particular brand of sausage was made.
If you have a deep and trusted relationship with a peer company, and you really understand how their measurement techniques match to yours, you might learn something from benchmarking. Unfortunately, ethical access to the fine details of peer company operations is rare.
More strategically, measures reflect your philosophy of business. Zappos is well known for not measuring handle time, because they believe personal relationships in customer service fuel their corporate engine of growth. So, what point is there in measuring your handle time against Zappos’s?
That’s an extreme example, but different business decisions drive different metrics, and obscure insights that might be gained by comparison. I know some support leaders are fixated on time to first technical contact, others on contact rate, others on service levels, and still others on NPS. None of these is wrong, but they will affect the measures.
So, benchmarking against other companies is fraught with difficulties. Still, we need aspirational goals. What to do?
First, sometimes benchmarks can be good to cause a deep rethinking of process. For example, I worked once with a company that was very proud that they had engaged additional outsourced staff in order to move their time to publication for knowledgebase articles from 23 to 14 days. When they heard that some companies do it in minutes, that data point spurred useful conversation. (Clearly, this requires more than hiring some new reviewers.)
Secondly, the best benchmarks are self-benchmarks. When I get on the scale, I don’t compare myself with the average male my height; I’m interested in where I am relative to where I was the week before. Similarly, if I’ve improved my ability to capture, reuse, and improve knowledge, I’d like to see if my contact rate is going down, or my ability to close cases faster with reused knowledge is increasing. Any metric I report on is a candidate for self-benchmarking, especially if I’ve been taking actions to change something. Not every change is meaningful, but if you look at your own trends over time, in measures that are strategic to your business, you’ll have the best possible, least-gameable, most informative metrics of all.
Many of us “self-benchmarked” on doorways growing up.
A little late to the party David, but still a great post! I wrote about my benchmarking discussions as a consultant a while back: http://www.haimtoeg.com/?p=202 and reached similar conclusions.
In order to move the discussion forward I’d sometimes invent a ridiculously good number for a well known industry entity (“Oracle closes 95% of all cases within the first hour”) and encourage the client to contemplate how they can reach similar performance. Then I’d ask how knowing that number changed their positions as presented in the discussion. It never did, but in hindsight it may have acted as a motivator to explore various options they normally would not on their own.
Haim – Thanks for the link to your blog and (from there) to Bob’s. I feel better that there are at least several voices crying in the wilderness on this topic, although I’m a little sheepish that I hadn’t read your blogs first so I could credit them properly. (Come to think of it, I must have read yours, which makes it even worse I didn’t cite you.)
No need to apologize David – I wrote that almost two years ago and really can’t expect anybody to remember, I was just shamelessly plugging my newly revived blog. Over the years there’s been a number of discussions I have seen about benchmarking, so IMO you (and I, and Bob if he’s still there) are far from being isolated voices in the wilderness.