Competitive Benchmarking Without Fooling Yourself

Pieces per day, or lines per day

In 1981, Xerox's logistics and distribution unit needed a better warehouse. Its productivity had been improving 3% to 5% a year, and industry price cuts meant that was no longer enough ¹. A staff member spent most of that year, half time, combing trade journals and talking to associations and consultants for the best-run distribution operation he could find. He picked L.L. Bean, the outdoor retailer and mail-order house ¹.

On the obvious measure, Xerox looked fine. Set against L.L. Bean, the most efficient warehouse Xerox was then planning came out ahead on pieces picked per worker per day, because Xerox orders often included several pieces. But L.L. Bean picked almost three times as many lines per worker per day, and a line, one trip to one bin, measured how far pickers walk. The Xerox authors called it the crucial measure of productivity ¹. Change the denominator and the story flips.

Competitive benchmarking is comparing your performance and practices with those of rivals or other well-run companies, to find gaps worth closing. The Xerox story contains the whole craft in miniature. The number told them where to look. The practice (stocking fast movers closest to the picking route, releasing orders through the day to shorten walks) told them what to do ¹.

The thesis of this page: a benchmark gap is a question, not a target. It is worth acting on only once you can explain it, which means the same denominator on both sides, a comparison set that includes the companies that failed, and a named practice behind the number that fits your own strategy.

Benchmarking closes gaps. It does not open leads.

What the people who popularized it warned

Benchmarking is often told as a Xerox story, and the earliest account is Xerox's own. In a 1987 Harvard Business Review article, Frances Gaither Tucker, Seymour Zivan and Robert Camp describe Xerox starting in 1979 by comparing unit production costs with Japanese copier makers, taking their machines apart, finding that US production costs were much higher, and adopting the lower Japanese costs as targets ¹. The term, they note, comes from land surveyors comparing elevations ¹.

Read the same article for its warnings, which are more useful than its origin story. Benchmarking against direct competitors, the authors say, poses problems: it may uncover practices unworthy of emulation; it may help you meet a rival's performance but is “unlikely to reveal practices for beating them”; and competitor information is hard to get ¹. Their preferred method was benchmarking against noncompetitors with the best functional practices, partly because people are more receptive to ideas from outside their own industry ¹. And they found that managers start by comparing costs but learn that understanding the practices behind the costs matters more ¹.

So competitive benchmarking, in the narrow sense of comparing yourself with rivals, came with its own caution label from the start. For the general definition, see benchmarking in the Lexicon.

Three ways a benchmark lies

Most benchmark errors fall into three families. Each has a cheap check.

The sample problem deserves the most attention because it is invisible. Jerker Denrell's 2005 article on selection bias shows how dramatic the winnowing can be: the number of US tire makers peaked at 274 in 1922, and by 1936 just 49 were left ². Anyone benchmarking tire companies in the 1930s was studying the survivors of a population more than five times larger.

Denrell's sharper point is about risky practices. If a practice makes results more extreme in both directions, its failures go out of business and disappear from the data, and the survivors make the practice look good. Looking only at existing companies, he shows, can make a practice that on average hurts performance appear to help it ². His fix is direct: extend benchmarking exercises to include less-than-successful firms ².

The cause problem has two parts in his article. Performance feeds on itself, so a company with a large share last year will probably have one this year even if it is now badly managed; and high performers can afford programs that struggling firms cannot, so a practice may be a result of success rather than its source ².

From gap to practice to fit

1. Name the decisionBenchmark to answer a question: should we redesign returns handling, can we afford a price cut, where is our cost disadvantage? A benchmark with no decision attached becomes a league table.
2. Define the measure in fullNumerator, denominator, period and what is included. Choose a denominator that counts the event that drives the work: lines, not pieces; orders, not revenue; visits, not customers.
3. Build the comparison set on purposeInclude direct rivals for context, noncompetitors that are best at the function, and companies that tried the practice and failed. Write down why each is in the set.
4. Normalize before comparingAdjust for scale, mix, geography and accounting choices. If an adjustment closes most of the gap, the gap was never about performance.
5. Explain the remaining gap with a practiceName the specific way the leader works differently. If nobody can name it, you do not yet have a finding.
6. Test the fitAsk what else the leader does that makes the practice work, and whether your strategy would support it. Then set a target.

The last step is where Michael Porter's argument bites. Advantage, he wrote, comes from the way activities fit and reinforce one another, and a competitor that imitates only some activities in a system “gains little,” and its performance can decline; he cites Continental Lite's attempt to imitate Southwest as a disastrous example ³. A practice copied without the activities that support it brings the cost of the practice without its payoff. The strategy vs. tactics page makes the same point about copying moves without the choice above them.

A worked example

The company, its peers and every number below are invented.

The first benchmark pointed at a $6 million building. The denominator check moved the problem from fulfillment to returns, and the fit check cut the remedy in half. That sequence, not the first number, is what a steering committee should see.

The case against benchmarking rivals at all

The strongest objection comes from Porter's 1996 article on what strategy is. He grouped benchmarking with total quality management and reengineering as tools of operational effectiveness, and warned of “competitive convergence”: “The more benchmarking companies do, the more they look alike” ³. As rivals copy one another's improvements, he argued, strategies converge and competition becomes “a series of races down identical paths that no one can win” ³.

He is right, and the same article says where benchmarking belongs. Porter did not reject operational improvement; he separated two agendas. “The operational agenda is the proper place for constant change, flexibility, and relentless efforts to achieve best practice,” while the strategic agenda is about a unique position, trade-offs and fit ³. The rule that follows is practical: benchmark the activities where there is no trade-off with your position, and never benchmark the choices that define it.

• Benchmark freely: invoice accuracy, warehouse picking, the cost of closing the books, IT uptime. Being worse than rivals there buys you nothing.
• Benchmark with care: service levels, assortment breadth, sales coverage. A gap may be a deliberate trade-off. Ask whether it is before closing it.
• Do not benchmark: the choices your strategy rests on. If your position depends on doing something rivals do not, a gap against them is the point.

Put Porter's warning next to the Xerox authors' own, that competitive benchmarking may help you meet rivals but is unlikely to show how to beat them ¹, and the conclusion is the pull line at the top of this page. Benchmarking is how you stop losing on things that should not decide the game. It is not how you win it.

Who chose the peer group?

At scale, the most consequential benchmarking decision is the comparison set, and it is often made by the people whose performance is being judged. Executive pay is the visible case. The SEC's rules for the compensation discussion of US listed companies list, among the examples of material information, whether the company benchmarked pay and, if so, the benchmark and its component companies ⁴. The choice of peers can drive the answer.

Inside a company, the same thing happens quietly. A unit picks peers it compares well against; a function picks a survey whose definitions flatter its cost. Three guards help.

• Fix the peer set before the data arrives, with written reasons for each company, and change it only on a schedule.
• Have someone outside the unit choose or review the set. Kahneman, Lovallo and Sibony argue that the people screening a recommendation for bias should be independent of the team that made it ⁵.
• Use benchmarks to re-anchor, not to copy. The same authors suggest competitive benchmarks as one way to give a team a fresh reference point for a number it may have anchored on, while warning against slavishly copying and pasting benchmarked competitors' practices ⁵.

Monday: rewrite one benchmark

Take the benchmark your team quotes most often. Write its numerator and denominator in full for your company and for each peer. Add one company that tried the leader's practice and failed. Then name, in one sentence, the practice that explains whatever gap remains.

If the gap survives all three steps, you have something worth funding. If it does not, you have saved the company from chasing a number. Xerox's lesson in 1981 was not that L.L. Bean was faster. It was that Xerox had been counting the wrong thing.

Questions

What is competitive benchmarking?

Competitive benchmarking compares your performance and practices with those of rivals, or of other companies that excel at a function, to find gaps worth closing. The useful output is not the gap itself but the specific practice that explains it, checked for whether it would work within your own strategy.

What are the main mistakes in benchmarking?

Comparing measures defined differently or built on a denominator that does not drive the work; studying only today's successful companies, so practices that also caused failures look like success factors; and mistaking a result of success for its cause. Setting the benchmark directly as a target, without explaining the gap, compounds all three.

Should you benchmark against competitors or other industries?

Both, for different reasons. Xerox's early benchmarking team preferred noncompetitors for functional practices, since information is easier to get and competitive benchmarking may help you match rivals but rarely beat them. Competitors are useful context for cost position. Include companies that tried a practice and failed, to avoid selection bias.

Does benchmarking make companies look alike?

It can. Michael Porter warned that the more companies benchmark, the more they converge. His answer was to keep best practice for operational activities where there is no trade-off with your position, and to protect the choices that make your strategy distinct from being benchmarked away.

Where to go next

• Competitive Analysis Template: A Rival Profile You Can Score
• Strategy vs. Tactics: The One-Question Test
• Benchmarking: Definition & Examples
• Halo Effect
• Operational Excellence