The problem and the solution
Who was the more dominant golfer, Tiger Woods or Bobby Jones? Was Jack Nicklaus better than Walter Hagen? How about Ben Hogan vs. Phil Mickelson?
And while we’re asking provocative questions, were any of those fellows superior to Annika Sorenstam, Babe Didrikson Zaharias or Inbee Park?
For much of golf history, those questions have been beyond the pale of objective analysis, placed there by the changing circumstances within which athletes of different generations competed. Those changing circumstances are too numerous and varied to mention, but at minimum they include such major variants as: changes in equipment, training regimens, travel, financial incentives, population bases, playing opportunities and weather.
Is there any way to normalize all of those variants in such a way to construct a simple, objective and accurate methodology to compare and rate the relative level of athletic performance across barriers of time and gender? Yes, there is … and it’s deceptively simple.
The method involves standard deviation, a statistical tool designed to measure levels of exceptionality among fields of data. If you are familiar with the concept of the bell curve—the farther toward the edge of the bell a data point lies, the more exceptional it is — you are at least casually familiar with standard deviation.
Keeping that in mind, measuring exceptionality in sports performances across time or gender is no more complicated than abandoning our usual methods of measuring individual or team results – by “points”, “wins” or “strokes,” – and in its place asking another question: How dominant was the performance in question?
Why standard deviation works?
When Bobby Jones famously won his golfing grand slam in 1930, he played on courses that were far shorter than those used in championship competition today. The clubs and balls he used are several generations removed from equipment that a modern professional would consider acceptable. Jones’ conditioning regimen, too, would be considered primitive by current standards. And his mode of travel to those tournaments – by train or steamship – would been viewed as needlessly taxing. Superficially, those realities make it sound impossible to evaluate Jones’ 1930 season relative to that of a modern player. Yet it is far from impossible; in fact it’s relatively simple. Here’s how.
Unlike scores, which when considered across time are influenced by all the naturally changing circumstances of play, standard deviation is immune to the influence of those factors. Although Jones’ equipment would be considered hopelessly antiquated by the standards of today, it was not viewed that way when Jones was in his prime. In fact it was typical of the clubs and balls used by the best of his contemporaries. The courses Jones played may be outdated today, but they were viewed as fair and equal tests then … of Jones and all of his fellow competitors. Those men had access to the same training equipment and methods, and they traveled tournament-to-tournament in the same trains and ships.
In short, while conditions certainly have changed dramatically over time, at championship levels, at least, athletes have always competed on essentially equal terms. That means while Jones’ four-round total of 287 and two-stroke margin of victory over Macdonald Smith and Horton Smith at the 1930 U.S. Open at Interlachen may not sound comparable to Brooks Koepka’s four-round total of 272 and four-stroke victory over Hideka Matsuyama and Brian Harmon at Erin Hills in 2017, we actually can construct such a comparison by looking at the exceptionalities of their relative performances.
When we do, this is what we find. Jones performed that week at a rate 2.47 standard deviations superior to the average of all players completing four rounds of play under the same conditions on the same course. Koepka performed at a rate that was 2.24 standard deviations better than his fellow competitors. Normalized for changing conditions, equipment and circumstances, 1930 U.S. Open champion Bobby Jones defeated 2017 U.S. open champion Brooks Koepka by about one-quarter of a standard deviation…that’s a fraction less than two strokes.
How about other sports?
Standard deviation can be used to normalize performance exceptionalities across time for any team or individual sport that uses a standard scoring method. In addition to medal play golf, that includes all the major American team sports – baseball, football, basketball and hockey. Were the 1927 New York Yankees actually the best team of all time as popular sentiment dictates? Would Vince Lombardi’s Packers have handled Bill Walsh’s 49ers, Pittsburgh’s Steel Curtain teams of the 1970s, or the modern Tom Brady-led Patriots? Were Michael Jordan’s Bulls teams more dominant than Wilt Chamberlain’s Laker clubs or the Larry Bird Celtics? See the links on this page to answer all those questions and more.