Manufactured controversy and the “hockey stick”: A football analogy

Even if you're only vaguely aware of the climate change debate, then you will probably have heard of the "hockey stick". You know, this bad boy:

Source: Mann et al. (1999).

This famous depiction of global temperatures going back into time has generated a lot of controversy. It doesn't seem to matter much to sceptics that the initial hockey stick(s) — i.e. those produced by Mann, Bradley and Hughes (1998, 1999) — have since been replicated by multiple studies using different lines of evidence and computational procedures. No, we are invariably told that the hockey stick is a fraud and has been conclusively debunked.

The problem with these debates is that they are necessarily technical and involve concepts that are very unfamiliar to most people. Whenever I tried to explain things to my friends and family, I could see their eyes glazing over as soon as I mentioned the words "principal component analysis". So here is a sports analogy that captures the essence of what critics of the hockey stick got wrong:

Imagine that you are football (i.e. soccer) player who has discovered a great way of scoring goals when you are within the opposition's penalty area. In particular, you also happen to be a physics geek and have calibrated the optimal force and direction with which to strike the ball so that it lands in the back of the net every time. You demonstrate this impressive feat by first placing the ball at the front-right corner of the penalty box and then repeatedly slotting goal after goal with expert precision. Well done, FC Barcelona and Bayern Munich scouts are already fighting for your signature!

Hold on. Some rival player now arrives on the scene and says that you are talking complete nonsense. He begins by pointing out (quite reasonably) that penalty goals are usually taken from directly in front of the goals; not from the right-hand corner where you were showing off your abilities. He then "proves" how bad your system is by taking the ball to the centre of the box and, using your exact same kicking force and direction, proceeds to drill his shots wide of the goals. Worse, he then moves on to the left-corner of the box, where his shots are now so off-target that they are threatening to decapitate spectators in the stands...

By now, you should of course realise what is wrong with this story. The rival player has completely failed to recalculate his kicks based on the new position of ball! The underlying physics remain entirely intact. You would use exactly the same principles in determining what force and direction to use in scoring goals, even if those factors would vary according to where you were shooting from.

You forgot to recalculate, Wayne!

This, in simplified form and among other things, is the same mistake that the early critics of the hockey stick made. Except that now the relevant decision variables involve selection rules for the number of principal components (which help to identify the key patterns in your data) and where your data are centered along the time-series. For example, in a 2005 paper Steve McIntyre and Ross McKitrick (MM) argued that Mann et al. had used an unusual centering convention in their original study. When MM applied a more standard centering convention in an attempt to replicate Mann et al.'s key results, the hockey stick vanished. The problem with MM's argument, however, is that when you re-center the analysis you also have to re-determine the optimal number of principal components. (Just like the footballer in our example, you have to adjust your kicking calculations depending on where you are taking the penalty from.) MM failed to realise this and simply used the same number of principal components (i.e. two) as Mann et al. originally had. Correcting for this mistake and including the right number of principal components (i.e. five) required by the new centering convention means that — hey, presto! — the hockey stick magically reappears.

If you are interested in reading further, here are two excellent posts dealing with this manufactured controversy over the hockey stick. I should also say that Michael Mann's book gives a very accessible overview of whole palaver. He doesn't use a sporting analogy, though, so I still have the upper hand on that score.