Why we should be wary of expert predictions for 2014

As ever the first day back at work coincides with a flood of forecasts about what will happen in the world in the year ahead. But predictions are often more interesting in retrospect than they are in their own time. For example, each year The Economist produces its one-off ‘The World in…’ publication which asks well-informed academics and writers to tackle an issue that relates to their own specialism. This year these relate to issues such as Scottish independence (it’s a ‘no’, by the way), the rise of African economies and a potential customer backlash against technology businesses and the rich geeks who own them. Interesting though it is to read all of this, The Economist is at least honest in publishing a list of its hits and misses, whereas most people appear to just pretend the misses never happened.

The workplace sector is particularly prone to fall foul of the distorted vision thing. As Simon Heath explored in his recent analysis,  there is a perpetual drive for firms in the office design and management trade to come up with some unified theory about how, where and when we will all be working at some point in the future. As Simon points out, these often remain a ‘2020 Vision’ in spite of the fact that this is only now six years away and many recent forecasts appear to suggest that how we work in 2020 won’t be that much different to how we work now.

There is often an underlying fallacy that distorts such predictions. It is that the people making them are primarily well-informed about just one subject and have a vested interest of one sort or another in focussing attention on it. When they make their forecasts, they work on the basis of the basis of The One Big Thing they do know about while assuming all other things are equal.

Yet not only do we know that it is absurd to claim that anything can happen in isolation, we also know from empirical research that the best forecasters are those people with a breadth of knowledge, rather than specialists. The writer Philip Tetlock in his 2006 book Expert Political Judgement examines the accuracy of a range of pundits, including when they are categorised along the lines of the Greek poet Archilocus’s aphorism that “the fox knows many things; the hedgehog one big thing.”

While the book offers a detailed and complex analysis of the nature and accuracy of punditry – including a critique of the media and governments for their propensity to seek out hedgehogs rather than foxes for comment and forecasts – its main conclusion is that, if you want a guide to the future, don’t rely on specialists. And even those with a broader range of knowledge are far from accurate compared to other ways of gauging the future.

As Tetlock writes: “Quantitative and qualitative methods converge on a common conclusion: foxes have better judgment than hedgehogs. Better judgment does not mean great judgment; none of them can hold a candle to formal statistical models. But foxes do avoid many of the big mistakes that drive down the probability scores of hedgehogs to approximate parity with dart-throwing chimps. And this accomplishment is rooted in foxes’ more balanced style of thinking about the world: a style of thought that elevates no thought above criticism. By contrast, hedgehogs dig themselves into intellectual holes. The deeper they dig, the harder it gets to climb out and see what is happening outside, and the more tempting it becomes to keep on doing what they know how to do. Hedgehogs are thus at continual risk of becoming prisoners of their preconceptions, trapped in self-reinforcing cycles in which their initial ideological disposition stimulates thoughts that further justify that inclination which, in turn, stimulates further supportive thoughts.”