The crisis in economic forecasting

First they failed to predict the great financial crisis. Then they were wrong about the short-term impact of the Brexit vote. Do economists need to go back to school?

John Kenneth Galbraith: a dim view of economic forecasting

2011 Gamma-Rapho

First they failed to predict the great financial crisis. Then they were wrong about the short-term impact of the Brexit vote. Do economists need to go back to school? Simon Wilson reports.

What's happened?

"The only function of economic forecasting is to make astrology look respectable." Canadian economist John Kenneth Galbraith's pithy assessment of his own profession has rarely seemed more apposite, according to the dismal science's growing chorus of detractors.

Advertisement - Article continues below

Last month, no less a figure than Bank of England chief economist Andy Haldane issued an unusual mea culpa, admitting that forecasting failures particularly the failure to predict the financial crash of 2008, and mistaken warnings of immediate catastrophic consequences if the UK voted to leave the European Union meant the economics profession was "to some degree in crisis".

Haldane characterised the failure to anticipate the 2008 crash as a "Michael Fish moment" for economics a reference to the 1980s TV weatherman and his infamous assurance in 1987 that southern England would not be hit by a hurricane, just hours before the hurricane struck.

So economics is in crisis?

As a general caveat, it's important to remember that economics is far more than macroeconomics, forecasting and finance. Macroeconomics perhaps the most public face of economics is the study of how the economy behaves in aggregate; it involves analysis, theorising and prediction about economy-wide phenomena such as inflation, price levels, rates of growth, national income, gross domestic product (GDP) and changes in unemployment.

Advertisement - Article continues below
Advertisement - Article continues below

However, says Oxford professor Simon Wren-Lewis on his blog, if you look at a university economics department, then typically less than a fifth of the staff are macroeconomists, and in some departments there might be just one. Instead, most economists, such as those working on labour economics, experimental economics, behavioural economics, and so on, "would not have felt their sub-discipline was remotely challenged" by the 2008 crisis.

But macro is in the dock?

It's certainly in the spotlight. Economic output represents the aggregated activity of millions, or billions, of people, influenced by forces seen and unseen. You could argue that, given the variables involved, it would be a marvel if forecasters ever got anything right. The trouble is, the evidence suggests that when it really matters (ie, when recessions are coming down the track) they don't.

What's the evidence?

In 2001 Prakash Loungani, an economist at the International Monetary Fund (IMF), looked at the accuracy of economic forecasts throughout the 1990s both public sector and private sector in the International Journal of Forecasting. He concluded that "the record of failure to predict recessions is virtually unblemished". As Tim Harford notes in the FT, in 2014 he repeated the exercise with a colleague, Hites Ahir. Staggeringly, he found that even in September 2008 well after the demise of both Bear Stearns and Northern Rock, and the same month that Lehman Brothers collapsed the consensus among economists "remained that not a single economy would fall into recession in 2009". In reality, 49 of the 77 countries under consideration fell into recession that year.

Advertisement - Article continues below

A large part of the problem seems to be that forecasts are largely backward looking. A Harvard study by Lant Pritchett and Larry Summers, cited by The Economist, found that forecasters tended to assume that fast-growing countries will keep speeding along, while the economic tortoises continue to crawl. In fact, the opposite is broadly true: "regression to the mean is perhaps the single most robust and empirically relevant fact about cross-national growth rates".

What's the reaction from economists?

Some seem to think that we shouldn't take forecasts so seriously. Haldane's mea culpa on behalf of the profession clearly rankled with former Bank colleague David Miles, for example, who wrote a (friendly) demolition job in the Financial Times. If existing economic theory told us that events such as the financial crisis should be predictable, then "maybe there is a crisis", he argues. But "economics says no such thing.

In fact, to the extent that economics says anything about the timing of such events it is that they are virtually impossible to predict; impossible to predict, but most definitely not impossible events. Any criticism of economics' that rests on its failure to predict the crisis is no more plausible than the idea that statistical theory needs to be rewritten because mathematicians have a poor record at predicting winning lottery ticket numbers."

So why forecast at all?

Good question. As the plethora of forecasts rolled out to back or oppose Brexit demonstrate, economics, as a social science, can readily be skewed to serve political and business interests. Meanwhile, ever more complex mathematical models and jargon have given the discipline an aura of false precision. Yet, while we cannot know the future, that doesn't mean we shouldn't try to plan for it. But perhaps our methods need to change.

Is there a better way?

Forecasting isn't just a struggle for economists if 2016'spolitical shocks taught us anything, it's that forecastingis hard, says Tim Harford on his Undercover Economistblog. This is nothing new in 2005, a study by psychologistPhilip Tetlock found that across the 1980s and 1990s, expert"political and geopolitical forecasts had been scarcelybetter than guesswork."

However, Tetlock has alsofound that our forecasting skills can be improved.In Superforecasting, written with Dan Gardner, he explainshow in more detail. But one key characteristic of successfulforecasters is that they "aren't afraid to change theirminds, are happy to seek out conflicting views and arecomfortable with the notion that fresh evidence mightforce them to abandon an old view of the world". Giventhe defensive, political nature of academia, perhaps it's nosurprise economics has found it hard to adapt post-2008.



Investment strategy

How the fear of death affects our investment processes

Many of our investment decisions are driven by one simple fact: the knowledge that, one day, we will be dead. Here, in an extract from his new book, J…
2 Jan 2020

The good investments of the 2010s – and the bad

John Stepek takes a look back on which investments did well and which did badly in the decade that’s about to come to an end.
26 Dec 2019

How long can the good times roll?

Despite all the doom and gloom that has dominated our headlines for most of 2019, Britain and most of the rest of the developing world is currently en…
19 Dec 2019
Buy to let

The death of buy-to-let property is a useful cautionary tale for all investors

Investing in buy-to-let property was once a perfectly valid thing to do. But the government killed the market. John Stepek explains what investors sho…
29 May 2020

Most Popular

EU Economy

Here’s why investors should care about the EU’s plan to tackle Covid-19

The EU's €750bn rescue package makes a break-up of the eurozone much less likely. John Stepek explains why the scheme is such a big deal, and what it …
28 May 2020
Industrial metals

Governments’ money-printing mania bodes well for base metals

Money is being printed like there is no tomorrow. Much of it will be used to pay for infrastructure projects – and that will be good for metals, says …
27 May 2020

In support of active fund management

We’re fans of passive investing here at MoneyWeek. But active fund management has its place too, says Merryn Somerset Webb.
25 May 2020