The Dismal Track Record of Economic Forecasts

The only function of economic forecasting is to make astrology look respectable.

Ezra Solomon and/or John Kenneth Galbraith.

 

 

The world economy is in crisis.  Twenty million people are unemployed in Western Europe and the figure is rising.  America faces two severe deficits – the federal budget and the balance of trade…

In this grim context, the opinions of economic gurus increasingly dominate business, politics and international affairs.  Yet orthodox economics seems powerless to help.

Economic forecasters notoriously failed to predict the Japanese recession, the depth of the collapse in the German economy …

If this quote sounds like it was from 2009, it just shows how little changes in the accuracy of economic forecasting.  In fact, it is from the introduction to the 1994 book “The death of economics” by Paul Ormerod (published by Faber & Faber Limited).

Not for the first time, the 2008/09 Global Financial Crisis (GFC) exposed the profession of economic forecasting as being as reliable as astrologers.  Chart 1 shows the sequence of average forecasts for 2009 from the Economist magazine panel of economists (they are polled each month).

As late as September 2008, there were no signs that economists expected anything other than a mild slowdown in the advanced economies.  But two months later, they were predicting recessions: and they became ever more pessimistic until mid-2009. 

Chart 1

 

In the case of Australia, the economists were too far too pessimistic in early 2009, completely underestimating the impact of fiscal stimuli and the resilience of Australia’s economy. 

William A. Sherden, in his book “The Fortune Sellers” (published by John Wiley & Sons in 1998) analyses the forecasting skills of several professions, including weather forecasters, futurists, and economists.  In the case of economists, he reviewed the leading research on forecasting accuracy contained in twelve studies published during the period 1979 to 1995 and covering forecasts made during the 1970 to 1995 period.  His conclusions were: 

  • Economists cannot predict turning points in the economy;
  • Economic forecast accuracy drops with lead time;
  • Economists’ forecasting skill on average is about as good as guessing;
  • There are no economic forecasters who consistently lead the pack in forecasting accuracy;
  • There are no economic ideologies whose adherents produce consistently superior economic forecasts;
  • Increased computer model sophistication provides no improvement in economic forecast accuracy;
  • Consensus forecasts (more correctly averages of forecasts) offer little improvement;
  • There is no evidence that economic forecasting skill has improved over the past three decades.

These conclusions, if correct – and there is no recent evidence to the contrary - are damming and suggest that businesses and governments should not rely on economic forecasts.

The implications for business and government decision makers are: 

  • There is a need to invest in getting better forecasting models and indicators;
  • It is important to be nimble and flexible, developing and rehearsing plans for a range of economic scenarios;
  • We should develop a new framework for government policy evaluation, placing less emphasis on economic evaluations and more on vision and values.

Of course, economic forecasting is not easy.  One has to have a model which quantifies the impacts of factors which drive economic growth and these include interest rates, exchange rates, the price of oil, and population and labour force growth.  Then one has to forecast these driving factors, no easy matter.  Sometimes accurate forecasts of driving forces are made.  At other times these forecasts are inaccurate but in compensating directions so the economic forecast is accurate.  And sometimes the forecasts of the driving forces are wrong in directions that reinforce each other's erroneous impacts, leading to wildly inaccurate economic forecasts.  All too often economic forecasts are published without the key forecasts or assumptions about the driving factors.

But economic policy makers and forecasters are not paid handsomely to do an easy job.

I will be adding content to the theme of this article from time to time, so please revisit this page periodically.

Charlie Nelson
August 2010
extended October 2010

A detailed update and extension to this material is included in our forecasting book.  It includes key insights on how to improve the accuracy of economic forecasts.  Published May 2014.

Additional content added September 2010

All "credible" economic forecasters got it wrong.

Persistently pessimistic predictions of the US economy

Additional content added October 2010

Reserve Bank Blunders

False prophecies of recessions

Additional content added January 2011

Economists confess their failings

Additional content added April 2011

Australian government forecasts wrong

Additional content added July 2011

Economic seers blind to the obvious

Update June 2012

Australia's Treasury department is to conduct a review of their forecasting methodology and performance.  The review will be undertaken by a team within Treasury, overseen by an independent external reference group with "relevant expertise".  It would be better to have independent people on the review team and for submissions from interested parties to be called for.  That would deliver a less inward looking outcome.  But I await with interest for the report.

For more information visit http://www.treasury.gov.au/ConsultationsandReviews/Reviews/2012/Forecasting-Methodology-Performance

Update August 2012

No champagne for Australian economic forecasters.

 

Home