Canterbury University's Tom Coupé on whether we can rely on economic forecasts, the bad reputation of forecasts, why they are important, whether we can become better at forecasting, and whether forecasting happiness is any easier

Canterbury University's Tom Coupé on whether we can rely on economic forecasts, the bad reputation of forecasts, why they are important, whether we can become better at forecasting, and whether forecasting happiness is any easier

This week’s Top 5 comes from Tom Coupé an Associate Professor in the Economics and Finance Department at Canterbury University.

As always, we welcome your additions in the comments below or via email to david.chaston@interest.co.nz.

And if you're interested in contributing the occasional Top 5 yourself, contact gareth.vaughan@interest.co.nz.

1. Can one rely on economic forecasts?

At the end of May, the treasury published its 2019 Budget Economic and Fiscal Update, which includes the Treasury’s projections for various economic statistics until 2023. One of the Key economic forecast judgements and assumptions the report makes is that ‘Net migration declines from 50,000 people in the 2017/18 fiscal year to 25,000 in 2021/22 and is steady thereafter’.

But, as a recent Interest.co.nz article points out, not everybody agrees with these assumptions.

 “Finance Minister Grant Robertson doesn’t believe immigration numbers will fall by as much as Treasury’s projections suggest. A key assumption in Budget 2019 was that net immigration would fall from 50,000 people in the 2017/18 year to 25,000 in 2021/22.

Asked by interest.co.nz whether any government policies around migration would need to change for this forecast to be met, Finance Minister Grant Robertson said: “I’ve long held the view that the Treasury’s model on this is - depending on your view - optimistic or pessimistic, and I don’t necessarily believe we will reach 25,000 in this forecast period.”

National's Finance Spokesperson Amy Adams said that when she was Associate Finance Minister she too considered Treasury's migration forecasts to be "a bit of a nonsense", largely because they were based on lower historical averages.”

2. Economic forecasts’ bad reputation

In fact, economic forecasts are often met with scepticism, as witnessed by some headlines like this recent one from Bloomberg.

“With recession talk returning to haunt financial markets and the corridors of central banks, a review of the past suggests that those who are paid to call turning points in economic growth have a dismal record.”

Similarly, from the Guardian and from Time magazine, or the Financial Times.

Consistent with this, a European survey found that, when asked: ‘Personally, how much trust do you have in the official statistics, for example the statistics on unemployment, inflation or economic growth? Would you say that you tend to trust these official statistics or tend not to trust them?’ half of the Europeans answered they tend not to trust official economic statistics

Compared to this, trust in official statistics in NZ still seems high. A 2017 trust survey showed that 68% of business respondents and 98% of central government respondents trusted official statistics.

3. Economic forecasts are important, however

While economic forecasts might not always be accurate, you can’t avoid them as ING chief economist Mark Cliffe explains:

“Given that economic forecasters are so often caught out by shocks, one might ask why they bother. The short answer is that they have no choice. Even when they are well aware of the fallibility of their analyses. People simply cannot live without predictions. Because all decisions – in business, politics, or even one’s personal life – are based on some idea of what the future holds, demand for forecasts is insatiable. People want to be able to justify decisions that they would have made anyway for other reasons. And when things go wrong, they can always blame the “experts.”

4. Can we become better forecasters?

This top 5 was inspired by a book I recently read: “Superforecasting: The Art and Science of Prediction” by Philip Tetlock and Dan Gardner. This book describes the results of the ‘Good Judgement Project’, a large scale forecasting tournament where several thousands of people were asked to forecast things like “will the price of gold by higher than X at date Y” and “will the Euro fall to below Z by date T”.  Tetlock and his coauthors then studied the characteristics and approaches of those consistently doing well at forecasting, people they call ‘superforecasters’.

Based on their analysis, Tetlock and Gardner came up with 10 commandments for aspiring superforecasters, which can be found here. One of the things they stress is that one should try to make precise, numerical forecasts:

“Few things are either certain or impossible. And “maybe” isn’t all that informative. So your uncertainty dial needs more than three settings. Nuance matters. The more degrees of uncertainty you can distinguish, the better a forecaster you are likely to be. As in poker, you have an advantage if you are better than your competitors at separating 60/40 bets from 40/60--or 55/45 from 45/55. Translating vague-verbiage hunches into numeric probabilities feels unnatural at first but it can be done. It just requires patience and practice. The superforecasters have shown what is possible.”

Having numerical forecasts also improves accountability and makes it easier to learn from both one’s failures and one’s forecasting successes:

“Don’t try to justify or excuse your failures. Own them! Conduct unflinching postmortems: Where exactly did I go wrong? And remember that although the more common error is to learn too little from failure and to overlook flaws in your basic assumptions, it is also possible to learn too much (you may have been basically on the right track but made a minor technical mistake that had big ramifications). Also don’t forget to do postmortems on your successes too. Not all successes imply your reasoning was right. You may have just lucked out by making offsetting errors. And if you keep confidently reasoning along the same lines, you are setting yourself up for a nasty surprise.”

Note that the NZ treasury does regularly analyse its past forecasting accuracy and has a dedicated forecast accuracy section on its website:

“The Treasury has updated the analysis of its forecasting performance, last published in 2016.

The key findings are:

  • The accuracy of the Treasury's GDP growth and tax revenue forecasts has improved over the last three years.
  • The accuracy of the Treasury's CPI inflation forecasts has declined a little, through a period of unusually-low inflation.
  • When compared to other forecasters of the New Zealand economy, the Treasury's forecasts of real GDP growth and CPI inflation are amongst the most accurate.
  • In each of the last six years, the Treasury has achieved its target of having one-year-ahead Budget tax revenue forecast errors of less than ±3%.”

5. Is forecasting happiness any easier?

Of course, maybe it’s time to stop forecasting GDP and focus on forecasting happiness. Unfortunately, that doesn’t seem any easier.

“Daniel Gilbert has been researching affective forecasting for decades. His bestselling book Stumbling on Happiness, his popular TED talks, and countless writings have shed light on our lifelong odyssey to happiness, as well as the surprises that inevitably tag along. Happiness, Gilbert points out, is a fast moving target. As passionate as we are about finding it, we routinely misforecast what will make us happy, how long our joy will last, and how intense it will be.”

We welcome your help to improve our coverage of this issue. Any examples or experiences to relate? Any links to other news, data or research to shed more light on this? Any insight or views on what might happen next or what should happen next? Any errors to correct?

We welcome your comments below. If you are not already registered, please register to comment.

Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.

17 Comments

Comment Filter

Highlight new comments in the last hr(s).

People simply cannot live without predictions. Because all decisions – in business, politics, or even one’s personal life"

Yes, but why did we change from tea-leaf reading and entrails-peering, to economics? Why didn't we decide to measure the real? And when the real was put under our noses, why did we choose - emphatically in the case of economics - to ignore it?

https://en.wikipedia.org/wiki/The_Limits_to_Growth
In 2014, Graham Turner concluded that "preparing for a collapsing global system could be even more important than trying to avoid collapse."[44]"

https://en.wikipedia.org/wiki/William_R._Catton_Jr.#Overshoot:_The_Ecolo...

It always amazes me, that if walls could talk...... Catton lectured at Canterbury.......

I find it interesting that you always seem to complain about economic forecasting models, yet always refer back to the limits of growth which you must realise is fundamentally a severely biased modelling approach and was never designed to be a forecasting tool?

It's also not clear what exactly you despise about economic forecasting - is it the modelling techniques? The data choices? The forecast errors?
I'd be interested to know.

Aside from that, a great Top 5 from Tom Coupe.
Out of sample forecasting is an exceptionally difficult task. In many cases the goal is pretty much to beat a coin flip.
Few people realise the difficulty. If they did, I dare say they would have a renewed respect for those whole actually understand time series statistics/econometrics.

I always thought of it as another propaganda channel, it always points at something 'they' want us to believe in. It's like "fake it 'till you make it" kind of thing. With hidden agenda

Can you explain why you think the Limits To Growth study was biased?

At its essence it looked at what would happen to resources, population and the ecosphere if our economies increased in size exponentially.

Graham Turner's review showed it to be a good model.

Very inconvenient if you are an economist who believes economic growth is the normal state of affairs.

It is a static model. No dynamics at all.
Nor is there a strict assumption of mean reversion within the data. Surprising given that everything else was pretty much modeled on the basis of assumptions. Hence why I don't understand why people take the limits to growth as gospel but then (arbitrarily) rubbish modern economic modelling.

And, no. The model is way off in it's performance.
Population growth is the only thing that is relatively close and that is just a Malthusian result. It is not at all accurate in it's predictions of the spline of resource consumption, output, pollution, etc.

Static models only need reviewed, to see how they're tracking. And that one has been reviewed, in 10-year updates. Plus the Turner appraisal, plus this:

https://limits2growth.org.uk/wp-content/uploads/2016/04/Jackson-and-Webs...

And you haven't answered the question. What 'bias'? The only bias I know of, projection-wise, is a regime which thinks that at a certain 'price point', a substitute will always be found. I knew enough 10 years ago, to know that there is no substitute for fossil energy. My failure was to assume that economics knew some of what it was talking about, thus I agreed with Goldman Sachs that we would be seeing $200-a-barrel oil.

I've move on in my thinking (lost a lot of respect for economists in the process, I can tell you). This is where I'm at now:
https://ourfiniteworld.com/2019/07/10/why-stimulus-cant-fix-our-energy-p...

The sad failure is the siloing in academia. That treatise of Cattons (Overshoot - which I suspect you will resist reading) should have been general reading at Cant'y. So too, should this:
https://www.youtube.com/watch?v=PZrPDKTo-50

considering where she comes from.......

I have no problem with the accurate reporting of data (real). What I have trouble with, is folk studiously avoiding the inconvenient (externalities, plus total failures to factor-in) then projecting percentage growth of consumption on a finite planet. Those folk must be treated as believers in mythical deities must be treated, at this stage in Homo Colossus' affairs; they must be ignored. We have physical, existential problems largely brought on because we believed the projections of economists (they failed to predict oceanic plastic soups, climate change, The Ozone hole, a 50% reduction in ocean biomass......in short, they cannot predict anything that matters).

We need to move on; they need to move in the direction of Systems.

Not dynamic. The components of the model are all endogenous - bias.
Misspecification. The simplistic choice of variables and ignorance of the true functional form of those variables - bias.
Ironic you would conveniently ignore this given that when apparently you have such an issue with economic models that ignore 'externalities'.

By the authors own admission it was never meant to be anything other than indication of future resource extraction and their conclusion was that at some time in the next 100 years, resource use would peak.
You are literally taking this narrative and making the data fit to it. The definition of bad science.

I'm sorry. But you need to be called out on this ongoing vendetta against something you quite clearly have double standards about.

Now we're getting somewhere.
That's a systems model - it has multi-directional feedbacks - how is 'edogenous' (a) applicable and (b) bias? References please.

Ignorance of "the true functional form" - definition of 'true functional form' please. Bias as compared to what, exactly?

There are no double-standards, only a species of short-term interest, which chose to indulge itself within a bounded system without measuring the limits of that system. It drew down exponentially, polluted exponentially, and grew it's population exponentially. Only one way that ends - badly. I need no referencing to state that (unless it be the Apollo photo of Éarth Rising...). It's just a fact.

And I'd point out to you, that the 'sometime in the next 100 years' prediction was made 50 years ago. If you were being factual in your rejection, you'd have said 'within the next 50 years'. That was cherry-picking - with bias.

You and I live in a system which is totally built to run on fossil fuels - never-more trucks, never-more aeroplanes, and merely a belief that we can somehow change the whole lot in no time flat. While having trouble financially, refugee-ly, politically and resource-ly. And while not understanding that there is NO replacement energy-source of fossil-fuel equivalent, although we will indeed end up running on renewables, by depletion-initiated default.

Read the first Para of this: http://energyskeptic.com/2014/dennis-meadows-collapse-is-inevitable-now-...
then read the rest.

http://www.conversationearth.org/limits-growth-dennis-meadows-116-encore/
"Our computer-generated scenarios all showed this growth stopping in the early decades of the 21st century, and, I must say, looking back now, it seems that we’re right on schedule.”

"Our work challenged the foundations of modern economic theory. It made life for politicians very uncomfortable, and threatened corporations who were looking to increase their markets. So, all of them, especially the economists, really lit out after our work, and criticized it roundly….”

Kindly correct your '100 years' and apologise, eh?

I research, and stick to fact. What double standards are you on about? I acknowledge that rear-view counting may well be of use. What I absolutely contend is that projecting recent growth in physical consumption (and it appears we need dairying, tourism, houses and coffee - all physical - you lot haven't proven any decoupling so far, outside of deferred maintenance) and using that to predict more, is imbecilic.

The problem is that we extract. consume, excrete. That's like a steering-rack being passed across a pinion - and economists are monitoring the rate at which the pinion revolves. As the rack gets pushed faster, they can indeed monitor a 2% increase in revolutions per year. But can they anticipate the end of the rack? That needs a different measure; remaining stocks (including stocks of absorption-capability) being somewhat obvious, one would have thought. Which makes Julian Simon look somewhat ignorant, in hindsight.....

Struth, I find it hard enough to believe the numbers in hindsight let alone the numbers in forecast.

.. I suspect that the future isn't what it used to be ...

Don't like making predictions, especially about the future

I am forecasting an obesity and type 2 diabetes emergency for NZ ... as such , I'm sending some suggestions to our leader , Julie Anne Genter ...

... firstly , bread ! ... to combat this health emergency thick sliced bread will attract a levy of 50 cents per loaf , regular sliced bread will be rebated 10 cents per loaf ... and super thin sliced 3 mm bread will be rebated 20 cents per loaf ....

Unsliced loaves will be banned , effective 1 April 2020 : anyone caught carrying or selling such contraband will be immediately fined $ 100 000 , even if the loaf was not in a plastic bag ...

You heard it first here , folks : our nation is in State of Health Emergency : the war on obesity has begun !

Time to fund a school lunches programme like japan that teaches a sound respect for healthy food culture, manners and responsibility to others. And releases mums from the tyranny of the lunchbox. Otherwise a good chunk of our youth will have metabolic disese by 35. Just look around . It's a crisis.

GBH,

it's good to see you back in form. Don't go back to being Mr Grumpy.

Superforecasting and The Good Judgement Project is covered in my current reading,David Robson's The Intelligence Trap. It's a fascinating book and i would recommend it.

My last 2 books were Kate Rowarth's Doughnut Economics and Johnathan Aldred's Licence to be Bad, How Economics Corrupted Us. Both books effectively rip classical economics to shreds and rightly so.

Most economists should submit themselves for retraining as something useful to society,but I'm struggling to think what that might be.

Engineering or food-production. We need to morph everything in no time flat.

I'm just staggered that this: https://www.aemslab.org.nz/background
can come from the same institution, with no apparent facility to mesh, compare and discard - in this case, to discard economics.....