David Hargreaves ponders whether we need a rethink of how we structure and fund our primary gatherer of statistical information - and maybe we should start with scrapping the 2018 census and starting again

David Hargreaves ponders whether we need a rethink of how we structure and fund our primary gatherer of statistical information - and maybe we should start with scrapping the 2018 census and starting again

By David Hargreaves

If your role is the gathering of information, with a nod also towards creating 'news', it must be a bit galling to find yourself at the centre of an emerging news story. 

For Statistics New Zealand this unpleasant scenario is playing out, particularly in relation to the 2018 census, but also to some extent over its new series of migration statistics.

At time of writing it appears that the 2018 census is becoming something of a political football and the blame game seems to be getting into full swing, with National Party leader Simon Bridges saying Stats NZ are "showing themselves not to be up to it", while Prime Minister Jacinda Ardern is saying we are now "bearing the brunt" of decisions by the previous National Government.

Typically enough the biggest concern for the politicos seems to be over the electoral boundaries and any influence the census might have on that.

I think it's very unfortunate and very unfair on the folk at Stats NZ to put them in the middle like this. Arguments will likely centre around whether the whole thing was well enough funded or whether the problems were of some other derivation. But I don't think that's a question you will ever get a satisfactory answer to.

The Government reportedly provided $120 million for the census.  

In general terms simply throwing money at something is no panacea and no guarantee of success. But equally, corners should not be cut, particularly not with something as vital as statistical information that will inform long term Government planning and loads of money in areas like infrastructure, education, health, etc. 

Gold becomes copper

The very short version of what happened of course is that this census was done with a focus on online returns. The upshot was a 90% completion rate. That sounds okay. But it equates to 1 in 10, or about 400,000 people missing from the information gathered. That's a heck of a gap in the information.

Stats NZ is now trying to plug the gaps and is set to update by the end of this month when information may be released from the census.

Effectively Stats NZ appears to be trying to extrapolate a full snapshot from that 90% figure. 

As I say, 90% sounds okay, but the great thing about the census (and the previous one had a 94.5% response) is that it is meant to be as near as dammit a complete picture. That's a very rare and valuable thing and the information therefore gathered is like gold.

Patching holes

Now, Stats NZ will have some very fine people looking to patch up the holes in the information they have. But however well they do in patching this up, there will, always be a suspicion over the information and whether it is of the quality on which big spending Government decisions on things like infrastructure can be confidently committed. Where we want information as gold, we might end up with copper. Which ain't good enough.

Anything where information has to be extrapolated, say from a sample size, is always subject to some question marks. 

To my mind the labour/employment stats have always been a bit of a worry when viewed through how many times they throw up a result that just catches everybody off guard and then often needs revising. Now the new migration stats are, in their early stages, looking shall we say volatile.

And these are all important pieces of information that again inform big decisions. In the case of migration, obviously if thousands less people have come into particularly Auckland this has big ramifications for housing and infrastructure.

Having pristine statistics is not some 'nice to have' fanciful thing. It's actually absolutely vital and therefore should not be subjected to Government penny-pinching.

I guess we should see what ultimately comes out from the 2018 census. But I believe it is tainted. 

Fix it

I really think if the country's serious about properly fixing this, then the next census should be brought forward, say to 2021. Go back to the door-to-door approach. I know nothing focuses the mind quite as well as someone literally knocking on your door asking you to fill out forms. Hey, it's annoying, but it's only every five years. Whatever it costs I think we need to spend - because the quality of the information gathered will ensure that big ticket spending in future is not targeted in the wrong places.

So do that. Bite the bullet.

And I do think maybe this would be a good opportunity to do a thorough overhaul of how we gather statistical information in this country and how it is used. I'm not sure how you would go about that, but maybe some sort of peer review of Stats NZ's operations. 

This overhaul should look at how Stats NZ is funded and resourced. And whether it is adequately funded and resourced at the moment. My suspicion would be that it isn't. And that's it's trying to do too big a job without enough resources. But without a proper independent review of it we can't really know.

We need now more than ever to ensure that the statistical information on which future planning is based is of the highest quality we can get.

Our politicians need to get behind this and accept that if it takes more money to do the job, well so be it.

We need our politicians all pulling together on this one. Not political point-scoring. But it is difficult to stop politicians being politicians. On this issue, everybody should make the effort.

We welcome your help to improve our coverage of this issue. Any examples or experiences to relate? Any links to other news, data or research to shed more light on this? Any insight or views on what might happen next or what should happen next? Any errors to correct?

We welcome your comments below. If you are not already registered, please register to comment.

Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.

12 Comments

$120 million. Why does it cost $60 per household when majority of households are nearly free (online forms) and information processing is also nearly free. Does it really cost several $100's per household to chase up the recalcitrant ones. How about fines, restricted internet (fill in forms before netflix turned back on), restricted electricity or with-holding benefits for non-responsive households?

It's $120M for the process from start to finish. Ie planning the questions, setting up the website, then collating and analysing the data that has been collected.

While the in-field costs will be a large proportion of this, it's not like they are spending $120M on 1 single days work.

My parents are 80 and 83 respectively. One is blind and the other has Parkinsons Disease and shakes like a leaf. The don't even have internet, but if they did do you think they'd be able to use it to complete the census form? Does this make them recalcitrant (obstinately uncooperative attitude towards authority or discipline).
No, they just can't see or type on a keyboard.
I used my internet to complete the form on their behalf.......and then a month later they received a letter requesting they urgently complete their census form.
The whole thing was a shambles.

Exactly the right suggestion, DH. Junk the '18 fiasco, stop trying to patch it, concentrate all effort on a 2021 job done right.

But the electoral boundaries etc for the '20 political circus might need an Estimating job. I'd suggest a GiveaLittle page to collect funds for a Giant Dartboard, but that would be just mean....

Not sure why everyone is so obsessed with the 2020 electorate boundaries. We can simply continue to use the boundaries that we have right now.

Is that ideal? No. Is it practicable? Yes.

Another story of another government department/organisation falling short of expectations. If this is the tip of the iceberg, then we can only imagine what other government departments are stuffing up when nobody's looking. This is a real life example of the results of 40 years of substandard education feeding into action in the real world - and they're not even the real world!

Good article David. I have to accept what Adern said about the census as the new format was touted in 2016 and not delivered until 6 March 18 (actual date but early voting happened?). Election was on 23 September 17 so 6 months with Christmas/New Year interval has to be taken into account. What really concerns me is the population statistics, given the DHB's are reliant on funding from the result. 400,000 is a huge gap where ever that may fall. I agree - needs to be done again with door knocking imperative.

Census person should walk around holding a set of scales and weigh everyone. Global stats say we are the 3rd most obese country in the world but I just can’t see it, or can work out how they know how much everyone weighs. In nz the only time a doctor weighs you is if you are visiting him regarding health problems. So pretty much we are only weighing all the fatties.

From their comments so far, Stats have said they will use other data collected by government to fill the holes rather than extrapolation. This means that they are using real data from real people (even though they didn’t fill out the census)

A family member went around a South Auckland suburb chasing up those who hadn't filed online returns.

Many of them were new to the country and a) Deeply suspicious of any government attempts to capture information about them, b) Didn't see or understand why they needed to do it, as they'd never done it before and c) Weren't about to take directions from a woman.

The whole thing was underfunded - hourly rates and the job conditions were apparently much worse than the previous time.

As far as funding goes the only result is you get what you pay for.
I have worked on several Census's There is no substitute for shoe leather and properly skilled staff to do the job.
Census is a snap shot of the Country on one day. from which a hell of a lot of decisions are going to be made, so it has got to be accurate.
Digital Analysis and guess work is a poor substitute,

Not just the census and migration figures. Why does it take NZ Stats twice as long to release quarterly GDP growth as their USA counterparts, even though the latter has nearly 100 times more actual GDP to measure and is subject to frequent Government shutdowns? Why does it suddenly take twice as long to release NZ visitor arrival figures, critical to the Tourism industry, even though the physical arrival card process hasn't changed? Why are Provisional Visitor Arrival figures, which until recently gave the critical Tourism industry an almost real-time indication of performance of each key offshore market, now not even released at all? Hard to fathom how, in the information age, our information provider is providing worse service.