By Khyaati Acharya*
Big data is one of the transformative mega trends shaping the way modern businesses operate. By analysing the vast quantities of data at their disposal, businesses can reshape their operations to meet shifting customer requirements in real time, with the benefits reaped in greater profits.
However, while greater volumes and more detailed data hold significant potential for both public and private organisations across industries, a new report indicates that Kiwis are not using data as effectively and as widely as we could.
That New Zealand is lagging behind international counterparts in data driven innovation in part reflects our small, geographically isolated economy. But it also reflects a lack of understanding in many organisations about the capabilities data may hold.
Recently, the Innovation Partnership, in collaboration with Sapere Research Group and Covec economics consultancy, released their latest report Data Driven Innovation. The report looks into the immense value-adding potential of the ‘data revolution’ and the benefits that greater and more effective information use will have for businesses, consumers, governments and taxpayers.
More importantly, the report asks whether New Zealand is reaping as many benefits as it possibly could from data stockpiles, improving decision-making, cost-cutting, and streamlining operations management.
The report estimates that in 2014, data driven innovation (DDI) created $2.4 billion of value in New Zealand. The authors argue that “data driven innovation could easily deliver $4.5 billion in benefits within five years with higher uptake from business and government”.
While the headline figures look impressive, the report also notes that the impact of DDI in New Zealand is still much lower than that of comparable economies. They estimate that if New Zealand firms were to embrace DDI at the same rate as Australian firms, the value extracted from the data could double from $2.4 billion to $4.8 billion.
There is no quick fix to this problem. But policies that promote open government data, and reduce constraints on collecting, sharing and re-using data will go a long way to ensuring New Zealand catches up to our international peers.
However, although New Zealand is already recognised as being a world leader in open government data, some of the current legislation in place could frustrate attempts to close this gap.
The Privacy Act (1993) is one such piece of legislation. The act is built on two principles; ‘data minimisation’ and ‘clarity of purpose’, which dictate ethical practises around collection, storage and use of data. According to the report, these two principles mean that “New Zealand’s regulation of the collection of personal data is at the more restrictive end of the spectrum’ compared to international counterparts”.
Big data changes both the economic benefits firms can enjoy and the potential privacy costs when information from different sources is linked together. If New Zealand is underachieving on big data, we may not have that balance right.
And, as analysing and understanding data becomes an increasingly vital component of organisational decision-making, building skills and adjusting educational curriculums commensurate with the data boom becomes essential.
Policies that support data-analysis training and education could have a more substantial – and less contentious – bearing on greater data uptake. Adapting to the growing demand for people with data analysis skills is critical for ensuring New Zealand keeps pace.
Data Driven Innovation also notes concrete examples of where data uptake has translated in real measurable benefits. The Accident Compensation Corporation (ACC) is using data analytics to more effectively sort the 1.8 million claims they receive per year, leading to more appropriate prioritisation of cases. ACC is also utilising data to track injury patterns and causes to identify riskier-industries and invest more wisely in prevention strategies.
Another example is Christchurch Hospital, which, in collaboration with a database called ‘HealthOne’, a data-centric method of patient management, has improved general health outcomes reducing acute medical admission rates to 30% below the national average.
Greater data use by Fonterra has led to more effective adjustments of product supply according to forecast demands. Data driven insights have enabled more accurate quantities of milk, milk powder and butter to be produced and distributed, particularly for the ever-growing Chinese market.
Evidence on the benefits of wider data uptake is clear. There is a great deal of unharnessed value to be created from ever-expanding data quantities. We need to carefully position ourselves so that every company makes use of the big data revolution to guide their businesses.
Fortunately, the Data Evolution Index, which analyses the performance of digital ecosystems, identifies New Zealand as having high levels of digital development. That this upward trajectory is expected to continue is certainly encouraging.
------------------------------------
*Khyaati Acharya is a research assistant at the New Zealand Initiative, which provides a weekly column for interest.co.nz.
7 Comments
Theory meets real world.
Splat.
Sure, big data has great potential. But in some cases, including the biggest teat-puller in the land you quote, others have seen that potential for themselves, and moved to monopolise it.
So if'n yez wants the data from Your farm, back outta the datastore, ye gets to pay the gatekeeper.....
As an old economics lecturer of mine used to say, "Monopolies are Very Bad Things. Unless you happen to Own one..."
I was on the outskirts of a 3 year project to integrate disparate data sources (so it could be analysed) that cost several millions and just got abandoned. The analysed bit never got off the ground as no one was prepared to spend the millions need to get the results.
Couldn't agree less with above comments. It will replace the Census. It will replace much of StatsNZ manual surveys and data gathering and speed them up. It will be integral to how interest.co.nz works; it already is in use here. I wish I had it available when I was running manufacturing companies. However, the downside is that it takes work, different types of analysis. Learning to use it involves lots of dry holes at first. 'Different' is not for everyone, though. It upsets the status quo. The main thing is that it raises questions you didn't realise were important before you could ask them.
I dont disagree that such data isnt valuble the problem is there has to be a NET benefit. As an aside, just for a moment examine the success rates of "big" Government projects, it is abysmal. So we'll see many millions over-spent on a project that will be late, not deliver what was originally expected and it may not even complete wasting a huge amount of tax.
Next consider what you pay for specialist IT ppl to write the code v the ppl you replace (or save). I worked on one project that aimed to save at most $50k per year. In the end it was absorbing 2 full time java programmers and a PM all apid in the $100k range plus my time to support the hardware and OSes, ie somewhere north of $350k per annum. The interesting thing is a BA and myself sat down and worked out the savings and questioned the benefits and the BA got "moved on". Two+ years later the project has collapsed as the costs kept getting higher and the scope smaller. (On top of that I had to install a free open source system to replace the "dropped" bits and currently its about the only thing returning value.
Then there are vendors and providers over-selling the capability of their systems, that alone is a nightmare.
On top of that because NZ is so small and a non-competitive market, software vendors charge the absolute top dollar for their software. As an example I looked at a piece of monitoring software that we thought wasnt bad, we could if in the USA buy it off the web page for $10000US. When we inquired about doing just this we were told no we had to buy off a local distributor who's "Keen price" was $52000AU. This is just one example of many.
We welcome your comments below. If you are not already registered, please register to comment.
Remember we welcome robust, respectful and insightful debate. We don't welcome abusive or defamatory comments and will de-register those repeatedly making such comments. Our current comment policy is here.