Public Service Commissioner Brian Roche on AI, accountability, and the questions New Zealand hasn't answered yet

Interest.co.nz is unpacking how AI will change your everyday life - the risks, the opportunities, and what to actually expect. Our new series brings you the policymakers, experts and industry leaders from New Zealand and overseas, with a new story every workday over the next couple of weeks.

AI will have a transformational effect on the public service, the Prime Minister believes, and he's deploying Public Service Commissioner Brian Roche to make it happen.

Christopher Luxon wants AI action fast and Inland Revenue is Roche's proof of concept for major technological change. It's already been through the overhaul he wants for the rest of the public service.

But there are questions to answer for both those at the public service coalface and the public who depend on it: privacy, data integrity and sovereignty, and how to bring a workforce with you on what could be one of the greatest system changes the public service has ever seen.

Luxon's brief 

Luxon wants efficiency, speed and a better public service experience. And he knows the need for change is urgent. 

"Why can't a mum... work out her Working for Family tax credits, and work out how she may get support for eyeglasses for her daughter, rather than trying to get around the Ministry of Health website and the IRD website and the Ministry of Education websites," Luxon says.

"Why, when you want to get proof of identification, does it take such a long time... to get one for a mortgage, say, or you need to have proof of income?"

"A lot of it is just paperwork shuffled, that's how things used to work in 1995 but in many parts of the world, all of that's been digitised in government," says Luxon.

"Our CEOs of our respective government agencies, even our ministers, aren't as AI literate as they need to be as well, and so being able to upskill everybody is going to be very important, but we don't have time, we're going to keep moving really fast, and so it will be transformational."

The execution

Roche says AI is already in use in the public sector.

"But what we need to do is have an AI world where it's the first thing, I want it to become the way we do our business, that AI is at the heart."

"We actually have to adopt technology, and the head count thing is unavoidable. The government has said, you're going to have to save all this, so we'll do that. We're going to aggregate agencies, we're going to reduce layers and management, we're going to simplify process, we're going to get rid of duplication, and all of those things will come back to technology," says Roche.

To deliver on what the Government has asked - save $2.4 billion, decrease the workforce by almost 9000 roles and transform technology - "we also have to streamline process, simplify, remove duplication, aggregate agencies, so AI is a really important part... but it's not the only thing that we need to do".

Inland Revenue leads the way

On which agencies were leading the way in AI, Roche says it's Inland Revenue. 

IRD went through a $1.5 billion, multiyear overhaul of the entire revenue system, migrating from a 1980s model to a modern platform in 2021, decommissioning 400 legacy applications in a bid to be more customer-focused.  

"What we did in Inland Revenue is actually what the whole of the public management system is going to go through," says Roche.

For how the public service will look in the next two years, Roche says AI will be how New Zealand does business, rather than an add on. 

"It's a rapidly changing technology. It's described as equivalent to the industrial revolution - in our collective lifetimes nothing is going to be as dramatic, and I think there is some truth to that."

"When I think about what the public sector actually does, we do policy, we do regulation, and we do service delivery, those are the three core things, and legislation - all of them have the ability to have AI applied to them, but what we don't want to have is a world where it's only the machine," says Roche.

"The quality of our human capital will become more important in an AI world than it is now, so we'll need to have people who can use the large language models to do their core business. There's still a human factor, and if I look at our own policy, AI can only ever prepare a draft, you as an individual... you're accountable for whatever the advice is."

What's left to answer

When it comes to New Zealand's public service AI roll out, questions around privacy, data integrity, sovereignty and workforce development are still to be answered.  

"Part of the challenge of our adoption of AI is being able to maintain the trust and confidence of the New Zealand citizens about how we store data. It's a live issue for us at the moment," Roche says. 

"It's really hard to give guarantees in this area, but we know that the level of expectation is really high, and we have to deliver to that expectation."

'Sovereign AI'

The holding of data by offshore companies is also an issue; "because governments generally are really beginning to consider this issue about what level of control do they need to have over the data, it's this whole concept of sovereign AI".

"Not all data is equal, but there is some data that you really need to be able to maintain control over, and so you know the British, French, Australians, everyone's working through it, and we're working through it," says Roche.

"I don't have an easy answer at this point, but as a genuine concern."

Then there's accountability. Who is accountable when AI causes something to go wrong, and to what level of AI authority is the public service comfortable with?

Roche says accountability is really important.

"Chief executives remain responsible and accountable for what happens in their agencies, so to the extent that there is an issue with an agency, we would look at on a case by case basis, but accountability is with chief executive. If they have decision rights, then they have accountability."

"If they don't have decision rights, the person who does is accountable."

Then there's how to build up the workforce, those needed to roll out and use new technology. 

"I don't know the answer to that, because that's the biggest issue in workforces generally," Roche says. 

"How do you learn work skills, when you first start work... you just learn by doing things. Some of those tasks will be done by machine, so how are we going to develop talent?"

"It's work that we're doing as a group of chief executives... We are working on that, because the quality of our human capital is going to become our differentiator, and I feel really strongly that we're not investing as much in our human capital as we need."