AI in Government: Opportunities and Obstacles

Join us as we explore the intricate world of Artificial Intelligence within the UK government. This episode investigates the findings of the Committee of Public Accounts' report on the use of AI in the public sector. 

We examine the immense potential of AI to revolutionise public services, from automating routine tasks and boosting efficiency to leveraging data for targeted support. 

However, we also unpack the significant hurdles that could hinder successful and ethical adoption. Discover the challenges posed by outdated technology, the crucial need for public trust through transparency, and the persistent shortage of digital skills within government. 

We also analyse the approach to AI procurement and the vital importance of strong leadership to navigate this rapidly evolving landscape.

Key Takeaways:

• AI holds significant promise for transforming government operations and public services.
• Outdated legacy IT systems and poor data quality present major barriers to AI adoption.
• Building public trust through greater transparency in the use of algorithms is crucial.
• The public sector faces a significant skills gap in digital and AI expertise.
• Government needs a systematic approach to learn from AI pilot projects and scale successful initiatives.
• Concerns exist around AI procurement practices and the dominance of large technology companies.
• Strong leadership and clear accountability within government are essential for successful AI implementation.
• A new Government Digital Service has been established to unite efforts on digital transformation under the Department for Science, Innovation and Technology (DSIT).

Source: UK Democracy: Impact of Digital Platforms Volume 765: debated on Thursday 3 April 2025

Chapters

• 1:37 Out of date legacy IT systems
• 3:13 Risk-averse data sharing within government
• 3:59 Public trust
• 5:24 Gaps in digital skills
• 7:20 Wider AI adoption
• 8:47 AI procurement process
• 9:58 Leadership and governance
• 11:35 Can the public sector seize the opportunity?
• 12:13 Conclusions

Transcript

Welcome again to The Bench Report, UK politics straight from the benches. We are your hosts, Amy and Ivan. Today, a subject that we both are very interested in for reasons you have hopefully worked out by now. Indeed. And what's got us particularly engaged today is a deep dive into this recent report, from the House of Commons Committee of Public Accounts on the use of artificial intelligence in UK government.

This report yeah. We've gone through it, this detailed report, and we've pulled out the most crucial insights. Think of this as your fast track to understanding the real story behind AI in the public sector, the opportunities, and more importantly, the hurdles. Our aim with this deep dive is to highlight the moments. Those key pieces of information that will give you a clear picture of where things stand with AI adoption in government and what challenges lie ahead.

So whether you're prepping for discussion, trying to wrap your head around this rapidly changing field, or just curious about how AI is impacting our government, we've got you covered. We'll be starting by looking at what's slowing down the wider use of AI. This includes things like, outdated technology, the quality of data held by government, the level of public trust in AI, and whether the civil service has the right skills. Then we'll explore what's being done to encourage more AI adoption, such as learning from the AI projects that have already been tried and the government's approach to buying AI technologies. And finally, we'll discuss the critical role of leadership, how the government is structuring itself to make the most of AI.

So the report really starts by laying bare some fundamental problems. It points to these out of date legacy IT systems and the poor state of government data as major obstacles to successful AI implementation. What's fascinating here, it isn't just that these systems are old, but the sheer extent of the problem. AI needs good, clean data to work effectively. And the report makes it clear that a lot of government data is neither of those things and is often trapped in these older systems that don't easily talk to new technologies.

It's quite staggering. They mentioned that back in 2022, '70 '2 high risk legacy digital systems were identified. And as of now, 21 of those still haven't even had funding allocated to fix them. Think about that. Critical parts of our public infrastructure potentially running on tech that's not fit for purpose.

It makes you wonder about the risks beyond just hindering AI, like potential security vulnerabilities. Exactly. If we connect this to the bigger picture, these outdated systems aren't just an AI problem. They impact the government's ability to be efficient and secure across the board. The committee is right to be concerned about the slow progress.

It's not a quick fix requiring sustained investment and clear priorities to upgrade both the software and the underlying hardware, including the move to cloud services, which itself is complicated by these older systems. It's not just the pipes and wires, is it? The report dives into the quality of the data itself. Apparently, a large chunk of government bodies see poor data quality as a significant barrier to using AI effectively. It's not just about having the data.

It's about having data that's accurate, consistent, and in a usable format. What's fascinating here is that this isn't solely a technical challenge. The report also touches on the culture around data sharing within government. There seems to be a reluctance, a risk averse approach that sometimes prevents departments from sharing data even when it could lead to better public services or more effective AI applications. Which seems a real shame because some of the evidence submitted suggests that greater data sharing, even with trusted partners in the private sector, could really spark innovation.

But then you immediately hit that crucial point about public trust, the absolute necessity of handling sensitive data responsibly. It's a tightrope walk, isn't it? It is. The public needs to have confidence that their data is being used ethically and securely, especially with AI involved. This leads us directly to the next major hurdle highlighted in the report, public trust itself.

And it sounds like building that trust is taking longer than it should. The report shines a light on the algorithmic transparency recording standard. This is supposed to be a place where the government details the AI tools it's using so the public can see what's happening. What's really striking is the incredibly low number of entries. As of January 2025, only 33 records have been published on this website.

33. That really makes you wonder what's going on behind the scenes. And And it doesn't exactly scream openness, does it? It's hard to build trust when you can't see what's being done. Not at all.

And the committee is understandably worried about this lack of transparency. They make it crystal clear that being open about how AI is being used is absolutely fundamental to getting the public on board. Without that, there's a real danger of people becoming wary of sharing their information, which would then make it even harder to develop AI for public benefit. The report does note that the government itself recognizes the need to do more to communicate effectively about its AI use to demonstrate that it's being done in a trustworthy way. They talk about developing technical standards and strengthening oversight of those AI applications that carry the highest risk.

And it's encouraging that they see the need for both support, helping departments use AI responsibly, and clear rules, the stick of central assurance as they put it. This consistency across government departments seems crucial. Moving on, another significant barrier that keeps cropping up is the ongoing shortage of digital and data skills within the public sector. This isn't a new issue, but it sounds like it's particularly critical when we're talking about something as specialized as AI. Absolutely.

The report throws out some pretty stark figures. In 2024, around half of all advertised digital and data roles in the civil service went unfilled. And the gap in pay compared to what people could earn in the private sector, especially for more technical roles like technical architects, is really significant. We're talking about a 35% difference in some cases. That kind of difference makes it incredibly tough for the government to compete for the talent it needs.

What's also concerning is the mix of skills within the existing digital and data workforce in government. The report hints at a potential overreliance on bringing in external consultants for more specialized technical skills. While that can be useful in the short term, it's not always the most sustainable or cost effective approach in the long run. You want that expertise built in house, don't you? Definitely.

And the committee has flagged these digital skills gaps before, so they're understandably a bit skeptical about whether the government's latest plans will really be enough to fix the problem. These plans include things like trying to highlight the positives of working in the public sector, encouraging movement of people between public and private roles, and various training initiatives. And the blueprint for modern digital government does outline some key reforms, like looking at how people are employed and whether the pay is competitive, as well as ensuring that digital leaders have a seat at the table on executive teams. The report points out that currently chief digital information officers often aren't represented at that senior level. This raises a fundamental question about how much influence digital expertise really have in government decision making.

So we've looked at what's holding things back. Now what's being done to help wider AI adoption? The report stresses the importance of learning from the AI pilot projects that are already underway. Apparently, there's quite a bit of experimentation happening across different government bodies. Yes.

Everything from analyzing images to summarizing documents and spotting trends in live data, the Incubator for AI has even developed tools to help with tasks like taking meeting minutes and analyzing responses to public consultations. So there's clearly a lot of activity happening at the ground level. But the real challenge seems to be turning these small scale tests into actual widespread use. And the report highlights that there isn't really a joined up way of collecting all the lessons learned from these different pilot projects and sharing them across government. It's like everyone's reinventing the wheel in their own little corner.

This lack of a central system creates a real risk of different departments and agencies wasting time and money by doing the same things and making the same mistakes. It's crucial to have a way to share what works and what doesn't. The report does give a nod to the evaluation task force's guidance on how to assess the impact of AI tools and to DSIT's plan to pilot an AI knowledge hub. This hub is intended to be a central place for sharing good practices and learning. That's definitely a step in the right direction, but the committee emphasizes that it will take real effort and strong leadership from DSIT to make this work and prevent this siloed approach to AI pilots.

It needs to be more than just a website. It needs to be an active driver of collaboration and knowledge sharing. Another crucial area for supporting AI adoption is how the government actually buys these technologies, the procurement process. The report suggests that the current ways of buying things might not be well suited to the fast paced world of AI. That's right.

The traditional approach of buying software and hardware upfront doesn't always fit with the subscription based models that are common in the AI sector. There's also a concern that a few big tech companies might dominate the market, making it harder for smaller, more innovative suppliers to get a foot in the door. Some of the people who gave evidence to the committee worried that the government's procurement methods just aren't agile enough to keep up with how quickly technology is changing. This could lead to the government being locked into using older, less effective technologies and missing out on new opportunities. The government has acknowledged these concerns and is taking steps to address them.

They're planning to create a specific AI sourcing and procurement framework and established a digital commercial center of excellence to build more expertise in this area. How successful these initiatives are will be key to ensuring the government gets good value for money and fosters a diverse and competitive AI supplier market. Finally, the report turns its attention to leadership and governance. It points out that when the report was being put together, responsibility for AI and government was split between the cabinet office and DSIT, which created some confusion about who was in charge and how things were being coordinated. The report highlights that the draft strategy for public sector AI adoption at the time didn't clearly state which department had overall ownership.

To address this, there's been a significant shakeup. The central digital and data office, the incubator for AI, and the government digital service have all been brought under the umbrella of DSIT. And this newly unified government digital service, now led by a second permanent secretary level government chief digital officer, is seen as a chance to strengthen leadership and really drive forward digital transformation, including the adoption of AI. The committee welcomes this move, but rightly points out that it's still early days. They'll be keeping a close eye on whether these changes actually resolve the previous issues around complexity and accountability.

The report also mentions the creation of a new digital inter ministerial group, which is intended to bring more focus to digital transformation and link it to the government's wider priorities. DSIT has emphasized that this shows a significant increase in how much government ministers are involved in this agenda. And the cabinet office believes this group will improve how strategy is developed, how priorities are set, and how different parts of government work together. However, the report underscores that strong political leadership and clear prioritization are absolutely vital for DSIT to be able to effectively influence other departments and drive AI adoption across the public sector. The committee wraps up by questioning whether the public sector will truly be able to seize the opportunities that AI offers, given the significant scale of the challenges they've outlined.

DSIT acknowledges that digital leadership across government needs to be strengthened with better representation at senior levels and more technical expertise among leaders. As part of their blueprint, they're aiming to have a digital leader on all public sector executive teams and boards by 2026. And they're planning to release a new digital and AI roadmap later this year with more detailed plans. It's a clear recognition that leadership is fundamental to making this all work. So to quickly recap the core challenges this report lays out.

We see the significant hurdle of those outdated legacy IT systems and the poor quality of government data with a concerning number of high risk systems still awaiting funding for essential upgrades. Then there's the slow progress in building public trust through transparency, clearly illustrated by the incredibly low number of records on the algorithmic transparency recording standard website. The persistent shortage of digital and data skills within government remains a major worry with high vacancy rates and a significant pay gap compared to the private sector making recruitment and retention incredibly difficult. The report also highlights the current lack of a systematic way to learn from the various AI pilot projects happening across government and scale up the successful ones. And there are valid concerns about whether current government procurement processes are flexible enough to keep pace with the rapidly evolving AI market, potentially disadvantaging smaller, more innovative suppliers.

Finally, the report underscores the critical need for strong leadership and clear accountability within government to effectively drive forward AI adoption even with the recent reorganization under the new government digital service. The report makes it clear that while AI has the potential to significantly transform public services by automating routine tasks, boosting efficiency, and providing more targeted support. Actually realizing this potential depends heavily on the government successfully tackling these fundamental interconnected challenges. Considering the sheer scale of these challenges, from those deeply rooted technology issues to the skills gaps and the crucial matter of public trust, what single area strikes you as the most critical for the government to get right to successfully harness the transformative power of AI for the benefit of the public. It's a complex puzzle, and addressing any one of these in isolation won't be enough.

But perhaps focusing on building that foundational layer of trustworthy data and modern technology infrastructure would create the strongest platform for progress in all other areas. Please subscribe to the bench report to spark your passion, stay informed, and change the world. Find us on social media at bench report UK, and look out for our new substack. Take care.