Auditing with data: for Performance Auditors and Internal Auditors that use (or want to use) data
The podcast for performance auditors and internal auditors that use (or want to use) data. Produced by Risk Insights.
Auditing with data: for Performance Auditors and Internal Auditors that use (or want to use) data
37. Using data in performance audit with Gemma Diamond and Morag Campsie (Audit Scotland)
Morag (Senior Manager) and Gemma (Director) are Performance Auditors with Audit Scotland.
In this episode we discuss how they are using data in their audits, their experience in developing Audit Scotland's data capabilities, collaboration with other audit offices and what the future roadmap looks like.
We also discuss their focus on ensuring their reports meet their audience’s needs, and how data visualisations and charts are being used to communicate key messages.
About this podcast
The podcast for performance auditors and internal auditors that use (or want to use) data.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).
You're listening to The Assurance Show. The podcast for performance auditors and internal auditors that focuses on data and risk. Your hosts are Conor McGarrity and Yusuf Moolla.
Conor:Today we are speaking with Gemma Diamond and Morag Campsie from Audit Scotland. Welcome to the show ladies.
Gemma:Great. Thank you for having us.
Morag:Yes. Thank you.
Conor:Okay. So most of our listeners will be fairly familiar with Supreme Audit Institutions and Audit Offices. But it might be useful just to give us a little bit of an intro about Audit Scotland, its mandate, and your roles.
Gemma:Yeah, absolutely. So quite standard in terms of an audit institution, Audit Scotland, but a bit unusual in terms of how it works with audit bodies across the rest of the UK. So Audit Scotland audits all of the public sector bodies in Scotland. So that includes councils health boards, the Scottish government and other central government bodies. And we do both the external audit, that means the signing of the accounts, and also value for money performance audit work. We work closely with the national audit office who covers the whole of the UK, but audit Scotland essentially deals with those powers that are devolved to Scotland. We also work closely with Audit Wales and the Northern Ireland Audit Office as there are lots of commonalities between the audit offices. I'm an audit director in our performance audit and best value team. I've got a background in both financial audit and performance audit, and I've spent all my career auditing the public sector. Been with Audit Scotland now about 10 years and over the last five years or so, moved into auditing digital.
Morag:I am a senior manager and I've actually got a joint portfolio. So I do some performance audits and financial audits as well. Started out as a trainee financial auditor about 13 years ago, scarily. And, more recently just doing a wee bit of both, and the performance audits I've been involved in have been largely looking at digital transformation. But over the last year or so, I've been more involved in leading our team on data analytics and trying to draw that team together.
Conor:Okay, fantastic. Both of you ladies mentioned there, the focus on data analytics, and that's going to be a core part of the discussion today, and obviously we'll have a focus on how that's being used in performance auditing in particular, Maybe just to give us a bit of grounding early on, can you tell us a little bit more about that performance audit itself? How many audits would you do a year, what's the reach, subject areas, that type of thing?
Gemma:A general number that we normally say is we normally do about 10 performance audits every year within that we have some that we do every year. So for example, each year we produce an overview of national health service. There's a whole range of things, including financial sustainability, but also some of the performance and outcomes information as well. We also do the same in the local government sector. Our other audits cover a huge range of work. So including digital transformation, including education system, some can be quite broad and some can be quite narrow. When we look at our specific project, for example, that might have went into difficulties and we go in and have a look and see what's happened. and see what lessons can be learned from that, to share around as well. So quite a wide range of work, What we're doing at the moment is really having a look and seeing how best we can, report our work and look at our work. So for example, we've in a way, been trying to innovate how we best reach our audience, audit reports can tend to be quite lengthy. So we have now got them down to a 44 maximum pages. They just used to be a little bit longer than that, but recognizing again, our audience has moved to digital, we no longer really send out paper copies to anybody. So how best can we make reports look, and invite the reader in, and invite them to engage with the report. So we're looking at shorter reporting options. How can we support it with infographics, with how we display data, how we can use blogs and videos as well to start to bring those findings to life. That changes as well in terms of the scope of some of the audit work trying to be a little bit more flexible and agile, in how we do things.
Conor:Your performance audit program - is that a forward program over a year, two years, three years, that sort of thing?
Gemma:We've actually now quite a five-year forward work program that is done on a rolling basis. So we try and keep it live In common with a lot of audit agencies around the world following the pandemic, we've been having a rethink in terms of what was in there and what we look at, but what we try and do is have a bit more detail for the nearest two years, and then over the five-year period have the themes that we are interested in. Themes we see as risk areas that we want to have a look in more without actually saying, on this we're going to do an actual audit. We're going to, keep on looking at it, keep on monitoring it and see what audit work would best fit. We have a formal refresh of it each year, but we try and keep it as live as possible to adapt to what's happening out there at the moment.
Conor:The topics that are chosen, is that done by Audit Scotland in isolation or is there for example, engagement with some of the entities you audit or stakeholders to inform the program?
Gemma:Because we keep it live, it's something that we consider all the time in terms of what are the risks coming through, what's coming through from our conversations with stakeholders. How does that change the risk position? Audit Scotland - we have quite interesting, I would say, governance arrangements in terms of, we have an auditor general who covers the health sector and central government. And we have an account's commission, which is a body of 12 people who are responsible for the scrutiny of the local government sector and Audit Scotland is the body that serves both the Auditor-General and the accounts commission. So we obviously have a lot of engagement with the Auditor-General and the accounts commissioners as it's their work program and they themselves have quite a lot of engagement with, for example, local government bodies or bodies across the sector. We also have a lot engagement with the Scottish parliament and the public audit committee at the Scottish parliament, in terms of what risks and work that they are interested in and from risks that are appearing at committees across the parliament. So really wide engagement, that informs a living and breathing work program that can adapt to risks as they come up.
Conor:What are some of the current ways you're using data across your performance audit and best value programs?
Gemma:For each of the audits we do, we always have a good look to see what data's out there. We use it in different phases, so we monitor a lot of the risk areas, a lot of the policy areas and through that, what we'll be doing is just keeping an eye on what the data is telling us. Are there any new datasets? What's performance looking like? Is it going up? Is it going down? Is it showing any risk areas that we might want to investigate a little bit further? So we have that data that we would look at just as standard. And then when we decide to go into an audit area and look at it in more detail again, that involves having to look at well, what data is available. Publicly, what is the data that the public bodies themselves might be holding, but we can't get publicly. What is the data that isn't there that we would like to be there. Can we create that ourselves? Can we set out some questionnaire that gathers some of that data for ourselves. There are occasions when there's just not enough data there for us to go and audit. And in that case, we would have those conversations with the bodies themselves in terms of how best to get some of that data so that we can, do an audit.
Morag:The availability of data, but also data quality- there's just not enough good quality data out there to draw conclusions from. That's a big factor as well in our considerations. Currently, one of the big pieces of data we're looking at is around COVID recovery and looking at some of the business support grants that the government's issuing and other support mechanisms there and trying to track that as well. We're trying to develop tools, in the first instance, internal tools for our auditors to help them see and track the money. And then potentially once we get assurances again around the quality of the data, then we might look to have interactive tools that are external facing that the public and parliament can interact with as well. So that's one of the big things and that's quite a fast moving thing and something we've managed to put in place quite quickly as well.
Conor:So Morag you mentioned there the old chestnut that every audit office around the world encounters at some stage. And that's poor quality data held by the entities that are subject to the audit. And you said there that Audit Scotland has come across that problem on quite a few occasions now. Has there been any moves by those entities to invest in or improve the quality of their data based on your conclusions to that end?
Morag:I think there has, but I think it's quite difficult. There's a number of factors in there. Having the skills and people to be able to collect the data and then work with that and then checking and verifying it. Sometimes organizations are quite small, so they don't have access to those skills, so it can be quite challenging for them. And also, the technology side as well. Maybe not having the best tools to help them analyze and check and verify the data as well. So it's a big challenge, but I think we have seen the body taking on the recommendations and considering how you might best do that. And also obviously one of the big ones being the Scottish government themselves. and it's an area that the Scottish parliament's really interested in as well. So the Public Audit committee that obviously we present reports to have noted this issue as well. So they're quite keen to see this move forward. And in fact, they had a session just last week, that the Auditor-General was at, which was looking at that very thing around data and outcomes, and what some of the issues are around that.
Yusuf:Morag you're talking there about getting data in and, Gemma, you also mentioned something around, producing reports and deliverables that are, broader than what you have at the moment, but able to be consumed more easily, by what is, a very varied audience. How do you determine. What to put out in terms of, publicly accessible information. So dashboards and infographics and the like So you obviously have your performance audit reports, but beyond that, when you're putting out a dashboard, for example, for the public to use, how do you determine who exactly that audience is? Which you then use to drive exactly what that's going to look like.
Morag:You know we try and think about that early on when scoping of the audits who your audience is. because it comes down to when we're drafting the report how we write the report as well. So as much as the data analysis and visualizations that we might do as well. And that can range from, members of parliament, Member of the Scottish Parliament, I should say, and also elected members, councillors at councils, but also obviously the public as well. I think over the last year or two we've tried to put out more dashboards on our website, using Tableau, which allows people to go and drill into the data, at a local level for their own local authority and council too. That's something we're looking to do a wee bit more of and make them a bit more easy to use as well, and a bit more interactive.
Yusuf:It's quite an important thing. Because obviously when you're writing reports, you are thinking about, the parliament and, relevant A-G or, who exactly is going to be. consuming what you produce and when it comes to the public, it's so much more difficult because you've got to think about all sorts of permutations around how something may be used. so good to see that there is, a plan for that, because anything that is publicly available is good. It's a step in the right direction. but we can take that even further by focusing our resources and our outputs on who will be consuming that.
Gemma:Yeah, it's something we recognize. Absolutely. And I think, there's, some times has been a bit in the past of, Oh, we have this data, we should just put it all out there and then people can do what they want with it, so making everything available and I think what we're trying now to think about is how best can we help people use that data and get what they want from it? So thinking about particularly if we have national data, but somebody wants the data about their own area. How can we make it presentable so that they can go in select that data and also tie it more closely into what we're saying in our report. So if we've got five key findings in our report, then how can we have it so that they can drill down into that data, but it's about those findings in our report rather than just here's all the data that we found and that we used in our report, here go and have a look, more that actually, we won't give you everything, but we'll give you the data that most supports those findings. So that's something that we're having those conversations about, as well, I think it's a natural reflex to want to make everything available to the public. But with a little bit more thought we can make it available, but in a way that really helps support our messages and helps support what they want from it as well and make it easier for people to do things. So I think there's always a bit of a balance there and I think there's always a bit of, learning as you go with these things and seeing, actually sometimes you don't know how it's going to be used until you, put something out there and sometimes you're surprised by how well a really small thing does and how much readership say a blog gets. Just recently our Auditor-General put out a little animation that we did that just had some data around inequalities. Things that we've been monitoring, typically during the pandemic and what was happening in the widening of inequalities, and a range of data that we collected, that we put together into a little animation and that's done, brilliantly, got lots of focus cause it's a really short thing that engages people. Again, no audit judgements in there. But just us saying, we're interested in this, this is something that we're going to build into our audit work so come back and see us. This will be part of our audit focus. So it's building through that maturity curve in terms of what we do with the data, how we best show it, how we best engage, with the audience and learning from the feedback that we get each time we do something.
Conor:It sounds like Audit Scotland has a very big focus on data in its work. Are you able to tell us a little bit about how you guys have set yourself up in terms of systems or infrastructure to be able to deal with that?
Gemma:We've really over the last year made quite a significant change in how we do things in trying to bring together a central team, around our digital audit and data analytics, and bringing in a few external people with specific skills to help us to do that. I think what we had before was very good, but what we found was that our people and skills were quite siloed across different parts of the business. So we had some people who were really good in working on data analytics in our financial audit processes. We have some really great people with skills in particular software in our performance audit bit. We obviously have our whole digital team that supports all our infrastructure. and what we've found, we've been able to pull together into a central team, supplement that with some very specific skills that we didn't have in the organization is that we're now less in a silo and more concentrated those skills, into a team, but very much, you've still got the connections out with, the main business groups as well.
Morag:As Gemma said, we had pockets of people across the organization that would do different types of data analytics, depending on the audit. It's fair to say that largely a lot of the analytics that we're doing, we're using tools like Excel or Idea and we recognize that as we're trying to look at bigger data sets that they're not the best in terms of being efficient and effective, for looking at big data sets. So we looked at what other UK agencies were doing and working with them quite a lot more over the last year or two, and trying to move ourselves to using things like R and Python as welll to help do some of that analysis. And that's when we realized we had some people that were self-taught and skilled themselves up in the likes of R, but, doing that themselves and we didn't really know what was possible. So that's why we brought in a couple of people in with a bit more experience around that and also building the right infrastructure too as well. cause that was another thing we had people going on some of these training courses, but then we didn't actually have the tools available or the infrastructure available for them to then put into practice what they've learned. So now we've got that in place, which means we're able to do and develop a lot more to using things like R Shiny, so doing quite a lot of dashboards using that now, which has been really good. And involving the users a lot more as well. And helping to design those has been a big thing as well, because quite often, it's been just these few people that had been working themselves to develop something but not actually engaging with the people that were then actually going to be using the tool. So that's a key thing for us as well, it's a work in progress, but I think we've made huge steps the last year or so.
Yusuf:if you had to think about two years from now three years from now, how different would that look if at all, would you be looking to train up performance auditors that are involved directly in conducting performance audits to do more of that work themselves? or are you likely to maintain a central team that does a lot of that work for them?
Gemma:it's going to be a bit of both and a bit of trial and error and seeing what works in the best way. So I think at the moment, we've recognized that the central team is working well and a good way for us to bring those skills together across the organization. And certainly we are looking to invest in that team more to help us go a little bit further, faster, but what we recognized as well is that we do need to train auditors. What we want is that there are tools that any auditor can pick up and use with minimal training. There might well be some training needed for everybody, but hopefully some auditors who have an interest and want to grow their skills in this area. Maybe have some baseline skills. So for example, if we talk about the COVID tracker that we've just been doing. That was a really great partnership between our central team and somebody who was in the audit team who's looking at that who's monitoring all the announcements who had looked at building some of our skills, but were very, they would say themselves, quite rusty in that, but working with that central team, they've been able to develop those skills and can now essentially be a little bit self-sufficient in maintaining that tool within the audit team. And I think for us, that's a really great model to aspire to that where we see that there's these dashboards and audits, which are data heavy, that we try and get the teams to be as self-sufficient, as they can be, recognizing that the central team's really small, we're talking about essentially three or four people within that team. So there's no way we can support everything that's going across the organization. So to get the audit teams to some level of self-sufficiency with a lot of support and then essentially to have, as a broad principle, the fact that we want the tools to be able to be picked up by any auditor and used within the course of business, without making anybody fearful of the tools, but just use them as expected as they would do any of the tools within the audit process.
Narrator:The Assurance Show is produced by Risk Insights. We work with performance auditors and internal auditors. Delivering audits, helping audit teams use data, and coaching auditors to improve their data skills. You can find out more about our work at datainaudit.com. Now, back to the conversation.
Morag:They come into a pool of data champions, or whatever we might want to call them. In fact, one of my colleagues actually said we should call them data PyRates because it uses R, that one's stuck with me. But I think it's quite a good model just to have that core team, but, it spans out to a pool of others that can be called upon and they themselves can help support other auditors, to expand that knowledge going out the way. Yeah, that's what we're hoping and, ultimately having people that they, the whole, an organization, that's not scared of doing data analytics and kind of making them realize that actually data analytics is what we do anyway. They might not realize it or badge it as that, but essentially we're analyzing data everyday. That's just what we do.
Yusuf:That's exactly. something that would come out of our mouths it's one thing, learning the tools and learning how to actually conduct the data work, but in terms of mindset, amongst the performance audit team, and maybe even, more broadly across, the combined teams, what does that look like? Have you had any challenges in getting people to change the way in which they think about how data is used and the possibilities
Gemma:When something new comes along, there's always a tendency for people to want to hold onto what we do. So to hold onto the trusty Excel worksheets, because they've worked in the past and they're familiar and comfortable and they might be horrible, but they are a known quantity. But what we realized as well is that there are a lot of people across the business who are very interested in trying something new, who are really keen to try something new and with the right level of support, can really take on that challenge and really do well. And I think for me, the more we do that, so the more we show - actually, this isn't something that's really scary, this is something that you absolutely can do. We're here to support you. We're not going to leave you on your own, that will hopefully spread across the business. So the more success we have with the individual audit teams, as people then move across audit teams and do different things, they'll take those skills with them and organically, it will spread. Another thing we're interested in doing a t Audit Scotland, in common with lots of other audit agencies, get a lot of new trainees in every year. So we have trainees that come in largely into our financial audit business, but they do also come into our performance audit business and work across both, get their accountancy qualification and move on up through the business and what we're looking to as well as in terms of how can we best Work with some of these new trainees who are quite often coming in with relevant skills and backgrounds, potentially some coding experience. Now, do you know, that is just the way the market has changed. So how best can we capture their interest and get them involved? So again, as they move up through the business into different audit teams, they've got that knowledge, enthusiasm for it as well. I think there's a number of different ways, but for me, it's really about showing. I think the best thing we can do is really show it, working, give people that confidence, and they'll take that confidence into other teams and other areas and to make it easy. Do you know if you make it easy for people, if you make it so that the tools look nice, sometimes quite often, if something looks pretty, that will win the day, as well, but if it's easy to use, intuitive. which a lot of them are now, do you know, a lot of these tools really are quite intuitive. They quite quickly can produce something that looks really nice and that there's nothing better than getting your audit director to go. Ooh. Wow. That looks fantastic for everybody to go, right yeah, absolutely. This is what we're doing. So I think, for me, it's that it's those kinds of things, but it's, so it is being aware that some people will find this a little bit challenging and how best can we support that? I think as well, something that our wider digital team are aware of in terms of, we're all still working from home at the moment, right in the middle of a, essentially a second wave and we're going to be out of the office for a long time, working virtually using new tools. so it's something that, again, I think just we're aware of in that broader sense, in terms of digital skills and people getting used to new technology and way of doing things is that actually there are absolutely some people who find that really quite challenging and how best can we support them in the broadest sense to get to grips with all the new tools and new ways of working that have been introduced in the moment as well.
Morag:That working from home thing, has been a bit of an eye opener, as Gemma says as well. And I think it's got people thinking a bit differently. So thinking about some of the tasks that they do that could maybe be automated. so there's some tools as well that we're developing around text analysis and document comparison and things like that as well. And I think people working at home and working more digitally, that's made them think about things differently. So you know how we can maybe, introduce some tools that can help with that as well?
Conor:From what you've said, it sounds as if you've come a fairly long way on your data journey in not a long period of time. So I think you said there's been a major focus this past year. To the extent that you can tell us, have you been able to share any of your lessons or things you've observed on your journey with any of the other audit offices?
Gemma:Yeah, we work really closely with other agencies across the UK and Ireland actually. we meet quite regularly and actually within the audit teams, there's a lot of showing what we're doing as well in terms of right, here, we've tried this, look what happened? We found this bit quite hard and sometimes tried to share code across audit agencies as well, so that we try not to repeat ourselves. And I think it's fair to say that we're all In a way it's similar places of our journey, but some agencies accelerated faster than others in different areas. So there's quite a benefit to that. If we can all work together, we are all only essentially have quite small teams. So if we can pull that knowledge and as you say, those lessons, that we've learned that's much better. And I think again, over the last year to six months, I'm really trying to make the most of that collaboration. there's joint projects that we're starting to work on. And what we're trying to do is really see if we can pull together a joint team to start a project, right from the very start, because sometimes what happens is if something's developed in one of the other audit agencies, and then we all come in quite late, it's been done in a way that's quite specific to that audit agency and it gets a little bit harder to share and unpeel some of that. So what we're trying to do is say, actually, if we all come together right at the very start, have a joint team that's jointly resourced. Will that work? Can we get something that then is absolutely just built through any of the audit agencies to use? So I think there's been a real focus, on that collaboration. Certainly we wouldn't be where we are without working that closely with the other audit agencies. Absolutely. I think that's been something that's been so helpful to us. and hopefully it's the same for the other audit agencies as well.
Conor:So that joint team concept, that's really interesting from a couple of perspectives. So when you say joint team, do you mean a joint team of individuals from audit offices working on the same audit per se, like the exact same audit or is it they all do the same audit in their own jurisdiction?
Gemma:So it's a joint team to develop a tool or to look at a process and how we look at something and to then say, actually, how can we do this differently? What technology have we got that could make this process easier? So in that way, it's not actually an audit, it's a team to look at a certain process essentially and see about, how can we make this better? How can we use this data, pull it in, analyze it, push it out in an analyzed form or get people to analyze it? How can we best do that? So it's a little bit different to an audit, it's at really early stages, but it's a bit of a proof of concept to see if we can make it happen. And, because, I think we all see that there would be huge benefits, from that, we've all got, as I say, limited skills in house, we've all got quite limited investment that we can, put to it as well. We are essentially, audit agencies, these skills are quite limited within the organization. So we definitely see that there's more benefit to have by bringing it together.
Conor:That's a really common sense approach, sharing insights and ideas and challenges that probably all the audit offices are having. And just getting efficiencies in terms of utilization of your resources. But it really goes to the focus on innovation and saying that, our audit office doesn't have all the answers, but we probably have the same problems. There's something really helpful and exciting there in that.
Gemma:We absolutely don't have all the answers. We know that, it would be madness for us to sit there and say, yeah, We got this, we've got everything we need. We know exactly where we're going. We could tell you in a year's time, 18 months, exactly what it's going to look like. No way. Do you know We absolutely know where we want to be. We've got our ambitions and we know how we're going to try and get there. But, what we've learned, of anything over the last few years is that no road is straight in this area, you'll start down a path and you'll hit upon something that you just, either can't get round or it's going to be much more difficult or you try something else. And it works brilliantly and there's lessons to be learned from some things. So I think we know that actually, you've just got to work in. quite short bursts. So we've just recently I suppose, reset our vision and our ambition for this to say where we want to be and, what we want to do and think have your eye on that ambition, but it's going to change quite dramatically, probably how you're going to get there. over the course of next few years, we don't know what technology is going to be available. We don't know what new software is going to be there in a year's time. What we find at the moment is that. the market. Just. Isn't providing the tools that are exactly what we need, so that's why there's been quite a lot of in-house development of tools to get what we need. But I think there'll be such a demand for it from the public sector audit agencies that, that market's going to start to change and be able to have some of those tools. We can see that maybe potentially starting to change and this, again, some work with the other audit agencies through some Innovation gov tech schemes that the UK government has got the, trying to build that and look at that market. So again, that's an area thing that we can see changing That might just shift that focus between how much we have to do ourselves into what we can actually just buy in off the shelf.
Yusuf:You've been involved in upskilling and bringing resources in that have particular programming, language skills. have you considered some of the, tech that's available. that Allows you to not have to code as well. there's a range of graphic user interface point and click tools, not only on the visualization side, but also on the, ingest and analysis and blending side that enable you to get there faster without having to code directly.
Morag:We've used, in terms of the visualization stuff obviously Tableau we've used before and Power BI because we use Office 365 as well. We've just started looking at that, but again, there's, a skills thing there to start to getting people used to using that, but that certainly something that we're looking at just now. On that front, we're thinking about a bit of a hybrid approach, probably because there'll be some types of data analysis that likes of Power BI would be fine for, but the other stuff the likes of RShiny we're finding has been really good. That's a large bit where we've been up-skilling and in terms of the data ingestion, because that's obviously consumes a huge amount of time trying to do the ETL on all the data that we bring in as well. so we are looking out there and again, working with our UK audit colleagues as well to look at what they've been trialing and, having conversations with suppliers around that to just see what's available. And the govtech project that Gemma mentioned that's another avenue as well, because through our colleagues in Northern Ireland working with some suppliers and that's more on the sort of financial audit side, part of that project is looking at how we ingest financial data as well, because that might become part of the tool. But that is actually a key one for us, to be looking at that ingestion, because it's just such an overhead to bring all that data in.
Conor:One of the observations we'd have around those audit agencies that have had a focus on data for the past few years. And this is our terminology not theirs. Is they've become a victim of their own success in that they've done some audits, they've done some fantastic analysis, they've produced their performance audit report, and then the entities have come back and said, we really love that. Can we have all the data that you've cleansed and analyzed for us and put out there? And also, can you do this for us and analyze this for us? Have you had any of those scenarios where you've seen real growth and appetite from the audited entities in terms of wanting to piggyback off some of the work your office has done to try and understand their own work better?
Gemma:That's really interesting question, actually. And, there's always been audits where we have. gathered information that the audited bodies have not been aware of or put it together in a way and analyzed it in a way with other information that they've not, and in a way, presented it in such a way to kind of say, we really think you should be doing this. Cause I think it's always a position for us in terms of what is our role as auditors, And I think there's a really important thing and we were actually just having this conversation, the day in terms of, we really need to be clear about what's our role on the data. So are we the ones who collect the data, holds the data, analyzes the data and makes it available to public? Is that what the audit role is on this? And I think there's exposed as risks and opportunities and benefits to all of that. And I think there is something around about if you're holding the data and becoming responsible for the data, there's an awful task there in making sure that the quality of the data is right in that cleansing of the data. It can be interpreted in the ways that there's always so many caveats around data. and that, so I think there is a huge responsibility that if you are holding or become the holder of the data, it's something that we are having ongoing discussions about is actually what's the role for us around about this and where should we be? And I think because we've seen over the last couple of years, this move where, Public bodies themselves are being more transparent about their data, are making the dataset slightly more available, having an APIs that we can connect up to in that thing that actually made potentially that need for us to hold it all. Isn't the same. And that we can just be connector essentially between some of those datasets. But we are thinking about how do we become part of that conversation in terms of trying to get public bodies, to be more open and transparent about their data to improve their data sets. so that there isn't such a role for us to be that holder of data. So I think it's a really interesting one. And I think it's probably one that we will continue to grapple with, for a little while. And there's a range of positions that you can take. And it might be that we want to hold data for some areas, but we don't need to do it for others, but it does for me raise a lot of questions about who then is responsible for that data and the investment that we would need to put in as an organization to making sure that it was of sufficient quality to be used.
Yusuf:some audit offices take the view that if they collect proprietary data for a particular audit or a particular purpose, that they shouldn't use it for any other purpose, Have you had that situation and how do you deal with that?
Gemma:Like any other agency, you go immediately to the open data and the public data first and use that where best you can. Our recommendation in every single audit report we do is that public bodies need to get better at monitoring their performance information and their outcomes, data that is in every single audit report we do, it's just not a strong enough principle yet in the public sector that they are good at monitoring. long-term performance and outcome data trends. There's always something for us around how often do we need to invest in this dataset? So if we need to create it new each time, that's quite a big burden. There might be certain areas that we want to do that. So for example, looking at educational outcomes, done quite a lot of audit work in that area. And every time our expectation is that the audited bodies themselves have a look and say, what information do we need to be able to monitor our outcomes? How do we know if we are making a difference or not? How do we know if these actions are having the impact that we want? That's something that we just need to continue pushing on because it's certainly not mature enough yet. The ambition's there, there is absolutely a real ambition around outcomes, in the public sector in Scotland, but just that systematic collection of data to monitor those outcomes is not yet there.
Conor:Using data to evaluate outcomes - it's a fascinating conversation and it hasn't really moved a long way in the past couple of years, generally across the board because it's a difficult one.
Gemma:I recognize where we've come on this but we are still at the start of the journey. And we have huge ambitions, but it's hard for those of us that work in it, we absolutely love it. It's fascinating. We can absolutely see what the benefit is going to be. And for us, with our management team within Audit Scotland, really bought into this and really want to see it. There's so much you want to do there's so much you could do, do you know, and that trying to say, actually, where do we focus? What are our priorities? If we've only got a limited skills and money, what's the one thing we would want to put that to. It's because we are enthusiasts. We want to do it all, but trying to focus on where can we have the most benefit right now with the skills and the money that we have right now, and keeping that longer term ambition, in mind. So we're very much in that very much that in that learning phase, I would say, working with others, trying to do our best, on your own, but absolutely still learning.
Morag:It's difficult and we are just starting out and it's been a challenging year, obviously, with everything that's happened with Covid, and we've been trying to create this central team and bring all that together as well as part of that. And then the added pressure of all the additional audit work we've had to do, or that audit work's taking longer just because of the way we're working. So trying to balance all that, but actually in a strange way a bit of an opportunity that's being created is through the Covid pandemic because it's shone a light on the importance of data and how that can really add value and have impact. And how we can work differently as well I should say. and that's helped get that wider buy in, across the organization. Because we've been able to develop some tools really quickly, people have seen what can be done and how that can, help with the insights that we can provide, then that will help take us forward.
Conor:Great conversation ladies, thank you so much. The takeaway for me was having a pool of data pirates, but I think Morag we might have to make you walk the plank on that one. Thank you Audit Scotland, Morag Campsie and Gemma Diamond been a fantastic conversation.
Yusuf:Thank you.
Gemma and Morag:Great. Thank you very much.
Narrator:If you enjoyed this podcast, please share with a friend and rate us in your podcast app. For immediate notification of new episodes, you can subscribe at assuranceshow.com. The link is in the show notes.
.jpg)
.jpg)
