48. Performance Audit Mini-Series #1 Forward Work Planning

This is the first of a series of episodes that focus on performance auditing.

In this episode, we discuss strategic audit planning (a.k.a. forward work planning).

For internal auditors, this is similar to annual (or equivalent) audit planning.

1. Understanding your community / public expectations
2. Understanding your elected body / parliamentarians
3. Understanding the entities within your audit office's jurisdiction

About this podcast
The podcast for performance auditors and internal auditors that use (or want to use) data.
Hosted by Conor McGarrity and Yusuf Moolla.
Produced by Risk Insights (riskinsights.com.au).

Transcript

Narrator: 0:07

You're listening to The Assurance Show. The podcast for performance auditors and internal auditors that focuses on data and risk. Your hosts are Conor McGarrity and Yusuf Moolla.

Conor: 0:20

Today we're going to kick off the first in a little series about performance auditing. Auditors general are required usually to prepare a forward work program. Generally most of them we see cover a period of three years, that sets out the topics that they are going to be addressing in terms of their performance audit mandate. So today we just want to talk about some of the inputs and considerations that go into developing that forward work program.

Yusuf: 0:45

This may be of relevance to both performance auditors and internal auditors. Strategic audit planning in the context of performance audit is interesting as a topic by itself, but also interesting to internal auditors who also would need to do annual planning or shorter range planning. Just to understand how that works in the performance audit world and what they could potentially take from it.

Conor: 1:06

Forward work programs are significant documents. Take quite a bit of time and cost to prepare because they have that sort of long-term view. So in preparing them, it's essential that you're doing proper scanning of the environment in which you're operating to try and understand what are the current or even emerging risks to delivery of public services. To be able to do that effectively, you need to look at three broad areas. The first thing you need to understand the community, what's happening within the jurisdiction in which you're operating, what is important to those people, and what services do they use. Secondly, you need to understand your elected officials. So whether that'd be your Parliament or your Congress. What are they talking about, what's important to them, and how can you tap into that to make sure that what you're going to address in your work plan is as relevant as possible. And then thirdly, it goes without out saying, you need to understand the entities. Those departments or local governments or utilities, how well do we know them. So let's start with the community. This is perhaps the least well done of the three aspects we just described. Obviously you can't send a survey to every member of the public. But what you can do nowadays is scan their reactions as individuals to things like what's happening in the media. What are their views on certain policies of the government or new laws or any decisions being made that impacts them? And the practical way to go about some of this is scanning the volume of activity or things like online blogs responses to newspaper websites, or media articles. What are people getting hot under the collar about? What do they seem to be passionate about? This activity and scanning these responses can give you a good steer about what's most relevant to them. The tools these days to be able to do that are manifold and quite readily available. And even things like sentiment analysis can be pretty useful in trying to understand what members of the community are concerned about.

Yusuf: 2:58

In doing that, you mentioned reactions to policy or views government policy. And many auditors general mandates don't extend to evaluating government policy. So government policy is, a given, and then you evaluate the programs that are developed under that policy, the administration of those, efficiency of the delivery and then effectiveness of the outcomes of those programs. So how do you differentiate between understanding policy and evaluating policy?

Conor: 3:28

So the first thing is that m ost legislation that sets up audit offices and gives that statutory role to auditors general have explicit statements in there to say the auditor general cannot evaluate the merits of policy, which is exactly what you said. It's not the role of an an audit office to try and second guess or undermine or to call into question the elected decision making body and what they've taken forward through the parliament or their Congress,. They have to evaluate the implementation of that policy. So your question comes back to how do you draw that distinction or that line? It's not that easy to do all of the time. Knowing that we're only evaluating implementation, the main consideration you need to have, is that the questions we are asking as performance auditors, are they things that can be answered by the public servants delivering implementation of the policy? Or is it something they simply can't answer because they weren't part of the decision-making framework in the first instance.

Yusuf: 4:31

At what level does that restriction apply? So you've got the merits of overall government policy, and then you'd have, for example, within a government department, in order to be able to implement certain practices, they would have policies that they create.

Conor: 4:44

That's a different thing. So those policies relate to the proper functioning of that organization, as opposed to implementing the wishes of the elected government.

Yusuf: 4:55

What performance auditors can do is quite extensive, but there is a limitation around that overall high level evaluation of the merit of government policy.

Conor: 5:05

the government might make a decision that for every baby born this year, we're gonna give the parents a thousand dollars. Now, an auditor general general, may not think that's an appropriate use of money, but if that's a policy that's been implemented by the government, then an auditor general has no right to question the merits of that policy.

Yusuf: 5:23

And Now we've had to look at what the community want, what do we do next?

Conor: 5:27

The second thing, we need to have a consideration of. The parliament or Congress and those elected officials. So whether we like it or not what they say matters for each and every one of us, because they impact our lives on a day-to-day basis. So we need to understand what they're talking about. So one simple, but very powerful way to do this is by analyzing the official record of what has been talked about in the parliament or in the Congress. Text analytics techniques can really help with this. But the other thing to bear in mind is that audit offices have an educative role. As well as an evaluative role. So, the more you can understand what is important to those people the better equipped you'll be to select your audits for your work program and provide information to your parliament.

Yusuf: 6:15

What are the sorts of things that you would look at that are important to them that the audit office can actually play a role in, what does that look like in real terms?

Conor: 6:21

If we take the official record and Hansard as we call it here in our jurisdiction of Queensland Australia. Everything is recorded, documented, and then published for members of the public. What we should be looking for are common themes that are being discussed. The volume of certain words. Topics or things that are important and could be about to be voted on, or are currently being debated. These are gonna become part of our day-to-day lives. So if an audit office can get a handle on what's being discussed what those topics are, why it's important. And even more so perhaps understand if there are any competing arguments about those individual topics, then that gives you really great insight into what's important and what you might want to plan for auditing in the future.

Yusuf: 7:05

You want to be responsive to new issues that are emerging, but a lot of the performance audit work is around evaluation of, outcomes. which there is a reasonable time lag between the starting point for which the methods are discussed and the actual implementation of the programs. So, how do you determine what right timeframes are?

Conor: 7:23

You could either look back to the commencement of this term of the current government. In most Westminster democracies, that's a three or a four year term. It's from then on that, the government of the day will be trying to actually get its policies implemented. So that's a reasonable timeframe to go back and start from.

Yusuf: 7:43

Alright what next.

Conor: 7:44

Thirdly, we said you really need to understand the entities you're auditing. There's lots of this valuable information locked up files, in colleagues heads. Experience would suggest that sometimes this can be overlooked and we don't place us as much emphasis on that as perhaps we need to. We're really doing everybody a disservice if we're not interrogating our own systems and in-house experts.

Yusuf: 8:06

What else is there that comes in from either the outside or the inside that is a little bit more specific then the broad, scanning that we do?

Conor: 8:14

An important thing to talk about here is the proliferation of open data. Many public sector organizations and peak bodies are putting up valuable data sets and resources and reports. One example, and this would apply to all jurisdictions regardless where you are in the world. Governments pay a lot of money on road infrastructure, both in developing it, building it, but also in maintaining it there's significant data holdings publicly available in almost every jurisdiction now that, talk about road length, maintenance schedules, capital budgets, all sorts of valuable information that you can basically get within a couple of clicks of the mouse.

Yusuf: 8:55

That all sounds good, but it does sound quite extensive. Without trivializing the value that is obtained by, rigorous analysis and research and evaluation to come up with a forward work plan. What is reasonable in terms effort toactually come up with that work plan.

Conor: 9:12

Perhaps if I reflect the question back in terms of how important are these programs, to the public and how important do we take our job because if we're doing a really important job for the community, as we are as performance auditors, and we know that potentially how we look at things and what we find will impact their lives, then we need to ensure that we are resourcing that appropriately and not just set and forget. We need to have the resources working in the background to make sure that those topics we're taking forward are the most relevant ones. And we're doing them in the right way, at the right time.

Yusuf: 9:50

Often in these forward work plans, you'd see a high level scope. of work Now, when the work actually then starts on a particular performance audit or, a particular topic, you would then do more detailed scoping and planning for that audit. How do you draw the line between the initial scope that is determined and the more detailed scope that is, figured out later on.

Conor: 10:13

You've probably done enough work at the initial broad scoping stage, just to get something into your program, if you can clearly articulate the potential risks that you're trying to address, and you need some sort of evidentiary base. So, you can't just pluck it out of thin air and say, I think this is important. Let's do this. As soon as you've got that reasonable evidentiary base as to why this is a problem or why this is important to look at this particular topic, you've done enough at that stage. Now, quite often that could involve some preliminary data analysis. And in fact, we'd suggest that that happens even as part of forward work program. But try not to get too bogged down in the detail of the analysis, particularly if it's an outer year project or topic, because invariably there will be changes over time. So you're possibly wasting your effort and resources if you get into too much of that detailed analysis of performance at the early stage, as opposed to just trying to understand broadly the risks that exist.

Yusuf: 11:15

At the outset when creating the forward work plan, do you set what the objectives of the audit are going to be?

Conor: 11:21

Yes

Yusuf: 11:22

So the risks will drive the objective.

Conor: 11:24

The risks to efficient or effective or compliant service delivery should drive the objective of the audit, even when you're at that early stage.

Yusuf: 11:34

Okay. And then do you later on maybe change that and say, you're not going to look at effectiveness here, we're going to look at efficiency instead.

Conor: 11:41

Absolutely. The way in which that service is being delivered, so the mechanisms for delivery, they may have changed. There may be new laws or regulations have come out in an intervening period. There may have been changes in funding, all sorts of considerations like that. So they are the types of things that then could change what you look at.

Yusuf: 12:01

Internal audit teams, in developing their plans, often it's not just about the risk, but it's also about the understanding of what the controls are in place. So how well something is being done. Do you do that within your forward work plan as well so understand the risks, but also you would, through your understanding of those departments. Say if you had a department that, you know, just executes very well, would you maybe consider not looking at that because there may be others that you can focus your attention.

Conor: 12:31

So you've just encapsulated there, everything we've been talking about. When you're selecting topics for your program, you have to use all the intelligence available to you. So if that intelligence suggests that department or agency X is a very schmick, well honed service delivery agency, then absolutely you'd have to use that when determining do I need to deprioritise looking at them or deprioritize looking at particular aspects of their functioning. Using that collective intelligence to make that sort of determination. And you can do that by speaking to your own internal staff who know that agency inside out, which goes to the point we tried to make earlier that sometimes that doesn't happen as easily as it should.

Yusuf: 13:18

So I want to just draw a few parallels between internal audit and what you explained there before asking one last question. The parallels are you would need to understand the executives in your organization. So that's second point you made, senior decision makers and what they are interested in and what their focus is. A lot of times that will be driven by strategy. The equivalent to really understanding what the community wants, because not the same within, a non-public sector entity. Obviously within the public sector, if you're an internal audit team within the public sector, then understanding what recipients of your service are, or want. So if you're, in education, for example, that would be parents and teachers and principals, and obviously kids in schools what exactly they want. But when we're talking about non-public sector entities, It could be the voice of your customer potentially. So you could want to understand what your customers want and need, but quite often, that's not going to give you enough to be able to understand what the audit plan needs to be. So you would want to go to strategy, obviously. And in an ideal world you'll tie them up. So what is the customer saying? What do the customers want? And then what is my strategy say? And where am I going? And then the last thing you mentioned was understanding the, agencies. So knowing how well the agencies actually operate. Within the internal audit world, that would be understanding the different departments or functions within your organization and how they operate and how well they operate. But the last question is, and this is relevant again to internal audit. Within the performance audit teams, when you're developing your forward work plan, to what extent do you look at topics or matters that other jurisdictions are looking at? So, your peers the reports that they've produced or what's in their forward work plans to what extent does that influence your forward work plan?

Conor: 15:01

Looking at sister organizations or different jurisdictions and what they've done or haven't done, should form part of the research component to your forward work plan. If, for example, you were considering undertaking an audit on a topic that had been done in another jurisdiction but they didn't. have major findings or, conversely they could have had major adverse findings, then that needs to flow into how important that issue is here. So we need to learn from others, basically, as part of the research. As to whether we would or wouldn't do a topic simply because it had been done in another jurisdiction, I don't think that's a useful mindset to have.

Yusuf: 15:41

So when you were working internally within an audit office , and developing those,, strategic audit plans as they were called back then, to what extent did that external scanning affect your program.

Conor: 15:51

It was helpful to understand what others were doing. But I found it far more helpful where others hadn't done anything on a topic, but it had through our environmental scanning emerged as a real risk area in service delivery. And then when you look out you say, oh, no other audit offices have tackle this or approached this I found that more interesting.

Yusuf: 16:13

Within internal audit, organizations have been moving towards consolidating the audit plan into fewer topics overall. So instead of trying to say, we're going to cover 36 topics this year, they're going down to, let's have that and see if we can do big audits. So even down to a third and do, even bigger audits than that.

Conor: 16:33

That's a really interesting point Yusuf. So these days some audit offices seem to be taking more of a thematic view. So they may say, over the next three years, one of our major streams of work is going to be in the digital space. And they may do multiple audits in each of the next three years that has a digital aspect to it. Potentially, we may see more of that in the future.

Yusuf: 16:57

In summary, forward work plans, strategic audit plans, it doesn't matter what you call it, basically your plan for the next few years for conducting performance audits. Three key things to consider the first is what is the community saying? Or what are the community want? What is the sentiment within the community around the services that are being delivered or that are planned for. The second is understanding the individuals that are running the show. And the third is understanding those departments and agencies and, thinking about all of the locked up information that we have internally that we can use. All of this would be useful for internal auditors to consider as well in how they develop their plans. Are you thinking about those various factors in coming up with your plan to deliver the best value that you can? Thanks Conor.

Conor: 17:42

Thanks Yusuf.

Narrator: 17:43

If you enjoyed this podcast, please share with a friend and rate us in your podcast app. For immediate notification of new episodes, you can subscribe at assuranceshow.com. The link is in the show notes.