Nonprofit Contract Performance Management

Audit ATX speaks with Auditor Ashley Rodriguez about how the City manages nonprofit contracts for family and social services.

Transcript available on Buzzsprout: https://www.buzzsprout.com/1419979/19616482-nonprofit-contract-performance-management/transcript

Transcript

Sam: Welcome audit enthusiasts and fellow Austinites to another episode of Audit ATX. 

Maxx: The podcast that dives deep into audits and investigations our office does to make Austin better for all residents. 

Sam: We're your hosts: Sam 

Maxx: and Maxx. Welcome back to Audit ATX, y'all. Today we're here with auditor and behind-the-scenes podcast manager Ashley Rodriguez to talk about a recent audit our office conducted. In this audit, Ashley and her team looked into the City's management of nonprofit contracts. Ashley, it's great to have you here on the mic on Audit ATX. 

Ashley: Thanks, Maxx. Great to be here. 

Sam: Echoing Maxx, it's nice to have you on this end of the podcast, Ashley. To kick us off here, could you tell our listeners about what the objective of this audit was and why it was conducted? 

Ashley: Yes, of course. The objective of this audit was to determine whether the City is effectively and efficiently writing and managing nonprofit contracts for family and social services. In fiscal year 25, the City spent about $100,000,000 with 95 different contract agencies that provide these types of services. These are services like mental health resources and permanent supportive housing, for example. 

The stakes are quite high here, as not only are these contracts a large city investment, but they are also an investment in vulnerable Austinites who need these supports. This audit was done to assess whether the city is doing its part to ensure contracts are being met, and therefore residents are being served. 

Maxx: All right. Well, it sounds like we have a lot to talk about. But before we dive into this completely, Ashley, what are some important pieces of context that you think, Sam, myself, and our listeners really need to know to best grasp y'all's work here? 

Ashley: It's important to note that nonprofit contracts for family and social services are spread among several City departments. This report looked at contracts within Austin Development Services, Public Health, Community Court, Economic Development, EMS, Housing, Homeless Strategies, and Management Services. 

I also want to give a picture of what exactly we mean when we talk about performance expectations for these nonprofits. There are two types of expectations that a nonprofit may be expected to provide. The first type is an output. This measures how many services are provided. There is also an outcome, and this measures whether the services provided create a positive impact. Both expectation types are important to ensure that not only are nonprofits reaching Austinites, but that services provided increase quality of life. 

Sam: Thanks for providing all that context for us. Definitely recognize that there are a number of players involved in this ecosystem, and that there are multiple lenses to evaluate performance. Can you walk us through the steps that the audit team took to complete the work? 

Ashley: There were quite a bit of steps done as part of this audit. Some of the steps we took included looking at the actual expectations the City requires to determine if the expectations were specific, measurable, and time-based. We also requested and assessed data submitted by nonprofits to the City that stated their performance. The last step was actually going to some of the nonprofits we looked at to see if the performance they reported to the City was supported by documentation. During these site visits we also gave nonprofits the opportunity to share their perspective on contracting practices within the City. 

Maxx: So can you tell us what the sample looked like for this work? I mean, when I'm thinking about it, surely your team didn't go through every contract and every nonprofit this city has contracts with, right? 

Ashley: You are completely right. I want to make it clear that the goal of this audit was not to audit our use of nonprofits. We are auditors for the City, and our interest is in assessing and improving City processes to ensure residents get the intended services. With that in mind, we just needed a large enough sample to let us know how all the City departments I mentioned earlier are handling these contracts to assess if expectations were met. We sampled 25 contracts in total. 23 of these contracts were randomly selected to represent each department in proportion to the number of contracts the department holds. The other two contracts in the sample were added during the audit. And that's because they were brought to our office and they were considered high-risk, which we could not ignore for our audit process. 

For these reasons, one being the purpose of our work to focus on the City, and two, the fact that our sample size actually cannot be projected to the entire population, we did not name nonprofits in the report. 

Sam: I see that there's one finding in this report. Can you go over what it entailed? 

Ashley: The finding for this audit was that the City does not ensure that nonprofits are consistently providing agreed-upon services. As a result, vulnerable residents may not get expected services and may have their needs unmet. 

Maxx: Gotcha. And what evidence did you end up finding that supports the finding? 

Ashley: Where we found issues was in the contract management area. Based on the performance reports collected for each of the contracts, 19 of the 25 contracts -- or 76% of the sample -- did not meet one or more of the performance expectations. If we want to look at this a different way, we can also look at this in terms of expectations not met. In this case, we would say that of 151 individual expectations, 67 -- or 44% percent of the sample -- were missed. 

Sam: So it seems that several nonprofits in the sample were not hitting their performance expectations. What does this suggest regarding the City's contract management practices? 

Ashley: Generally, I would say that the City does not have processes to incentivize nonprofits to hit these targets. Let me explain a little bit more about what I mean. 

It all starts with the nature of the contracts. These contracts are cost reimbursement-based, which means that the city pays nonprofits based on what the nonprofit spends, rather than paying based on whether performance was met or not. 

We also saw that the actual monitoring of the contracts is not effective. Although nonprofits must explain why they did not meet goals, there are no consistent corrective actions in place to encourage better performance. 

The last piece of this is that when contracts are closed, there are no meaningful assessments of the work done to inform future contracting decisions. 

We also heard from nonprofits during our visits that they had concerns about the City's management of these contracts as well. Nonprofits shared concerns about the performance expectations they are required to track, the reporting process, and the clarity of city staff's communication. 

Maxx: I appreciate the explanation, and I can definitely see that the city could be doing more to ensure that these contracts are meeting their intended purpose. Based on these findings, what recommendations were actually made for the City to improve here? 

Ashley: You make a great point, Maxx. And this is exactly what we want people to take away. We want the focus of this work to be on the City's processes and what the City could do to ensure social services the City has invested in reach Austinites who need them. Recommendations are definitely in this spirit. 

We have two recommendations attached to the audit. The first is to implement a post-contract performance evaluation process for nonprofits. And the second is to evaluate and strengthen the City's family and social services contracting model. This should include the implementation of ways to encourage nonprofits to meet performance expectations and provide quality service. 

Management agreed with these recommendations and have a few initiatives to implement them, including establishing clear criteria and guidance, and centralizing appropriate functions within lead departments, including Austin Financial Services and Government Relations. 

These recommendations are expected to be satisfied in 2027. 

Sam: Well, Ashley, thanks for taking the time to come on the podcast today to go over this report with us. We look forward to having you back sometime soon. 

Ashley: Thanks for having me. 

Maxx: Thanks for listening to this episode of ATX. For more information about this report, you can visit our website at austintexas.gov/auditor or shoot us an email at austinauditor@austintexas.gov. 

Sam: Or follow us on Facebook, Instagram and X @AustinAuditor and subscribe on Spotify or Apple Podcast.