In 2021-22, the state provided $406.3 million in dedicated funding for school districts to deliver additional services to students at-risk of academic failure. State law requires that district spend money from their at-risk fund only on programs approved by the State Board of Education. Kansas Department of Education (KSDE) officials told us the State Board has delegated the task of approving at-risk programs to the department. None of the KSDE-approved programs we reviewed met the statutory criteria necessary to be included on the list. Further, most of the approved programs we reviewed had little to no evidence of effectiveness. Additionally, many KSDE-approved programs did not appear to meet the purpose of at-risk programs, which is to provide above and beyond opportunities to at-risk students. The problems we identified with KSDE's approved at-risk list are the result of several factors. This includes things such as, KSDE's process for approving at-risk programs does not include some statutorily required criteria, the department does not follow it's own process, and the board does not provide any oversight to the department. This audit showed the same problems as in our 2019 at-risk audit, and none of our recommendations have been adequately implemented.
Most of the $176 million in at-risk expenditures we reviewed for 20 districts was spent on salaries and benefits. About 30% of the $5.2 million in expenditures we reviewed did not adhere to statutory spending rules. Further, it is unclear how much of the districts' at-risk spending targets at-risk students or provides them an "above and beyond" opportunity. Problems with the at-risk spending guidance KSDE provided to school districts may contribute to some of the unallowable spending we found.
Over the last 6 years, students eligible for free lunch have consistently performed worse on state assessments than students who are not eligible for free lunches. All 3 cohort groups we evaluated performed worse over time, but this trend was more pronounced for students eligible for free lunch. Additionally, the graduation rates and ACT scores of students eligible for free lunch were also lower than other students.
In 2021-22, the state provided $406.3 million in dedicated funding for school districts to deliver additional services to students at-risk of academic failure. State law requires that district spend money from their at-risk fund only on programs approved by the State Board of Education. Kansas Department of Education (KSDE) officials told us the State Board has delegated the task of approving at-risk programs to the department. None of the KSDE-approved programs we reviewed met the statutory criteria necessary to be included on the list. Further, most of the approved programs we reviewed had little to no evidence of effectiveness. Additionally, many KSDE-approved programs did not appear to meet the purpose of at-risk programs, which is to provide above and beyond opportunities to at-risk students. The problems we identified with KSDE's approved at-risk list are the result of several factors. This includes things such as, KSDE's process for approving at-risk programs does not include some statutorily required criteria, the department does not follow it's own process, and the board does not provide any oversight to the department. This audit showed the same problems as in our 2019 at-risk audit, and none of our recommendations have been adequately implemented.
Most of the $176 million in at-risk expenditures we reviewed for 20 districts was spent on salaries and benefits. About 30% of the $5.2 million in expenditures we reviewed did not adhere to statutory spending rules. Further, it is unclear how much of the districts' at-risk spending targets at-risk students or provides them an "above and beyond" opportunity. Problems with the at-risk spending guidance KSDE provided to school districts may contribute to some of the unallowable spending we found.
Over the last 6 years, students eligible for free lunch have consistently performed worse on state assessments than students who are not eligible for free lunches. All 3 cohort groups we evaluated performed worse over time, but this trend was more pronounced for students eligible for free lunch. Additionally, the graduation rates and ACT scores of students eligible for free lunch were also lower than other students.
Welcome to the Rundown, your source for the latest news and updates from the Kansas Legislative Division of Post Audit featuring l p A staff, talking about recently released audit reports and discussing their main findings, key takeaways and why it matters. I'm Morrie Xly . In July, 2023, legislative post audit released a performance audit that evaluated at-risk expenditures and statutory compliance. I'm with Heidi Zimmerman, principal auditor at Legislative Post Audit , who supervised the audit. Heidi, welcome to the rundown.
Speaker 2:Thanks, Maury .
Speaker 1:So to get started, can you give me some background about at-risk funding and how districts can spend those funds?
Speaker 2:Sure. So the state provides at-risk funding to school districts to help districts provide additional services to students who are at risk of academic failure. So at-risk funding is based on the number of students in the district who are eligible for free lunch under the National School Lunch Program. Uh, those kids also have to be under the age of 20 and full-time as well to qualify for that funding. But districts with greater than 35% of their students who are eligible for free lunch also get a little extra funding called high density at-risk funding. For the, for this podcast, we're gonna refer to both of them as at-risk , uh, but in 2122, the school districts received about 406 million in at-risk funds and statute directs how districts can spend , uh, funds from, from their at-risk fund. So first statute requires that the State Board of Education create a list of approved at-risk programs, and then the districts can spend on at-risk programs , uh, that the board has approved , uh, as well as educational personnel training and contracted services related to those approved programs.
Speaker 1:So what types of things is the board supposed to consider when deciding what programs to approve?
Speaker 2:Well, first of all, the board and the Kansas Department of Education, or K S D E , um, those officials told us that the board has delegated the responsibility of creating a list of, of approved at-risk programs to K S D E. So it is the department that creates that list. Uh, statute requires that programs on that list meet three criteria. So first, the program must be for at-risk programs and at-risk students. Second, the research behind the program must be peer reviewed. And then third, that peer reviewed research must show that the program per produces better outcomes over a five-year period than would otherwise be achieved. So programs on that list really need to check all three of those boxes.
Speaker 1:So it looks like you reviewed that approved list from several angles. What did you find?
Speaker 2:So we looked at the list in three different ways. So first, we reviewed a selection of programs that are on the list to see if those programs met statutory criteria. So we chose 40 , uh, some were chosen randomly and some were chosen because we saw that districts were using them, but because we didn't choose them all randomly, our results here are not projectable, but we look to see if those 40 programs checked all kind of three of those boxes that I just talked about. So it needs to be for at risk , the research needs to be peer reviewed and it needs to produce better outcomes over five years. And what we found was that none of the 40 programs we reviewed met all three of those criteria. The second, though, we found something interesting in that work that made us want , that made us want to look at these programs from a different perspective. So most education research that we saw does not follow students for five years. It's usually for a much shorter period of time a year, occasionally two, but very rarely for five years. So, statute says that approved programs should produce better outcomes over that five-year period , but that is a pretty high bar for education research. Um, and so we, we looked kind of a , again, at those 40 programs to see , uh, does , does peer review research show effectiveness for those programs, even if it's not over five years? So when we looked at those 40 programs, again, what we found was that for more than half of them, or 22 of them , we couldn't find any peer-reviewed research at all for them. So only seven of the 40 had moderate to strong evidence , uh, for effectiveness. And the remainder of them, the research showed either weak or mixed results for effectiveness. So as a whole, the programs we reviewed , uh, did not have good evidence supporting effectiveness. We also looked , uh, in one more way. So the state statute says the purpose of at-risk programs is to provide opportunities or interventions that are above and beyond regular education. And that term above and beyond is what's in statute. So based on that, we expected programs on the list to provide some type of above and beyond service. Uh, but we found that a few kind of categories of programs that we saw , uh, just don't really meet that expectation. So first , uh, many of the approved programs are just general opportunities that are available to all students. Things like career , technical education, virtual schools, dual credit courses , um, all of those programs were on the approval list. We also found that some were general teaching strategies like project-based learning , uh, which may be a great way to teach, but it is a general strategy for teaching. We also saw that there were several assessment tools. So for example, like FastBridge , uh, which again is a , which are tools used for all students. We saw a number of districts using those sorts of tools, and they use them to track all students , uh, their progress over the course of the school year. Uh , but those things don't really appear to provide an above and beyond opportunity to students. So we talked to the department , uh, about these things, and the Department of Education agreed that at least some of the programs on this list probably are not appropriate. Uh, they told us they are revising the list and they will have a new list ready for the 24 25 school year. Um, and they told us that some of their programs on the current list probably will not be on the next one .
Speaker 1:So your report mentions that the issues that you found were the result of several factors. Would you speak to what some of those factors were?
Speaker 2:Sure. So we, we did kind of go looking for, for reasons for why we saw some of these issues, and we did find a few things. So first , uh, the Department of Education , their process for identifying programs for the list does not require that the programs are actually for at-risk students. So officials told us that as long as some aspect of the program seems to work for at-risk kids, then that was enough to include it. Uh, we also found that department staff don't follow some of their own rules for ensuring that research is peer reviewed. Uh, they told us it was part of the process, but we found that a majority of the programs we looked at , uh, did not have peer reviewed research. So , uh, they don't appear to be following , uh, a rule. They kind of set themselves , uh, last, the board doesn't appear to provide any oversight. Um, as I mentioned earlier, the list , uh, creation has been kind of delegated to Department of Education. Um, one of the board members we talked to told us that they've done that because many state board of education members lack education backgrounds, and so they rely on the Department of Education's exp expertise. Uh, the department told us that they update the board about that list, but we couldn't find any evidence that the board has been updated , uh, in the last four years.
Speaker 1:Your report also looked at how school districts spend money from their at-risk funds. What did you find?
Speaker 2:We looked at 20 school districts. Uh, their , we looked at their accounting records , uh, for their at-risk fund to make sure we had a good understanding of how they spent their money. Those two school districts provided a good cross section of school districts, but they were not chosen randomly. So, so our results are not projectable. Uh, but those 20 districts spent 176 million from their at-risk fund in the 2122 school year. Uh, 97% of, of that money was spent on salaries and benefits. Uh, that other 3% was mostly for things like professional services and supplies. Uh, but because nearly all of the money that was spent was for paying staff, we looked a little closer to see, you know, who those staff were. So of the 5,200 personnel that districts reported to us that they paid for , uh, from their at-risk fund, 90% of those people were instructional staff , uh, mostly teachers and paras, but there were also some substitute teachers in there as well. Uh, most of the remainders. So the most of that other 10% , uh, were instructional support. Uh, so positions like psychologists, social workers and instructional coaches.
Speaker 1:Did you find any problems with how districts spent their at-risk funds?
Speaker 2:We did find a couple of issues. So to start with, we chose a sample of 179 expenditures is about 5.2 million , uh, from those, those same 20 school districts , uh, we chose items that reflected how districts spent their at-risk funds, but we also selected some that just kind of looked odd on their face. Uh, so because we didn't choose them randomly, again, our results here are not projectable , uh, but we reviewed them to determine if they were allowable under state law. So just as I kind of talked about very early on, the things that are allowable under state law are approved programs, and then the educational staff training or contracted services related to those approved programs. So what we found was that 71% of the expenditures we looked at were allowable. Uh, they met the criteria and statute and included things like teachers, paras , uh, and school counselors were some of the things that we, we saw that , uh, we considered allowable. 11 of our 20 districts , uh, had had no issues . So everything we looked at in the district , uh, was allowable on the other side. 29% of what we looked at , uh, were not allowable. And these were things like Sam's Club memberships, TVs office equipment. We saw some food service staff being paid for out of that fund. Um, so these were things that were, were not connected to an approved program. So out of our 20 districts, nine of our districts , uh, had at least one item that we looked at that was not allowable. We also had , uh, another concern , uh, and that was at it's unclear how much of the district's at-risk spending really provides an above and beyond opportunity. So statute, again, its statute that says that the purpose of at-risk programs is to pro provide an above and beyond opportunity. So there were a couple of things that we saw that that concerned us a bit. First , uh, and this goes back to the , that department approval list to programs. A lot of what is on that list , uh, is pretty general, and that seems to be leading to some pretty general spending. So for example , uh, GO Math is a general math curriculum. We saw a number of school districts using it. Uh, if a district's, if a district purchases it, purchases it, and then obviously the teachers would use it. Districts can use their at-risk funds to pay teacher salaries because they're using an approved program. But if that is the standard curriculum in the district, then that, then that spending would not necessarily lead to an above and beyond service for at-risk students. Uh, additionally we talked to districts about the process they used for determining how to spend at-risk funds, and we found that most districts do not directly connect spending to providing that above and beyond service. So most of the districts told us that they do pay a portion of teacher salaries with at-risk funds. Uh, but that portion is based on the percent of at-risk kids , uh, in the building or, or in the district. So if 30% of the kids in the building are at risk , then they pay 30% of their teacher salaries with at-risk funds. Uh, however, this again, does not directly connect spending to providing an additional service to at-risk kids. So while most of the spending we looked at was allowable under state law, it's not clear though that spending is really furthering the statutory purpose of at-risk services.
Speaker 1:So what are some of the factors that might have contributed to districts spending their funds in unallowable ways?
Speaker 2:So, K S D E provides guidance to school districts regarding how they can spend their at-risk funds. So we looked at the 2122 , uh, guidance document and compared it to state statute, and we found a couple of problems. So first of all, the guidance doesn't specify that expenditures must be for approved programs. It only says that the program should be evidence-based. Uh, the department told us that they mean approved when they say evidence-based. Uh, but based on some of our compliance work and what we heard from some of the districts, it's not clear that the districts , uh, are really understanding that distinction. Uh, additionally, the, the guidance document allows districts to spend at risk funds for certain staff that work at schools that are a hundred percent at risk. Uh, but this isn't allowed by statute. So statute ties spending to the approved list, not how many students are at risk in the building. Uh , the department agreed that in some circumstances that guidance may not be accurate. Uh , but we note these things because when guidance does not reflect state law, then districts may not spend their money appropriately. And in fact, we, we found districts who were not spending their money appropriately. And some of the districts that we talked to expressed some real surprise , uh, that their expenditures were not allowable because they were following K S D E guidance and they thought they should be allowable.
Speaker 1:So I see You also looked at student academic
Speaker 2:Achievement. How did
Speaker 1:You evaluate this and what did you find?
Speaker 2:So we looked at state assessments, graduation rates, and a c t scores. Over the last several years , uh, all that data was provided to us by the Department of Education, and we compared the results from students who are eligible to receive a free lunch to students who are not eligible to receiving a free lunch. Uh, we had to use that metric because in our data, there isn't really a flag for at risk , but there is a flag for whether or not the student was eligible for free lunch. And there is a great deal of overlap between free lunch and at risk . And so , uh, we were confident in using that, that metric, but we looked at state assessments , uh, first of all in two different ways. So we looked at math , uh, English language arts and science , uh, for the 2017 through the 2022 school year. Uh, obviously we didn't look at 2020. The kids didn't take state assessments that year because schools were closed due to covid. Uh, but we first compared the average assessment score of students eligible for free lunch to those who aren't eligible, and students eligible for free lunch performed worse than other students in all three subjects. And in all five years, we looked at, so students eligible for free lunch pretty consist consistently scored about a half level behind other students. So assessment scores are expressed as a level, so level one through four, one being limited ability for post-secondary readiness. Uh , four means a student shows excellent ability for post-secondary readiness. So an example of kinda what we consistently saw was that in English Language Arts in 2022, students eligible for free lunch scored on average , uh, 1.7. So that's just under level two, which is , uh, the basic level. But students not eligible , uh, for free lunch scored an average of 2.3, so a little above that basic level. So between those two is about a half a level, which is what we consistently saw. Another way that we looked at assessments was by following students over several years to see how they performed over time. So for example, we looked at third graders in 2017, and then we looked at that same group in 2022 when they were eighth graders. Uh, and we found that the proportion of students scoring a one, so that very bottom level, that proportion increases over time. And that was true for all students and in all subjects, but it was more pronounced in students eligible for free lunch. So for example, in math, 38% of students eligible for free lunch scored at level one. And the first year that we looked at them several years later, when we looked at that same group again, that percentage had increased to 67%. So 67% of students were scoring , uh, in math in in the bottom, the very bottom level. So in comparison, students who were not eligible for free lunch , uh, 12% of them were in level one that first time we looked at them. And then several years later, 31% of them were in level one. We also looked at graduation rates and a c t scores. Uh, but we found in a similar results that free lunch , uh, students who were eligible for free lunch , uh, their outcomes were worse than students who were not eligible. So in all the years that we looked at graduation rates for students eligible for free lunch we're consistently about 15% lower than students who were not eligible. We also looked at a c t scores, and the average a c T score for high school seniors eligible for free lunch was consistently about four points lower than the average for students who were not eligible.
Speaker 1:So this is not the first time that you've looked at at-risk programs in the last few years. So how did these findings in this audit compared to what you found a few years ago?
Speaker 2:We did do a similar audit in 2019. That audit didn't look at outcomes, but otherwise was pretty similar to the audit , uh, that we, we did. This time, that audit highlighted a few problems related to how districts spent their at-risk funds and problems with the department's approved program list. And those issues , uh, were mostly why the legislature asked us to look again , uh, this year. Uh , unfortunately as this audit shows, those issues still exist. Uh, additionally, we made some recommendations in 2019, and we found that none of the, the three recommendations we made have been adequately implemented.
Speaker 1:So finally, what was the biggest takeaway from this audit?
Speaker 2:So this is the second time we've looked at this in four years. Obviously, not much has changed. Uh, the continued problems with the department's approved at risk program lists are especially concerning. Uh , districts depend on that list to drive the programs they offer , um, and the services they offer as well. We found here that, you know , at-risk, kids are, are behind and they're not making up a lot of ground. And so it's critical that that approved list includes programs that are proven effective for at-risk kids, so that those kids who are struggling , struggling academically, that they get the effective help that they need.
Speaker 1:Heidi Zimmerman is a principal auditor at Legislative Post Audit . She supervised an audit that evaluated at risk expenditures and statutory compliance. Heidi, thanks for visiting the rundown in discussing this audit's findings with me .
Speaker 2:Thanks for having me. Morrie .
Speaker 1:Thank you for listening to the Rundown. To receive newly released podcast, subscribe to us on Spotify or Apple Podcasts. For more information about legislative post audit and to read our audit reports, visit ks lpa.org. Follow us on Twitter at ks audit or visit our Facebook page .