In this episode, Dr. Martha Mac Iver, associate professor at the Johns Hopkins University School of Education’s Center for Social Organization of Schools, joins Dr. Balfanz to discuss her new book, “Continuous Improvement in High Schools.” The book offers practical guidance to high school leaders and teachers on using a continuous improvement approach to enable more students to succeed.
Those designing education systems that work for all students need to resist the temptation to latch onto silver bullets. Context and circumstances always matter, even when educational strategies and practices are evidence-based. In addition, the more we seek to design education systems that work for all, the more we will find ourselves on the knowledge frontier: needing to figure things out in real time, rather than just trying to implement proven practices with fidelity. Using improvement science and a continuous improvement approach can help schools navigate these challenges and avoid potential pitfalls.
In this episode, Dr. Martha Mac Iver, associate professor at the Johns Hopkins University School of Education’s Center for Social Organization of Schools, joins Dr. Balfanz to discuss her new book, “Continuous Improvement in High Schools.” The book offers practical guidance to high school leaders and teachers on using a continuous improvement approach to enable more students to succeed.
Those designing education systems that work for all students need to resist the temptation to latch onto silver bullets. Context and circumstances always matter, even when educational strategies and practices are evidence-based. In addition, the more we seek to design education systems that work for all, the more we will find ourselves on the knowledge frontier: needing to figure things out in real time, rather than just trying to implement proven practices with fidelity. Using improvement science and a continuous improvement approach can help schools navigate these challenges and avoid potential pitfalls.
Robert (00:02):
Welcome to Designing Education, the Pathways to Adult Success podcast series. I'm Dr. Robert Balfanz, director of the Pathways to Adult Success program and the Everyone Graduates Center at Johns Hopkins University. I'm delighted to have you join us today. This is the third episode in a series of conversations we’re having with education leaders, thinkers, and practitioners across the country. In our conversations, we talk about what will it take to create an education system that truly empowers all young people and sets them on a pathway to long-term success. In today's episode, I'll be joined by my colleague, Dr. Martha MacIver. Martha is an associate professor at the Center for Social Organization of Schools, School of Education, Johns Hopkins University, and the lead author of a new book, Continuous Improvement in High Schools. The book is designed as practical guidance to high school leaders and teachers on how to use continuous improvement approaches to enable more students to succeed.
Robert (01:00):
As we work to design education systems that work for all students, we need to be ever mindful of the temptation to latch onto silver bullets, even when educational strategies and practices are evidence-based. Context and circumstance will always matter wherever we are: the more we work to design education systems that work for all students, the more we will find ourselves on the knowledge frontier, needing to work things out over time, rather than simply seeking high-fidelity implementation of proven practices. In both instances, using improvement sciences and continuous improvement approaches, potentially has much to offer. First used in automobile manufacturing, and more recently in healthcare, continuous improvement approaches have become increasingly popular in education. It is, however, a concept which, on one hand, is easy to grasp—we want to always work to do better—and, on the other, a rather complex methodology: how exactly do we effectively “always work to do better”? Welcome, Martha, and thank you for joining me today.
Martha (01:59):
Thanks, Bob, it's great to be here.
Robert (02:02):
So I'm going to start with a big question. Martha, how do you explain continuous improvement to schools? Can you give us a short primer on its core practices?
Martha (02:11):
Sure. In the book we really closely follow the framework that's set out by Anthony Bryk and his colleagues in their book Learning to Improve. So their starting point is the same as ours: to make the work problem-specific and user-centered; and school stakeholders really need to be involved in identifying the most important problems to work on. Next, we need to use data to understand the variation in all the outcomes around us, where things are going well in schools and where they're not going so well, and what factors could be associated with why we're seeing those different outcomes. We need on-time trying to understand the systems in classrooms or schools or districts that produce the outcomes that we see. We need to be thinking about how to measure things so we can know if the changes that we try out in continuous improvement are really making a difference or not. We need to use a systematic inquiry process to drive improvement. We have to be systematic in what we're doing. And finally, we need to connect with others who are doing the same kind of work to learn from each other, so that people at each school don't have to reinvent the wheel for themselves.
Robert (03:19):
Thanks, Martha. So in one way, that seems straightforward. But one of the beautiful things about your book is, it's really about keeping it real. And it tries to address how can we take a methodology which has worked for manufacturing and healthcare and get it to work at actual high schools, and in particular schools which serve high-needs populations and often find themselves in a constant state of stress and scarcity. What gives you confidence that high schools like these could effectively use continuous improvement methods?
Martha (03:49):
Well, I have confidence. They could do it because school leaders are really the most important actors in the continuous improvement process. Those who are on the ground at the schools are the ones who are really closest to the issues. And they generally have the most practical wisdom. And we have seen school staff at this kind of schools putting continuous improvement methods into practice.
Robert (04:08):
I would just add to that: in some of the work we've done, and some of this we've done together, that I think another power of this is that these are schools that have been under a lot of stress and scarcity. And oftentimes they feel like they’ve been buffeted by forces beyond their control. They're being mandated or told to do this or that, or get better or else. And continuous improvement really puts the power back on their hands and says, here is a way you could figure out what are the best solutions that work in your school under your circumstances. And what we've seen is, that gives them a great deal of agency and belief that they could actually do something to make their outcomes and their school better. So I want to go a little deeper now. One of the first things you recommend is that a high school, instead of taking a lot of time to figure out what to do to get better, should select a problem to tackle which evidence shows could have a big, positive impact on a large number of students. You can give us an example of what such a problem might be, and talk about why your experience working with high schools has led you to advocate this approach.
Martha (05:09):
Well, we know that schools have limited time to work on problems, and we've seen that in this work it can take a long time for groups of people to decide what they want to focus on. So schools need to improve their graduation rates. We encourage them to focus on what research has already identified as a primary driver of those graduation rates, and that’s ninth grade course failure. That's probably the strongest predictor of failing to graduate on time. We've learned a lot about how to address this problem over the years, and it makes sense for schools to build on what other people have learned rather than trying to figure it out for themselves.
Robert (05:46):
So that seems again straightforward, but another thing you bring out in the book that you're experiencing working with schools, is that problem identification, or even agreement—that we could all agree that students failing ninth grade courses is not a good thing—does not automatically lead to broad consensus among stakeholders. They should actually do something to address it. And if that's the case, how could school leaders bring about consensus?
Martha (06:11):
Well, you're right. We have found it's often difficult to get consensus among teachers about what to do about failing grades. There's a big ideological divide between those who think that course failure is good for students and those who see it as something to be prevented because the ramifications are so severe. So what we've tried to do is develop ways, professional development and frameworks, to help teachers talk through these issues during their professional learning community time. And we have seen teachers change their views and come to more of a consensus. Ideally, one can build this kind of consensus in this professional learning community approach to reflecting on grading. But unfortunately it takes time. And even if there is enough time, some teachers don't even want to spend time considering the issue. And so because of this, some school leaders have just made autocratic decisions about grading policies. What we've seen is that those have backfired and they really aren't building consensus. So we really do think it's important for school leaders to give their faculties time to learn together about the implications of grading systems and discuss how to use grading practices to increase student success in increasing their effort and mastering their course material.
Robert (07:32):
And just to build on that a little bit, some of the things we've learned in doing this work is that sometimes teachers are not fully aware of how the current grading system can make it very hard for students to recover if they've had some struggles early in the term. If you don't turn several assignments in, you might get a bunch of zeros. And if you average a bunch of zeros with even a bunch of seventies or low seventies on a test, you still end up with a failing grade and oftentimes what happens is, students will say, “I passed the test, but I failed the course. That must be ’cause the teacher doesn't like me,” and they miss the part about that you weren't able to get a lot of your assignments in.
Robert (08:10):
So neither the students nor the teachers identify what actually led to the failure. And therefore they're not in a great position to reduce it in the future or to deal with it in the present. So I want now to bring us to another aspect of continuous improvement, and this is one of my favorite ones actually. If I could put it in simple language, it says, it's not always about you. So often in education reform, teachers, leaders, and students are asked to look at themselves and say, what could you do better? How could you try harder? How could you learn to teach better? How could you learn to be a better leader? All of that is important, but a key notion of continuous improvement is understanding the set of actions or systems that are creating our current outcomes. This idea that it's not just your own individual effort, but it's the system under which that effort operates that often leads to the outcomes we have. One of the things that you recommend in the book that folks do is a practice of asking five whys. So can you give us an example of how we could use five whys to really understand the system that drives some of the outcomes we're currently getting?
Martha (09:26):
Sure. So the five whys is a practice that was originally used by the Toyota company to get to the root causes of the problems in the manufacturing process. And people have used it very well in their efforts in public health and in education it's been very useful. So basically it's a process of continuing to ask why something is occurring. So if you were to apply this process to the problem of failing grades, you might have something like this. You'd start with, why are student failing their classes? A potential answer is because their low grades on tests or quizzes or assignments have given them an average below 60. So the next question is, well, why were their grades so low on each of these tests or quizzes or assignments? A potential answer would be because they usually didn't complete the work.
Martha (10:15):
And so next you would ask, well, why didn't they complete the work? Why didn't they complete the tests? Or why didn't they turn in the assignments? And one potential answer is, because they were feeling so discouraged that they just gave up. So next you would ask, well, why did they give up? And here you could think of several different scenarios. And actually, at each part along the way, there could be several different reasons. Buti f you think about why students give up, it could be because they hadn't studied sufficiently to be able to answer the questions on a test. Or another explanation would be, they didn't have enough prior knowledge to begin with to be able to do the coursework, and so the assignments they're given in class, they just don't even know where to begin. Or they could be struggling with personal issues that kept them from being able to focus on the assignment at hand or on the test.
Martha (11:04):
And you can see that those three different reasons would have three different intervention strategies that would be important to do. That's just one example of trying to really understand. I think some teachers just stop at the beginning—Well, they didn't do it—and they don't even keep on asking the question of why they didn't do it and try to get to the root causes of what could help students be more successful.
Robert (11:35):
Thank you, Martha. And I think the build on that would be that the power of the five whys is that it's a way to give a pretty simple method, but a powerful method, to show folks how to dig deeper, to really understand the complexity of what's going on. And you're right. Oftentimes people stop at the very surface level of like, well, you didn't get your work done; well, try harder next time or learn the lesson. If you don't get your work done, then you don't necessarily pass a class. And that's a good life lesson. But they’re not really fully understanding the huge ramifications of ninth grade course failure on the life chances of that student. I know that in another part of the book, you talk about this idea that course failure is different from certain other failures. And that's another important thing to get people to have a deeper understanding of,
Martha (12:25):
Yeah, it's not like failing to make the school play or failing to make the team, you know. It really has huge ramifications. And I think that teachers are willing for students to take a course over, I guess they will give them a second chance in that respect, but they aren't often really willing to give them recovery opportunities during the course itself. And we want to get them to think a little bit more closely about how that might be helpful to students, to actually get them to change their patterns and actually do the work instead of just not doing the work.
Robert (12:59):
And one of the things that the five whys will show is that for different students, it's different. It is different foundational reasons, right? And that's why it's best to try to have a deeper understanding rather than just making some broad initial assumptions. So one of the joys of this book, I think, is that it toggles between helping explain how to use continuous improvement, methodology and education, and specifically in high schools, and then also very practically, how high schools can use these strategies to reduce course failure and improve outcomes in the ninth grade. And one of the ways you do that is, you identify and then interweave examples throughout the book. As you're talking about the methodology of five areas where high schools can apply continuous improvement strategies to improve and more specifically address course failure. Can you briefly share with us what these five areas that you identify are?
Martha (13:57):
Sure. We, we identified these five areas just through where research has identified some promising results. So the first thing we started with was early warning and intervention systems. That's what our center's been working on for more than a decade now. I think we're up to almost two decades and we really had a lot of work across the country on this. And we know that that if schools actually create systems to tell them when students are struggling and to help them figure out what to do, then they can make a difference. So we that's the first one. The second one is family engagement strategies, and there's really so much room for improvement at the high school level. And so we've done some work with our colleagues Joyce Epstein and Steve Sheldon about how to improve family engagement during this transition from middle to high school. And that's our second umbrella. And the third one is increasing students’ sense of connectedness to school. And fourth, we focus on ways of improving students’ non-cognitive skills through social-emotional learning interventions, where there's been quite a bit of recent research. And finally the last one we address, and this is based on a lot of work we've also done at the center, is how to improve instructional strategies so that we're more effectively engaging and motivating students to apply themselves to their academic work in high school.
Robert (15:23):
Great. Let's dig a little deeper into a few of these. Can you give us some examples of some things high schools might do to improve their use of early warning systems?
Martha (15:32):
Sure. Well, the first thing they really do need to do is to take stock of what's in place and how well it's working. It's sort of like mapping out the system and seeing where the holes in the system are, where things aren't working. So for example, the kinds of questions they would need to ask are, is their data system actually regular generating lists of students who have a failing grade at the end of the first quarter? And that's really important, first quarter, to be able to recover before the end of the first semester when sometimes the grade actually counts as a half credit. They need to know who's monitoring these lists and then how timely , and they need to see if they have a system in a place that can intervene with students who have a failing grade.
Martha (16:16):
And how well is that system working? What percentage of students are actually receiving some kind of intervention? So you can see in addressing each of these questions along the way, they can see a place where they really need to take stock, because all schools are at a different point. In the process, there's research that says that more than half the schools in the United States say they have an early warning system. The issue is how well it's working, and where it needs to be improved. One of the things that schools really have a problem with, is that they can identify students, but they're not really keeping track of interventions that students are receiving and following up on how well those are working. And it's really, really important for them to be doing this kind of monitoring work because what we see, in school after school, is that there are warning signals at first quarter, but they don't get themselves organized quickly enough to do interventions, to prevent the failure at the first semester. That really is a consequential failure. I mean, there are many schools where you have to recover that half credit. And so the whole goal in early warning systems is to prevent failure to keep students on track, or help them get back on track if they're starting to fail courses.
Robert (17:27):
Yeah. And, to add to that a little bit, one of the things that you mentioned that I've really seen is this is this idea of tracking your interventions and then being able to go back and see if they made a difference. And again, that seems so straightforward, but what happens in schools again, given this stress and scarcity they're often operating under, is that they're so focused on figuring out what do we need to do, that they don't take the next 30 seconds to record what they did. They just think, well, we'll remember it, and then we'll check in next time. And in busy environments, it's not always remembered. And even if it is remembered, no one actually knows that it happened. And therefore, it's hard for you to understand, is this intervention effective or not? So another thing I've observed about continuous improvement is in general, it builds the muscles of this idea of doing rapid data analysis and rapid data checking, not some big complex thing, but just, let's try. Ehat if we record our interventions and then we check them two months later; does that lead to better outcomes?
Robert (18:40):
And those are kind of the small decisions that are much better made close to the work in the schools, right? That's normally not something that anyone could ever effectively mandate. And if we don't have these continuous improvement tools and processes, then these simple but powerful things just often don't get done. So let's look at one more. And I know this is one that you've put a lot of work into and think this is really underappreciated, which is, how can we engage families to support their ninth grade students?
Martha (19:16):
Yes, this is really near to my heart. There is a lot of research that shows that families of high schoolers don't believe that schools are communicating as much or as effectively as they need to. And we actually have research from principals in high schools where they admit they're not doing it. I mean, if you compare their results with results from elementary and middle schools, they're not doing as much. And there's this sort of feeling like, “wow, students need to learn responsibility and we need to leave.” You know, we're focusing on the students, not on the families, but families are such an important part of students’ success. They’re really the ones that help make sure that students get to school. They're the ones who can monitor whether or not they're turning in the assignments.
Martha (19:58):
And these families are really clamoring for more information and they're not always getting the information that they need. And so we worked with a district to try to increase the communication that was happening starting in eighth grade and then moving to ninth grade. There are so many ways that schools can actually make small changes that can make a huge difference. So, for example, first of all, they need to take stock. It's really eye-opening sometimes for schools to take stock and see what's actually occurring and what's not occurring in communicating with families. And they need to think about how to communicate in multiple ways, because doing it only one way is not going to reach all the families they need to. A lot of families need help, for example, learning how to access the parent portal.
Martha (20:51):
Many families don't know about it, or they don't have the sign-in, or they don't know the process, or—there is research from randomized studies that shows if you just remind parents and give them an extra nudge and remind them how to get on, that they will get on more and their students will do better. And so because of the evidence that's there, we are really trying to help schools to think more closely about how they can engage families more, especially during that crucial transition to high school.
Robert (21:25):
Thank you. Um, a final key aspect of improvement science and continuous improvement is that improving alone is very hard. If each teacher or school or even district is asked to figure out everything by themselves, it becomes very frustrating and lonely with lots of wheels spinning. How does the continuous improvement process of building networks seek to address this, and what are some of the real world challenges to doing so?
Martha (21:52):
Yes. When you have a network of people at different schools or different districts who are working on the same problems and using some of the same approaches, there's great power in sharing their learning together. Groups can come together and they can come up with a common approach to measuring so that they can actually see how they're doing compared to their colleagues who are working on the same thing. Um, they can share where they've gone down, um, a dead end path and, and they wanna, you know, help prevent others from doing that. Um, they can, uh, it's really, it accelerates learning as, as, um, Anthony break says because people, um, can, um, benefit from the learning of others and get ideas from others. And, and, uh, of course the, the thing that prevents this is time and actually finding people who are working on the same problems in the same way. And so organizations like the Gates Foundation and the Carnegie Foundation are trying to help build these networks for schools and districts to work together. And we've been trying to do that at the Everyone Graduates Center.
Martha (23:03):
And so, I would really encourage those who are listening that if you're working at a problem and you're having trouble finding others who are working on the same thing or people that you can learn from, you really should contact us, because we can probably put you in contact with others who are working on similar issues.
Robert (23:18):
Yeah. And one of the things to add to this, that I observe in working with schools doing some of this continuous improvement, especially the networking phase of it: you ask them, do they see benefits in the network? Inevitably they come back to two things. One, that they really find power in seeing that they're not alone, that they're not the only one struggling with these issues and these challenges or with these circumstances. And then secondly, that they're not a failure—that there are other folks that have been struggling with that. Haven't figured it out either. It's hard and there are no simple solutions and it's complex and it's not like that the failure is because they haven't cared or tried enough, which they obviously do care and do try, but they sometimes feel the view is that they haven't.
Robert (24:07):
So both the idea that they're not alone in facing a challenge and they're not alone in not yet overcoming that challenge. One of the things they really expressed is the power of being networked. And one thing I want to explore a little more with you. It's an issue that we went back and forth on in the book. How do schools, especially schools that are facing a lot of challenges, and maybe have had a lot of staff turnover or leadership turnover, or (like everybody now) are really struggling to come back from the pandemic, just build a basic sense of connection among staff and kids? Again, how do they balance the idea of picking some big improvement strategies that do have a good evidence base and just seeking to implement them, versus where they really want to learn and use and find great value in power in using continuous improvement methods to address issues? How do they balance that question when they're facing a lot of challenges at once?
Martha (25:20):
Oh, that's a hard question. I think that continuous improvement is a method and it should be used as far as it's useful. And I think being willing to adapt… I mean, we've set forth these six principles and I know there are some people that are very concerned about measurement and making sure we try things out in a systematic way, and that's important. But I think, at this point in the balance, the most important thing is to identify something, try out something, keep some kind of records and some kind of tally about how things are working. But I think the most important thing is just to get started and not worry about doing things perfectly.
Martha (26:17):
I think that's one of the glories of continuous improvement, is that it recognizes that failure is just something that's normal and you just identify where things aren't working and you just keep going. And so I guess I would encourage schools—and I guess this is even difficult at this time, when you think about the pandemic and all the stress that teachers are experiencing—but I think teachers also enjoy being in community. We know that. And so if they can work together with colleagues that they get along with, and work together on a problem to solve it together and have the freedom to try things out and share what they're learning with the rest of their colleagues at the school, that's one way to do this.
Martha (27:11):
And I think we've given some descriptions of that in the book, of how small groups of teachers at a school, who are volunteers who want to do this… I think you just empower the people that want to solve problems to work together. Give them some tools and help them to think about the process of using data and then give them an opportunity to share what they're learning with their colleagues, and to make it something that's not imposed, but something that emerges from their own desire. And I think there are enough people at each school that want to do this, just to empower that small group of people to do what they will do well. And then to be able to share what they've learned. I think that's a great way to start.
Robert (27:54):
Thank you, Martha. As we work to design education systems to give all students a pathway to adult success, we're not going to get where we want to go without increasing the capacity of schools and districts to continually learn and improve. That is why your wonderful book, which I had the pleasure of coauthoring with you, Continuous Improvement in High Schools: Helping More Students Succeed, from the Harvard Education Press, is so timely. It seeks to give high school leaders and teachers practical guidance based on extensive experience on how to improve. For too long, we've been waiting for Superman or a set of evidence-based strategies that work for everyone. There's just too much variability in the education process for this to succeed. We need to keep building on our knowledge of what works, where, under what circumstances; and to do that, we need to have many more folks in the game of knowledge and wisdom building. As you gave some great examples of today, we need to fully engage our teachers and school leaders in that work, in continuous improvement. Improvement science methods are one of the tools we will need to get there. This is Robert Balfanz from the Everyone Graduates Cnter and the Pathways to Adult Success project thanking everyone for listening, inviting you to listen to our other podcast and the Designing Education series. Thank you, be well, and onward.