Gateway Course Placement with Dr. Melinda Karp

Learn about a study of placement as a catalyst for institutional transformation.

Continuous Learning Institute:
A resource hub for higher education professionals to support college student success. Subscribe for updates.

Student Success Podcast Homepage:
Access show notes, resources, & transcripts

Transcript

Al Solano 0:01

Welcome to the Student Success Podcast. I'm Al Solano, founder of the Continuous Learning Institute, or CLI, a higher education online resource focused on providing community college and open access university educators with practical information on how to get results at their campus. As a resource within CLI, the Student Success Podcast is focused on just that the challenges, opportunities, failures and successes of practices intended to improve student success and equity. The goal is to leave you with thought-provoking ideas, nuts and bolts, information and or lessons learned from the field so you can consider how you might apply them to your institutional context them to your institutional context For today's podcast. It's a pleasure to have Dr Melinda Karp again at the Student Success Podcast.

Al Solano 0:55

Melinda is a nationally recognized expert on improving students' transition to college and supporting them. Once there, she founded Phase 2 Advisory after nearly 20 years conducting research and working with colleges on education reform as Assistant Director at the Community College Research Center, teachers College, columbia University. The proud granddaughter of refugees, belinda works with national and institutional leaders, campus-based faculty and staff and philanthropists to ensure that all individuals have the opportunity to realize the intergenerational mobility higher education provides. Linda holds a BS in Human Development and Family Studies from Cornell University my alma mater too, and both an MA and a PhD in Sociology and Education from Columbia University. She chairs the Effective Advising Practice Guide panel for the Institute on Education Sciences' what Works Clearinghouse, and she is a member of the inaugural editorial team for the Journal of Post-Secondary Student Success. Welcome back again, melinda.

Melinda Karp 1:59

Thank you, grover and Red. I'm so glad to be here, nice, to see you. I actually spent three weeks up in Ithaca this summer and thought of you a little bit, al, as we were wandering around campus.

Al Solano 2:13

Oh, that's so cool. Well, you know, my next question is going to be if you wouldn't mind sharing a story, a hobby or anything beyond work. So yeah, I'd love to hear about Ithaca, if that's part of the story or anything else.

Melinda Karp 2:28

Well, it wasn't going to be, but it can be. Yeah, we like to head up there and sit by the light, and I actually one of the things about outside of work is the extent to which I get to read when we're out there, and that's one of the things I do in my free time. And as an academic, you know, reading is our job. But I have found that I've been reading really expansively lately, well beyond the higher ed literature, partly because it makes me better at my job. So reading a lot of memoirs, a lot of political analysis, things that aren't technically higher ed but I think really inform the way, you know, we understand the world. So it's fun and it's interesting. But it shows up in my work sometimes too. The kids always joke that my book they don't want to read my bookshelf because all my books are too smart, which isn't true, but that's what they think, so I'm going to go with that.

Al Solano 3:24

Is there a particular topic that's really they think? So I'm going to go with that. Is there a particular topic that's really resonated with you? I?

Melinda Karp 3:28

don't know the memoir aside from just you know, an innate curiosity in humans. So I love the stories and I find memoirs really useful because so much of what we do is really about having to understand how to walk in other people's shoes when you can't inherently walk in everyone's shoes. Right, like part of what we do as educators is meet students where they are. And if you don't know their lived experiences because the diversity of lived experiences are so broad memoir is really helpful. Plus, I just I like people.

Melinda Karp 4:00

The other one I'm doing a lot of is really thinking about how we got to this political and economic moment and because I think it helps understand how to navigate some of the challenges that folks in the higher ed space have been grappling with right. So I read this book, american Resistance by David Rothkopf, which really digs into what do you do when the world around you doesn't let you fully live out your values. And it doesn't technically have anything to do with higher ed, but I find increasingly that that story and that thinking about what do you do resonates because we are all constrained when we work in public institutions.

Al Solano 4:39

Yeah, we can do a whole podcast just on that.

Melinda Karp 4:45

Yes, we could. I mean, we won't right now, but, um, I've been. You know, reading expansively has, I think, at least helped me feel like I have some tools from other spaces. Um, as we work with, as I work with the college leaders to figure out how to navigate, you know, some of the trickier moments that that we've all faced in the past couple years and I imagine are are coming our way over the next six to 12 months, yeah, I think it's wonderful that you're doing that.

Al Solano 5:10

Too many of my colleagues they're researchers in higher ed. It's good that you can become an expert in an area and you can dig deep, but over time it can give someone a really myopic view of how to approach things and I'm finding that increasingly troublesome. Actually, they're not willing to learn other fields, they're very stuck on. Well, this is my theory and I'm going to stick with it and I'm going to be unapologetic about it and I'm like, oh, but you're being unapologetically unproductive when you're working with colleges, when you only have one view of how things ought to operate. So kudos to you. Thank you for doing that, melinda, expanding your mind beyond what all the higher ed stuff we read. And Ithaca I haven't been there since 1990. How's Ithaca? As beautiful as ever.

Melinda Karp 6:13

Beautiful, as ever. It's different. You got to get back. You know us old folks. They've changed campus a little bit but I just, you know I don't live on the East Coast anymore, but I'm a New Yorker, new York State are born and bred, and so it's nice to get home and see the lake. You know we were there in July, which is great. I always joke. I have no interest in being there in February. Anyone who knows Western New York or Central New York knows that February is not the time to go. But three weeks in the summer it's great, great food, great wine. Campus is delightful. Sunday, I'll get you back there.

Al Solano 6:47

Nice, yeah, I got to head back there. So last time you were here we discussed the holistic student support, student success teams and I shared your guide widely. I even used it at one campus to help them really think through the different types, and so thank you for that. And then you're returning back to the Student Success Podcast because you've done some really meaningful work around placement. So can you tell us a little bit about that journey, the why? Why did you choose placement?

Melinda Karp 7:22

Yeah, and thanks for using our stuff, and I'm always happy to share with your listeners the resources that we have on holistic student support and middle leadership. But this project is really about placement and institutional transformation, which is what we do at Face to Advice. Our working assumption is that our broad access institutions, particularly community colleges, have grown up to have certain structures and ways of doing things that come from a very long tradition instead of, at the time, rational reasons, right and now, don't work terribly well for many of our students, particularly if we think about this from an equity perspective. And so it's not enough to just reform or improve what we're already doing, although that's important, but it really is fundamental about transforming, like really shifting how we do business, and that comes in lots of forms, and so we were talking with a funder and this perspective, and so you know we were thinking about all of the work that's been done in the developmental education reform space, and it has been immense and it has been extraordinarily successful, right, if you look at the data, many more students are getting into and through their gateway English and math courses than was the case 10 or 15 years ago. In many instances, gaps between racially minoritized and or low-income students and their more advantaged peers are closing, although they are not closed. It has been perhaps one of the most widespread and successful reform efforts that we have seen in our sector and it's not nearly enough right, because many, many students are still not getting into and through gateway courses. Equity gaps are not closing. We are starting to see some emerging evidence from some leading states California and Florida notably that while gateway course outcomes are improving, completion and transfer are not, so something's happening there. Right, because the goal isn't to get students through gateway math and English, the goal is to get them to the finish line. So there's sort of this puzzle and at Asendium we were just sort of playing with this idea of like, well, what's happening here? What do we do that takes the successes of the dev ed reform movement and pushes the next envelope.

Melinda Karp 9:53

Folks who really started putting out this notion of multiple measures and people thought they were out of their minds, right, people thought that was radical. How could you possibly not place students using the standardized test or using their high school grades Like you have lost it? And these brave reformers said no, we think this is better, we have the data, we are going to do this and we're going to run around the country and we're going to get other people to try it, such that more people can try it, more people can test it and we can build the evidence base and, today, multiple measures. So that idea of using GPA or using a combination of grades and course taking to place students is not radical at all. Our thought was well, who's the next version of that? Like, who's the next radical thing? How do we find them? How do we write about them and sort of seed the field with these interesting things that then can be tested and if they seem to work, we can try and scale them so that 10 years from now we push the envelope even further. So this is really an exploratory study. It was literally go out and find the next wave of people, and we were really focused on placement, although when we get to findings I'm sure we'll talk about the ways that placement reforms now dovetail with other efforts. But we were really trying to find these renegades that we can talk about to people like you, al, and to get other folks to try it.

Melinda Karp 11:27

The second hypothesis around this was that placement reform is not going to be enough. Right that, really? And this goes back to these two's notion of transformation. Placement is great because it improves something we already have, but it doesn't fundamentally change how we do business right. So placement reforms historically have focused on being more accurate, getting the right students into the right classes. They don't interrogate by design how do we teach, how do we view college readiness, how do we think about the systems within our institution? And those are the things that I think most of us know like you've really got to push on.

Melinda Karp 12:06

If we don't push on pedagogy, it's going to be real hard to change student outcomes, because the classroom is where students spend the bulk of their time.

Melinda Karp 12:13

If we don't change how we talk to students, if we don't change how we support them, just being more accurate in placing them is going to be insufficient.

Melinda Karp 12:20

Those core functions of the institution have been the most resistant to reform over the past decade or two, and so we have this idea of maybe placement can be bigger than accuracy.

Melinda Karp 12:33

Maybe there's a way to use placement reform which, as hard as it is, is a relatively small bore reform as compared to like changing all your teaching. Maybe there's a way to use placements to push on these core functions and we call that placement as transformation, so the idea that by changing the practices around how you place students you might also change the broader functions of the college. And so we had this hypothesis that that might be happening out there and if we could understand that it might give us an additional lever to push on the broader notion of institutional transformation that we think is necessary to really get to a place of strong, equitable outcomes. So we had two goals. I'm just going to summarize because I was scouting for a while. One goal was to find these renegades who are playing with placement, that next frontier. The other goal was to understand if it is possible to use placement reform to push on transformation of an institution.

Al Solano 13:30

Thank you for giving us that context. Tell us a little bit about the methodology, and then we'll dig into the findings.

Melinda Karp 13:37

Yeah. So this was a floor for a study. So I want to be really clear. We were not really looking at outcomes outside of anything an institution had had on its own. That was not our purpose. What we wanted to know was sort of describing what folks are doing, how they were doing it, what they think the implications might be for their students and their institutions. So we used a technique called snowball sampling. We set up a set of criteria that sort of said these are the kinds of colleges we're looking for right. So colleges that are engaged in something beyond sort of traditional placement reform, so bigger or different than multiple measures, primarily or using a standardized test.

Melinda Karp 14:14

We were looking for institutions that had equity in mind and we were looking for institutions that, at least on the surface, we're trying to use placement to do something beyond accuracy. So we put out at all of our networks. We just emailed everybody. We probably emailed you out. We emailed, you know, all of our friends. We put it on I think at the time we were still using Twitter. We don't do that anymore. We put it on LinkedIn and just said if you know of anyone, send them our way. And then we just started screening institutions.

Melinda Karp 14:45

We tried to make sure that we had a combination of colleges in states that were doing developmental education reform at a policy level and some that weren't. We were looking for geographic diversity. We actually were really looking for rural institutions in particular, because they are often underrepresented in national studies. But we know that many low-income and racially marginalized students live in rural areas too. So we wanted to make sure we captured big and small and rural and urban and all the things you do, and we ended up with 15 colleges throughout the country in 13 states. So that was exciting and we do have some very rural institutions as well as some urban ones.

Melinda Karp 15:19

And then at each college we talked to between three and five individuals.

Melinda Karp 15:23

We talked to the person who was sort of in charge of this reform work, and so that varied a little bit from college to college.

Melinda Karp 15:30

We called them our key informants, so they were the ones who kind of knew the whole thing. And then we talked to a handful of folks who were implicated, as we say, by the change in placement, and that depended a little bit again on the college's approach, but it was typically a faculty member, someone in advising. In some instances it was folks in their IT office or because they had to rethink how they do all of their back-end SIS work. So it really varied a bit. It was the placement office, the tutoring office, various things like that. We did what we call semi-structured interviews, so about 45 to 60 minutes with a set of questions. That then sort of followed where the conversation went, took notes and then did all of the analysis and we did a couple of kinds of analysis that we can get into if it's of interest. I don't want to go too far down the methodological rabbit hole, but we ended up talking to about almost 50 folks across 15 institutions.

Al Solano 16:29

So thank you for unpacking the methodology. By the way, how novel to actually talk to community college people to get information. I'm seeing this we're going to go, look at compliance documents and count words and make assumptions about people. Thank you for actually taking the time to talk to community college professionals. That that's.

Melinda Karp 16:51

That's the way to do it I will say before we go, the one thing we didn't do it. It hurts me a little bit and for anyone wants to find future research, we did not talk to students. Um, and actually when we get to the findings you'll see that that is something that feels very glaringly absent from this research, and that was because the study was designed to look at institutional practices. But I think it's really critical to acknowledge that most of what we uncovered really tries to elevate agency for students and we didn't talk to students. So we actually don't know how the students, all the things we have to say, like we don't know if the students agree with what the institutions believe, and I think it's important to acknowledge that there was a good reason for not talking to students. But to your point about wanting to really get to the authenticness of what's happening in a college, it is so critical to talk to practitioners, but the students, they know things. Study didn't include their voices.

Al Solano 17:47

Yeah, no, it could be a follow-up, right, but still you dug in. With 15 institutions in 13 states, you know over 50 people. That's a lot of work, so thank you for doing that. Yeah, I mean the thing about students. Now I'm just having a flashback.

Al Solano 18:02

So I was a re-entry student and I remember I was just left the military and we're planners right In the military. We just learned to plan. So I was like, all right, let me, I want to go back to school. And so I went to the nearest community college just to do recon, if you will some reconnaissance like, well, okay, what do I? How does this work? Right? And then the first thing they told me is oh, you have to take some tests. I was like, oh shit, I hadn't taken math in I don't know five years and I wasn't that great at it. I was working full time when I was in high school. I actually worked like 40 hours a week when I was in high school is crazy. But anyway, I just felt like, wow, okay, I have to take tests. Okay, fortunately for me, I had a little bit of money.

Al Solano 18:47

I went to some tutor it was a tutoring center and at the end of the day I didn't really learn that much. They taught me kind of some test taking tricks, no-transcript to have placed into transfer level English and math. I was so, so fortunate. What's really interesting is that I think, yeah, I spent five semesters. I was a spring student. Of all the students that I knew that were not in transfer level English and math, because I was very active, I was part of clubs and everything when I was transferring, not one of them made it out, not one, not one. I remember this was before. I'm giving you the retroactive student perspective Right. So I remember that this was before my professor. The Internet wasn't really popping back then. Yeah, I don't even think it was. Yeah, it was early 90s, right.

Melinda Karp 19:55

You didn't have internet back then, Al Sorry.

Al Solano 19:58

Yeah, but I remember at the Veterans Center there was a list on a board and they're called friendlies. You want to guess what that is, melinda?

Melinda Karp 20:11

Those were the professors to go to, who weren't going to be awful.

Al Solano 20:15

Yes.

Melinda Karp 20:17

Oh yes.

Al Solano 20:18

So I was like wait, so these are the faculty who what they're like, these are the ones who care. And I said, well, wait a second. Are you telling me they're faculty that don't care? It just occurred to me there's a list and then, when I remember choosing the faculty, I remember I went to one of them. That was a math instructor and an office. I was just introducing myself and he was like really focused on just hey, I'm planning for today. It's basically it was lesson planning, so I'm going to put you in groups and then I'm going to do this and then I'm going to do that. And then right after class, I have something in the other room for 30 minutes with a student who had an A in this class and he's going to help you all and he was basically doing the early version, the money. There's a good chance I wouldn't even be here.

Melinda Karp 21:13

Yeah.

Al Solano 21:15

Right. And the second thing it really came down to I was able to overcome the placement hurdle, but it came down to caring, it came down to quality instruction.

Melinda Karp 21:28

Yeah, I mean and your story breaks my heart because you're just a story today and this is a different study, so I won't spend much time on it but we also recently did a study talking to students about how they find their courses, and one of the things that they told us was they use RateMyProfessor and they know it's not terribly accurate, but they are very focused on finding professors who are quote unquote good, and I think many folks hear that and think, oh, they're looking for the easy A, but what they're looking for is exactly what you described.

Melinda Karp 21:58

They're looking for professors who care. They need to know that they're going to be successful. Our students do not have time to waste. They do not have money to waste on classes that will not lead to actual credit. They don't have time, and so they're looking for instructors who are good and that means that we'll get them to the finish line not necessarily easy, but are not going to get in their way, and that's separate from the study we're talking about. But I think it's so critical to hear that, because the assumption we make is good equals easy, and that is not what students are saying. They're asking for this base level of care that you just described.

Al Solano 22:39

That's right and the sad thing, I think you're aware I develop a six-step process in Korean action and I bring teams together. I work primarily with faculty I would say 60 to 70% are faculty and I take them through this process to have these beautiful aha moments about their practice to continually improve and then sometimes I get in these teams is really fascinating. Our faculty who have a deficit mindset about their fellow faculty, for example, oh well, they like going to her class because she's easy and she grades easy. And I'm like, wait a second, how denigrating is that that you just made an assumption that she's easy when she works her ass off to continually improve her craft and when she grades she looks beyond points, she's really looking for learning. So it's a very, very interesting culture in higher ed still. But I know we took a little detour, but that's OK. I think you'll have that student perspective. I'm sure it could be part of a follow up study.

Study findings

Melinda Karp 23:54

Let's unpack high level, so you know one thing, and this won't be surprising to anyone who works in the field historically developmental education reform has been sort of three buckets of reform, right, and maybe institutions and states were doing more than one of those, but at any given time they were sort of discrete. One was placement reform. One was the curricular reform co-requisites or pathways, changing that curriculum into and through gateway classes. And the third was really the pedagogy making sure that the pedagogy we're using, particularly in developmental courses, is not the same old, same old students have experienced and failed with before. So those three things were sort of separate, even if some institutions were, some institutions were trying combinations of those. But those were three buckets of reform. What we're finding is that you really no longer can do that. Like most institutions in the vanguard are combining placement with something else, to the point where we have some colleges where they're like well, we don't even do placement anymore. So it's almost a misnomer to call this a study of placement, because a couple of our colleges don't do placement for anything because they don't have any of it. So that was interesting. The other thing that we found sort of at a high level was increasingly we're hearing discussions of integrating subpopulations of students who have been separate from the developmental education reform movement. So, for example, historically dev ed reform was really focused on credit-bearing, degree-seeking programs of study and so students who were coming in through GED or adult basic education, students who were an English language acquisition programs often had separate and weren't part of the dev ed reform conversation. This varies a little bit from state to state, but in part of the dev ed reform conversation this varies a little bit from state to state, but in broad strokes, dev ed reform was really on sort of your quote unquote traditional degree seeking students. We're starting to hear a lot more discussion about how to integrate particularly English language learners and adult basic ed or GED students into the dev ed reform space so that whatever change is happening around placement is being used not just for students in degree and credit-earning programs but actually for GED students or for ELL students. So there's a single door which feels like a really important equity innovation to us, because it means that students, regardless of their lived background, have the same access to those pathways.

Melinda Karp 26:34

The final thing we were surprised by was the extent to which we were finding innovation. So we did find you know, we found 15 colleges that were doing this, but it was harder than we thought. Our sense is that many places our institutions are really just doubled down on doing multiple measures, which is important. I don't want to denigrate that. That is a huge, important step away from a single test, but there were fewer folks trying to go beyond that than we had expected. We have some hypotheses as to why. I don't know that any of them are particularly good, but it took us a lot longer to find 15 innovating institutions that we expected as sort of an arc or a continuum. I am super glad that so many institutions have moved towards multiple measures. I am interested in the fact that in many institutions they think multiple measures is the end game, because I think if you look at the data, it's a really important intermediate step. I'm not convinced it's where we want to end. So that was just something you know. For all of you who want to be renegades like, there's a lot of space in the renegade space. So those are sort of our high level takeaways.

Melinda Karp 27:54

In terms of those 15 innovating institutions, we were able to come up with three sort of categories of innovation. One of them sort of takes multiple measures and pushes it, and we call that a reflective algorithm of takes multiple measures and pushes it, and we call that a reflective algorithm. Essentially, what this does is it takes the notion and I'm making the assumption that most of your listeners are familiar with multiple measures, but if you're not a multiple measures approach essentially says we're going to look at more than just a test score. It's typically something like a grade point average coupled with a test floor or coupled with the student's course taking. That then still in some way, through a formula or an algorithm, sort of tells a student where they're going to end up in, whatever the dev ed, curricular sequence, gateway course sequence a college offers. Algorithms go a bit further in that they integrate some version of student reflection, whether that's a student survey on basic needs, whether that's a student providing their own assessment of how confident or competent they feel about doing coursework. Often students are asked to just look at some sample problems and say like, hey, you know, do you think you could do this? Do you think you could do this with some help? Do you think you would really struggle? And then those self-assessments are combined with breakpoint average, more traditional multiple measures data, and an algorithm then spits out a student's placement. So it's allowing a bit more of the student to come in, but the college is still maintaining a bit of control over where the students like.

Melinda Karp 29:34

The second bucket is what we call guided self-placement, and you know, in fine higher ed terms, like everybody's got their own definition of guided self-placement directed self-placement, student directed self-placement, self-placement, student-directed self-placement. We're going to call it guided self-placement because what we are finding in these institutions is that this is an opportunity for a college to restructure how students are placed without letting it be a free-for-all. And so in this system, what we call guided self-placement, it's often very much the same data as used in those reflective algorithms. Students are asked to think about their grades, their experiences with English and math, their access to things like computers and Wi-Fi, but then the college gives them information about the courses that are available to them and then the students are provided with the opportunity to self-place given the guidance the institution provided. So it takes that algorithm and flips it. The students get to make the choice, but it also flips the onus on providing that information and guidance onto the institution.

Melinda Karp 30:48

The final version and this is where we really found our like radical innovators and I will say we only had three of those in our sample were what we call diagnostic just-in-time support and this was like super varied. But essentially what these colleges do is provide universal access to college credit courses, not even necessarily co-rec, but college credit courses coupled with an intentional provision of just-in-time support to all students. So that means that students are not coming in as ready or not college ready and only the quote-unquote not college ready students are getting support. Anyone can be college ready, anyone can be not college ready at any given time and then you get the support you need. And that's done in a couple of different ways.

Melinda Karp 31:33

In our sample. We have one that used a bootcamp. We have one that used a traditional co-rep visit but they didn't start it until week three of the semester, which gave the students the opportunity to figure out if they thought they needed the support. And then we have one really radical one that did all sorts of wild the support. And then we have one really radical one that did all sorts of wild, really cool grading with their adult basic education. So that bucket is both the most diverse and the most radical. But those are the three kinds of categories we found of innovation within the business.

Al Solano 32:04

And I find that my sample size is rather small. From my experience as a coach, I find that the third one you have to have the coalition of the willing. You got to have at least two math faculty, two English faculty to even get something started like this where it's universal access, because incorporating just-in-time support into their pedagogy is really flipping the way they've been taught math. They've been taught math and English in graduate school and often in very antiquated ways, and this is why I don't have a faculty deficit mindset, because they tend to do the things that were done to them and they don't know any better. So they do the things that were done to them and they don't know any better, so they do the same things to their students.

Melinda Karp 32:54

As these are the most transformative right. You can't do this just in time support, as you just alluded to, without really fundamentally shifting how people do their work, how they structure their day, who they engage with, how you view students and yourself Like you. You kind of have to transform the institution and you have to be very brave to do this just in time work, because the idea that everybody's college ready and nobody's college ready, like just that philosophical underpinning, like that, is not the paradigm we have worked under for many, many, many years.

Al Solano 33:31

Yeah, and I go back to my experience as a student. Here's a faculty member who was taking the time to lesson plan to figure out how he was putting us into groups, how he was going to teach, how he's going to provide support. Right, and I hadn't done math in five years and I was like a C plus student in math and then I passed it.

Melinda Karp 33:52

Yeah.

Al Solano 33:54

Right. So the pedagogy and the caring really does matter. Now here's what I find. I don't know if you found the same thing, because it's usually a small coalition of the willing and it takes time to bring more people on board. It's usually a small coalition of the willing and it takes time to bring more people on board.

Al Solano 34:14

Even with the coalition of the willing, we find often that the core success rates decrease. So let's say they were at 70% and now they went down by 10, 15 points, but the throughput rate is still much better than putting them in two or three classes below transfer level. I mean significantly better, right. So I have to remind some faculty because they say, well, yeah, but look, look at the data, course success rates are going down. I say, yeah, but you know what? There's more students, especially students of color, who are completing English and math than ever before. So what we got to do is go back to that 65, 70 percent, and that just takes continually working on your craft, right? So, yeah, I find that third one to be the most difficult one, and course success rates tend to take a dip, but throughput rates are fantastic. Are you finding the same thing?

Melinda Karp 35:09

So not entirely, and I will say that we were pretty bound that most of our institutions didn't have great data yet, and part of that was we collected the data about a year ago, a little less. Many of these innovations were put in place in part either due to or in conjunction with the pandemic, so they were very new, which meant that they didn't have a ton of data to go on and the data they did were pandemically colored, which you know makes it real hard to interpret. But these just-in-time colleges at least one of them was, and this is our most radical, this one that was mushing adult basic ed and all the things they were finding that they were having very strong success because it was such tailored support. Right now, this one you couldn't do with just the Coalition of the Willing, our just-in-time support institutions at least two of the three really needed broader based support because it is so radical. You can't only do this in a section or two. If you're asking all your adult basic ed instructors to push into your gateway courses, that's way more than the Coalition of the Willing and that is an interesting conversation to have. But because of that the students were getting what they needed when they needed it, which means that you can, and you have their system to do that, which means you can catch many more students before they fail right. So they were not really seeing a dip in course completion rates.

Melinda Karp 36:38

And what was interesting is that this institution in there the one I'm probably most obsessed researchers don't admit it, but we get obsessed with certain parts of our data sets and this is the college that I'm currently obsessed with because they're doing such interesting, radical, thoughtful things this was an institution that had scaled their placement reform all the way to their very large prison reentry program. So their prison reentry students were also going directly into gateway courses with pretty intensive just-in-time support. So when I talk about expanding sort of the envelope of who's included in institutional change, that's what we're talking about here. And they were seeing really good success. Again, not every prison reentry student was successful from the get-go because we know that typically students in those programs are going to come in with a lot of challenges in and out of the classroom, but they were getting what they needed when they needed it and seeing strong success.

Melinda Karp 37:38

And they didn't have DevEd at this institution at least as we historically have known it, what they did, just to put a finer point on it is they essentially took their adult basic ed and this is a state that adult basic ed is offered through the community colleges but offered through different funding streams. They took those faculty who are traditionally really taught around pedagogy and used them to enhance, either through co-teaching or offering free tutoring or offering workshops to enhance the instruction in traditional gateway courses and other first level courses as well, which meant that students could get what they needed when they needed it from double professors right, and they leveraged their adult basic education to mean that no one needed to do dev ed, they would just get support from training instructors when they needed it, which I thought was really cool.

Al Solano 38:35

Yeah, that's rare in my experience.

Melinda Karp 38:38

Yeah, it's wild. They had to bring in their IT folks. They had to rebuild their SIS to keep track for all of their grant funding. So when I talk about transforming which we haven't gotten to, but we will they really did have to rethink all of how they did business and it has not been easy, but they are seeing success and students are not. They're not in Deva.

Al Solano 39:02

That's a that's beautiful. So you you've had all these interviews from an implementation standpoint, because you know that the devil is, and forever will be, in the details. When it comes to implementation, did you find a theme of who were the most critical people to ensure the smoothest transition from planning to implementation?

Melinda Karp 39:30

No, so it's fascinating. So we did not. We don't even see a theme in terms of which discipline. So in some institutions it was both math and English. In some institutions this kind of innovation was happening in only one or other of the disciplines. And I will say we have a number of institutions where it was the math department that was innovating and the English department was resisting, which is kind of goes against, like the old wives tale in higher ed, that math is a problem Like. That's not really what we found. It's very campus dependent. We have some really innovative and thoughtful math departments and some really innovative and thoughtful English departments, departments.

Melinda Karp 40:13

What we did find actually was that almost uniformly, I think in about 12 or 13 of our colleges, there was some sort of external impetus that was a forcing measure, whether it was state policy, whether it was the pandemic, whether it was engagement in something like achieving the dream.

Melinda Karp 40:24

There was an external impetus that got folks thinking that there had to be a better way, and then it was a matter of getting some folks you know, as you always say, that coalition of the willing, who are the ones willing to put their necks out to think about how they would engage the rest of the institution or the rest of their department.

Melinda Karp 40:46

And so you know, even beyond the usual, like you have to communicate and you have to make the case.

Melinda Karp 40:51

We saw a fair amount of the successful institutions and, granted, they were all successful because they all did this, but there was a through line of the leaders really starting with individual and institutional values and priorities and really making the case, not just that you know our data aren't great, but that this change, this integration of student agency, this integration of acknowledging that students have strengths that they bring with us, is aligned with our values and figuring out what are the values of both the institution at large but also those individuals with whom they needed to like, needed to get them on board, to really think about, like, how do we start with shared values and build from that?

Melinda Karp 41:36

And I thought that was a really important insight that I don't see a lot in the change literature. Right, we hear a lot about communication, data, you know, using data to make a case, but this idea that our values are about students and so we're going to do this reform, whatever that looked like, whatever sort of category they were doing, because it aligns with our values and let's talk about that and I thought was really interesting and important through mine.

Al Solano 42:05

So transformation? You mentioned at the beginning of the podcast that placement can serve as a catalyst for transformation, but it's not in and of itself right. So can you unpack a little bit about that theme of transformation?

Melinda Karp 42:22

For sure. So we had a hypothesis I want to be clear, we didn't know for sure. We had a hypothesis that you could use placement as almost like a sneaky way to get to the other stuff, to get to transformation, that there's a way to do placement that's focused on accuracy and that is really important. The way everybody teaches, that's terrifying to a lot of folks and you get resistance and that's not the way to honor the work that's being done. So our hypothesis was that if you enter the discussion around institutional transformation with something perhaps a little less scary like we're just going to change placement there might be a way to get to those other core functions. How do you support students? How do you teach them? We didn't know. It was just our hypothesis and part of that was really our funder was like well, we know we need to get to these core functions. We know we haven't been able to crack that, not yet. Can you help us see if this is a way? So one of the things that we did was we sort of built a definition or a set of indicators of what it would even mean to be transformative, because that's like a big, gnarly, amorphous word and what we were really trying to get out was to what extent does placement move beyond this notion of accuracy to start thinking about placement as a way to address equity gaps and or completion barriers, as a way to get the college to change its underlying structures, or the behaviors that folks engage in on campus, or those norms like right, how do you use students? And or the extent to which placement has what we think of as like tentacles. So you know, placement can be a thing that just happens in the testing office, or it can have far-reaching impacts into all sorts of departments I alluded to, like IT and the SIS earlier, and so we were like, if that's happening, those are indicators that placement can and is being used for something more than a reform to the placement process. It's about transformation. So we had no idea. We tried to find colleges that looked like maybe they were doing this, but you don't really know, and it turned out that out of our 15 institutions, eight of them actually were using placement in a way that was transformative and said we're sort of the more traditional, we're going to do an algorithm to make sure the right students are in the right parts of our sequence, which, again I want to keep saying that's not bad, that's good it just we were looking at this bigger other thing.

Melinda Karp 45:03

What was interesting about this transformation case these eight colleges that were using placement transformatively is that they were using all different kinds of those categories. I went through so obviously that 80 college I mentioned. Placement was transformative for them. But we had a number of institutions that were using a reflected algorithm which, remember, is still an algorithm. It just had a little bit of additional kind of data in it than a multiple measures algorithm and a number of them were making really big transformative changes off of just that tweet to their algorithm. We actually had one college that didn't end up changing its placement system. They had thought they were going to change placement. Once they embarked down their discussion of new placement approaches, they realized placement wasn't the problem and they were rethinking their curriculum. So placement was being used as transformation because they realized they had to transform, not change placement. So there's a real wide range of how you get to this idea of placement as transformation. But it is possible to use placement reform to push these other changes, which was exciting for us to find.

Al Solano 46:20

So I want to go back to that question I had about key people and implementation. Right, because, for example, that college that said, well, we're not going to change our placement, but we're going to work on the curriculum, who made that decision? What body made that decision? Because, fine, maybe it's a I don't know a five-step process for them. But if you don't change placement and you still put students in developmental ed, even if you have 70% and this is what I tell math teachers when you do the math if you take three or four courses below transfer level and 70% of the first one pass, and then the remaining move to the next one, and then 70% pass and move on to the next class, and then 70% you're left with a handful of students and move on to the next class, and then 70% you're left with a handful of students. So curriculum changes are fine, but ultimately we got to. So I'm just, from a process and decision-making perspective, I'm wondering and maybe you don't know who it is, but I'm just wondering who makes those decisions?

Melinda Karp 47:36

Yeah, it's a great point. So two things that I want to unpack in what you just said. One, most of these institutions with the transformative work it really was, coming from either a department chair and or the department chair of, like the provost or the president, right. So those decisions, those conversations were being shepherded by folks with a fair amount of authority, at least within their departmental sphere, if not the whole institution. They also the two colleges that ended up not doing placement reform in favor of something else. In both of those instances I do not believe they had very, very long sequences, so they had already changed some curricular pieces. One of the institutions had wanted to move to a new placement system and then ended up being stymied by their state policy that insisted that they do a more traditional, multiple measures, and so in that instance what happened with the department chair was like, well, okay then what else? How can I use multiple measures to force these other discussions in my department around normative change, how we view students, how we'll teach students?

Melinda Karp 48:38

The other one was this institution that I mentioned. They were really focused on their English language learners and they had thought that they would try and rethink how they place their English language learners to place them more accurately. What they realized was that for them, placement wasn't the issue. It was the fact that English language learners are completely shut out, took it into state from credit-bearing sequences. So they shifted from placement into thinking about how they could take a more biliterate, bicultural approach to their curriculum. So they're thinking about things like can they offer key courses bilingually so that students engage in learning English while also earning college credit? So again, placement isn't really the thing, it's what are we doing correctly? But that stemmed from placement discussions, and so you're right, Like if you have four levels of pre-college credit, stepping away from placement reform is not going to help. That wasn't the instance in these institutions, partly because we were looking for institutions that had sort of done some of that work.

Al Solano 49:42

By definition, Got it, thank you. You know you should do another study of 15 shitty I mean not that good institutions and find out why they are not doing it. It's really fascinating. I tell this to a lot of the colleges that I work with because they'll say something like Al faculty don't want to do this. Or they'll say you know, administration is not for this, right. And then I say, well, look, we go to committee meetings and then one faculty usually the loudest one says you know, I don't want to do this, you know this person's against it, right. Then what happens is people stay quiet, they leave the meeting and they all put their arms in the air and go well, see, now faculty don't want to do this. Shit, it was one person. One Same thing happens with administrators, right? One dean says something in a meeting, while administration don't want one person. So it's a reminder to those listening that when you're trying to change, be mindful of that dynamic. It happens too much in higher ed.

Why shared values matter and the importance of being practical about equity in the work

Al Solano 50:57

As we wind down here, melinda, I was wondering for those that listen to this podcast. They're all over the place. You have very innovative colleges that are doing remarkable work with developmental ed reform. You have some that are just starting to work, some that are kind of midway there to work, some that are kind of midway there. Given all that rich knowledge that you have from this study, do you have any I don't know kind of more specific lessons learned, like be mindful of this or don't do this and do that? Do you happen to have some kind of lessons you might want to impart?

Melinda Karp 51:39

Absolutely, and I think it goes to what you were just saying about this idea how we not work around but work with the folks who maybe are resistant to change. So part of what we heard is what I said before, which is so much of this is coming at the work, from what are our shared values? So so much of what we heard was people who wanted to do this work, sitting with their colleagues and, instead of taking no at face value, saying let's understand, a, where our shared values are and how we can work on those, and, b, let's understand what you're resisting. So, for an example, we have one institution where you're resisting and let's create a problem salt together to get there. So we have an institution in our sample that I alluded to before. They do use a co-rec model, but they call it a late start co-rec model. And so everyone goes into Gateway English at the beginning of the semester and then, at about week three or week four, in partnership with faculty, students are given the opportunity to add the co-rec visit at that juncture. So they've already experienced the course, the instructor has gotten a sense of who might need more support and then the co-rec starts, because what they do is they leverage their college's late start calendar to offer the co-rec on the late start, which is really interesting because what they're doing is they're leveraging structures that already exist.

Melinda Karp 53:03

When they started this model, the woman who was designing it wanted to start. I want to say she wanted to start the co-rec at week four and the registrar was like absolutely, you can't do that. After having some conversations about the why and the shared values and what it goes, what goes into the leg start model and the job of the registrar, they realized that it didn't make sense to start at week four. Pedagogically it did, but structurally it didn't. So they started at week three. They came to a meeting of the minds right, creative problem solving from a place of shared value. And I think so often our institutions are so Byzantine that we hear no and we throw up our arms because we don't have the full locus of control that we would like and so many of our successful institutions. The tack they took was a this idea of let's sit down and talk about it, and b let's figure out what we can control and not let the perfect be the enemy of the good, but let's come to a solution, right? So that was, that was lesson number one. I think the other lesson is really this idea of transformation is a series of small and cumulative changes, and that's why placement of transformation has the potential to be really powerful.

Melinda Karp 54:29

So I'm thinking of a different college in our sample, in a really rural and conservative state, that used a reflective algorithm. You can't talk about equity in this state in the terms that we are accustomed to in some other states, like you just can't. And frankly, the lead in that college was like listen, no faculty member wants to go to a meeting and hear how bad their pedagogy is, that they're not equity minded, that they're not doing it right, like no one wants to hear that. So what she did was when they started to explore this notion of a reflective algorithm, which, recall, includes things like putting in a survey about what students might have in terms of basic needs, requirements or their experiences with English. In this instance, she sat with her department, she sat with her colleagues, she sat with her colleagues and they went through the survey together. She was like, well, which questions do you think we should include? And when they say, well, I want to know if they have access to a computer, she would say, well, why, why does that matter? And then back into a conversation about let's talk about basic needs. Let's talk about how you support and teach and use appropriate pedagogy with students who may be in rural areas, who don't have access to stable Internet, right? So rather than starting from the equity conversation, they started with a what do you need? And then back into it and led departmental discussions about equity minded pedagogy that way and led departmental discussions about equity-minded pedagogy that way.

Melinda Karp 55:58

She will tell you that, as a result, over the past five years, the culture of their department has changed. She was able to institute new hiring requirements. After a couple of years, they started to shift who they prioritized when hiring, from the professor of rhetoric to people who understood community college students. She was able to institute early light touch assessments, which we know are good practice. Over time, she shifted the culture and structure of the department by starting from a place of let's talk about why you care about this in the algorithm and let's together learn how to support our students. So she transformed it, not by being like we're all going to transform, but she started in a small space and I think that story is really resonant too from a really practical tactical way. You run those departmental meetings. You have that control. You may not control your whole college, but you control what we talk about, right? You control how you discuss equity in your meetings and how you discuss pedagogy unfortunately, because they're my allies, we're all part of the same team.

Al Solano 57:20

I see this too much of my racial equity colleagues who think that they can go to this institution you just mentioned and unapologetically call them white supremacists and think that that's going to change them. It doesn't work, and I've gotten shit over the years for not starting first with equity, not starting first with equity, and I'm like man, you just don't get it, do you? You have to have them do some of the work and have discussions and onboard them to the equity work. The outcome is equity. The approach matters so much. An approach is going to be different in the Central Valley of California, as it would be in the Midwest, in a rural college, as it would be in San Francisco, new York. Right, you have to be really practical and smart of how you're going to approach this equity work, and so where you know that the language triggers, you can still get them to do equity work. You just got to have a pedagogy to it, right? So thank you for sharing that. I really appreciate that.

Melinda Karp 58:21

Yeah, and it comes back to those shared values, because I refuse to believe that the majority of community college instructors don't care about the success of all of their students, right? So if you start with that shared values, you will get to equity in most places. Right, because it is to me. Doing equity work makes sure that every student is able to be successful, regardless of where they come from or what they started with. And if you come with that shared value and you start talking about how to go, you'd see who are our students and what do they need and how do we teach them. That's the conversation that this algorithm or this placement reform enabled colleges to have.

Melinda Karp 59:01

So it wasn't going like this is just a technical change to placement. It's why do we need to change placement? That opens you up to all of those critical, transformative conversations. Who are our students? How do we teach them? How do we engage with our colleagues in student services? How do we structure our institution? Not because placement is broken per se, but because we need to talk about who our students are and how we get them to the finish line, whatever that finish line might be. For that, Right.

Al Solano 59:29

We can get people to do equity work without them knowing it, and I think that's one of the reasons why there are some states where legislatively, culturally, they're getting rid of DEI right, and I've worked with many of these states. But what I find really, really fascinating, because when I work with teams, we dig into the data by department and what I find is that some of these states like I'm thinking of Florida, and it varies, right, so I don't want to make a blanket statement that this is the average, but I find so many programs where there's anti-DEI efforts, their outcomes for students of color are better than, let's say, California. You know why? That is Because they do the work they just do the work.

Melinda Karp 1:00:23

Yeah, and I don't want to shy away from the fact that, look, assessment testing has a racist past.

Melinda Karp 1:00:28

So we can't deny some of our histories.

Melinda Karp 1:00:32

And I think it's particularly important to go back to what we talked about at the beginning.

Melinda Karp 1:00:35

This moment in time we have to think about how do you, as you say, do this work in ways that invite people in, because otherwise we won't get to do it at all? And so this idea of placement as transformation, as placement as a tool or a lever to push folks to think more critically about their institutional structures and practices, to push folks to think more critically about their institutional structures and practices, with equity in mind, however a state defines that feels really promising right now, because we have to be honest, that there are people who want to do this work, who can't say it out loud and the work doesn't end right. So the strategies we uncover, the idea that in some ways, someone said it's all like sneaky. This placement, this transformation, it's a sneaky way to get to transformation, and that's not meant to be like a bad sneaky, it's just it's quieter and that's okay, because we have eight institutions in our sample who quietly created transformation on their campuses merely by saying we're going to change how we place students into their classes.

Al Solano 1:01:40

That's right. Just again, just do the work. There's just too much talking. The 15 that you're able to find is because they move beyond the discussions. Let's have more discussions. Let's look at data. Well, I need more data Now. I want to know what high schools our students are coming from. Like that matters. Like, what are you going to do with that data? Well, I still have concerns. Well, let's plan to plan and let's do more planning. And you're familiar with my three-month rule, right, which, when you want to get Excited all the time.

Al Solano 1:02:20

You only have three months in a year to really do major priority work at a campus. That's it. It's three. It's usually october, uh, maybe part of march and maybe part of april. It's not even three months altogether.

Melinda Karp 1:02:28

Right, so you got to overcome that three, but they're doing the work yeah, and, like you know what are some other tangible findings I will say one of them is to your point that we don't have a lot of time and we don't have time for more committees, these colleges did not let a good crisis go to waste and I hate to describe anything good to COVID, but a lot of them, it was COVID Like they had been speaking for a long time and then they didn't have a choice, and so I think for folks who want to engage, to do this kind of work as well, your point about no more committees, yes, and I would add, find a crisis that you can leverage, because that really is the impetus in many places to stop talking and start doing, because you don't have a choice. Whether that's policy, I hope it's not another pandemic, please, but something, because it was almost every institution on our campus that had an external kick in the rear end that made them move from talking about it to acting.

Al Solano 1:03:29

Yes, and it took a pandemic to get rid of wet signatures. Yeah Well, melinda, if you don't mind, I want to ask you one last thing to tie this up here. I want to go back to what you mentioned earlier, how you've been reading outside of the higher ed literature. You're expanding the ideas, the thoughts, the writings of other people, of other fields, other fields. Is there anything in particular that you want to bring back from exploring all that other literature?

Melinda Karp 1:04:05

that ties into what we've been discussing. Yeah, and I'm not sure if it ties in completely, because I actually find this study really optimistic in this idea of we can transform, even in difficult moments, the fact that these institutions can use placement to do bigger things. But I just, you know, I work with colleges all over the country and I am just mindful that it is August of 2024. And, as you said in the intro, I am the granddaughter of refugees who fled Hitler's regime, and so I want to go back to that. I think a lot about that book. I mentioned American Resistance.

Melinda Karp 1:04:39

It's by David Roth, and and it talks a lot about how do you make change when you're inside a system that feels not great. So he actually writes about the Trump, the first Trump administration. I guess I have to call it that now, as there's a chance it will be a second and, and it's not about politics per se, it's about what do you do when you are a public official and others and you are constrained in what you can do to fight against something you don't believe in, and I want to name that, for many of our colleagues this moment feels that way. So, as optimistic as our study is, I think there are many folks who are feeling really beat down, and I come back to this idea of do you stay or do you go? Do you fight or do you not fight? What does fighting look like when you can't wave your arms around and scream that you hate everything about what you're being asked to do? Right, and it's about American resistance, and I think folks should read it, partly because it is just amazing.

Melinda Karp 1:05:49

On where are those lines is how do you work within the system and when do you decide that the system has gone too far?

Melinda Karp 1:05:57

Right, one of the messages is at some point there will be a red line and you should know your own values, and so I don't mean to get on a political soapbox. What I want to tie this in a bow is by A naming that some of your listeners may be struggling right now with where their line is, and B many of your listeners are probably struggling right now with just the constraints of higher ed, and there's a lot to be learned from other sectors around how you can be a good actor even within a system that doesn't always feel good. Our students deserve it. I think, again, the study shows the ways that our students can be supported in lots of different ways, but I do want to name that this is not a great, super fun moment to be an equity-minded professional in higher ed, and so I have really taken heart in some of these more political books not about their politics with a small P, but more about just how to navigate complicated, messy things that don't have clear answers thank you for sharing that.

Al Solano 1:07:05

For colleges to achieve student success, college educators need to come together and help each other be successful, and in these kinds of times so so really appreciate you again, melinda, really thankful for the study that you've completed here on placement, and thank you for the podcast we did on holistic student supports. I'm looking forward to having the show notes with links to your resources and everything. So thank you so much for participating in the Student Success Podcast.

Melinda Karp 1:07:42

Thank you for having me twice. It's an honor and it's always so much fun.

Al Solano 1:07:48

Thank you for listening to the Student Success Podcast. Each episode has show notes which include helpful links and necessary follow-up information. To help you get results, please consider subscribing to the Continuous Learning Institute website. There are no advertisements, thank you.