Salty About Medical Education: Bryan Carmody on What the System Gets Wrong

Episode Summary

Pediatric nephrologist, medical educator, and "Sheriff of Sodium" Dr. Bryan Carmody joins Drs. Koka and DiGiorgio to challenge some of the most persistent narratives in American medicine. From the AAMC's physician shortage projections — which Carmody argues serve the interests of medical schools more than patients — to the mechanics of the residency match, application fever, ERAS pricing, and the largely unrealized promise of pass/fail Step 1, Carmody brings his characteristic data-driven skepticism to each topic. The conversation closes on what's arguably the most consequential question: what should residency selection actually be optimizing for, and why are program directors squandering the leverage they have to drive real change in undergraduate medical education?

Chapter Markers

00:00 Introduction

02:02 How Carmody became the Sheriff of Sodium

05:03 Why people keep getting medical education wrong

07:46 The physician shortage: skepticism and incentives

09:03 Rebutting the AAMC's 86,000-doctor shortfall projection

11:17 Supply-induced demand and the limits of training more physicians

17:06 Third-party payment, discretionary care, and the real drivers of access problems

20:27 Who benefits from the physician shortage narrative

26:36 GME funding: $45 billion, hospital incentives, and the case for or against it

30:01 The Match explained: history, origins, and why it exists

35:22 ERAS, NRMP, and the financial architecture of residency applications

40:21 Preference signaling: what it is and why it's quietly capping application volume

44:12 Is the Match a monopoly? The congressional report and the anti-competitive argument

51:18 Step 1 pass/fail: the promise, the timing, and why it stalled

55:43 What actually changed — and what didn't — after 2022

58:00 What program directors should be demanding — and aren't

01:08:12 What we're not doing well in resident selection

01:11:59 Using selection systems to elevate the quality of every applicant, win or lose

01:18:45 The neurosurgery combine

Co-Host Handles

@anish_koka and @drdigiorgio

Show Handle

@drsloungepod

Subscribe Links
Spotify: https://open.spotify.com/show/44vw8eirsKKnjgNIrdDvrR
Apple Podcasts: https://podcasts.apple.com/us/podcast/the-doctors-lounge/id1832097658
YouTube: https://www.youtube.com/@TheDoctorsLoungePod

🔗 Connect with the Hosts:

• Dutch Rojas on X

• Dr. Anthony DiGiorgio on X

• Dr. Anish Koka on X

• Dr. Dan Choi on X

• Dr. Sanat Dixit on X

Transcript

SPEAKER_00 0:00

You like your healthcare plan? You keep your healthcare plan. Nobody knew that healthcare could be so complicated. It finally beat Medicare. This is the Doctor's Lounge.

SPEAKER_01 0:16

We are live, and uh this Thursday we have the pleasure of having Dr. Brian Carmody. Dr. Brian Carmody is our it's a repeat from the famous Akkad and Coca show. I was just talking to him before we went on air live that I don't think we've had any guests from that panel that we had on here. We had a wonderful time discussing all things, well, many, many things, including the match when Dr. Akkad was here. And now we have you back, and you you you've taken opinions on lots of other things. So we're uh we're very uh interested to hear about those as well. But just to introduce everyone to the um everyone to you, um you're a pediatric nephrologist who practices at children's hospital, the King's Daughters in Norfolk, Virginia. Is that right? That's correct, yeah. You teach second year medical students, you're an associate program residency director at the Macon and John Joan Brock Virginia Health Sciences Center at Old Dominion University. Um you're a pediatric, you're self-described, pediatric nephrologist, medical educator, associate program director, husband, dad, Virginian, and self-program proclaimed Sheriff of Sodium. You have a blog and YouTube channel at the Sheriffofsodium.com. Your your handle on X is at JB Carmody and the tag with your tagline being salty about medical education. You're you're extremely data driven when it comes to analysis of medical education. So the match, physician shortage claims, uh, USMLE policy, residency compensation. So all of that stuff you very much is in your lane and uh you bring a lot of data and some great analytical, high-level critical thinking to it. So uh so super happy to have you. Welcome to the Doctor's Lounge.

SPEAKER_00 1:54

Thank you for having me. Yeah, that's a very kind introduction.

SPEAKER_01 1:57

Um so let's so tell me uh tell me a little bit about yourself um first. Uh how how did you how did you arrive at this uh place where you're you know writing as you are for um uh about all all things, all things that are not just nephrology. How did how did this happen?

SPEAKER_00 2:14

Well, you know, I I'd say it happened organically, because like you said, I I mean, when I'm not taking care of patients, I spend a lot of time doing medical education stuff. And um so I see what it's like for the medical students. You know, I I'm involved with our residency program. So so I feel like I sort of see both sides of a lot of the issues in residency selection and medical education. And um, and as you suggested, I'm not short on opinions. So a few years ago, I started sharing some of those opinions on social media and on my blog. And um, you know, not everybody agrees with what I say or what I put out there, but I feel like I've had the opportunity to speak into some of these debates and influence people's thinking in a way that moves them toward a more correct understanding of these issues. So I've kept up doing it, you know, because of that.

SPEAKER_01 3:04

You're being you're being very you're being kind. I'm very humble, I should say. I I think what some one of the things that seems to drive you is that the common easy narrative or the common easy analysis or the common summary of the latest controversy or in in medical education, because that's what you know a lot about, doesn't quite capture the real story. And it feels like that's why you're compelled to kind of like, you know, give give your own view, right? Because if everyone was saying what was what was right, you probably you probably wouldn't say much. But but you're like, no, I know there's you're doing the meme. There's someone wrong on the internet.

SPEAKER_00 3:40

Well, no, I think that's true. I I I try actually not to chime in on things where um the stuff that I would say other people are already saying. I don't, I don't feel that you know I need to to you know to to be in every debate. But certain things that seem misunderstood or um, you know, intentionally misunderstood, there's ones where I feel more, you know, more compelled to um, you know, to to speak up.

SPEAKER_01 4:05

So as part of that, you take these claims that are made online and become popular and some viral, and then you kind of stress test them. What's the most common reason why you think everyone is getting it wrong, just from on a meta meta level? Like why are people always getting things wrong?

SPEAKER_00 4:21

Well, I think i in many cases, people are incentivized to see things a certain way. They're incentivized to um to to um to analyze a question in a particular way or to see a certain side of an issue. When they start talking about it, some people will accept that line of thinking without sort of subjecting it to you know the scrutiny that it isn't deserves.

SPEAKER_01 4:45

Yeah, it's it's such a that's so right on uh that it is very hard for even well-meaning people to look beyond their particular group interests. So residents have their own residents have a bias when it comes to evaluating the match, medical students have a bias, program directors have a bias, but beyond all that, there is like objective truth. And the question is can you look beyond those biases? How are how are you immune to it? Perhaps you are similarly just No, I'm sure, I'm sure that's true. You're a nephrologist, so you know it's like the diuretic question that me and nephrologists get into all the time. It's like clearly I am correct about more fluids.

SPEAKER_00 5:23

So yeah, so I I mean I don't say that that's true. Uh you know, I'm a I'm a practicing physician. I uh I'm a pediatrician, I'm a you know, I'm involved in residency program leadership, I'm involved in medical education. I mean, I have some skin in those games. I mean, it's true. So I mean, I try to be objective, but you're right. I mean, we all come from a certain place and have a tendency to see things in, you know, ways that are probably favorable to those other interests. And I think you have to you have to be critical with your own views too.

SPEAKER_01 5:50

Yeah, and I and I think one of the one of the one of the ways to identify objectivity or folks that are escaping the perhaps echo chamber that or the confirmation biases that they may have is to see if they've changed, if they've actually they've have you changed position on something over time? Because that to me is is is a is a good kind of a good metric. So it brings us to physician shortages. What and I think uh you have a very similar opinion to certainly to me in terms of this constant discussion about physician shortages. So do we have a physician shortage?

SPEAKER_00 6:26

I I'm pretty skeptical of a lot of the rhetoric and and even some of the data that that that's true. It's in many ways, I think the physician shortage is the problem that justifies any policy solution. Anything that you wanted to do already, the physician shortage and looking in that direction justifies doing it. And so you see it invoked to support policy positions that don't in any way really get fixed or situations that get improved if you just train more doctors. So yeah, I'm skeptical of a lot of that. Now, I mean, that's not to say, I mean, I could be convinced that there are true physician shortages in certain areas or certain specialties. I mean, I I'm not saying that's um, you know, something you couldn't convince me of, but I am very skeptical, the broad claims that you know we need to do this and that and the other thing to fix the physician shortage. You know, I I doubt most of the things that it was guest talked about or would would would contribute to fixing any of those problems in any way just by training more doctors.

SPEAKER_01 7:24

Um But Brian, AAMC predicts a shortfall of 86,000 physicians by 2036.

SPEAKER_00 7:30

Check mate. So yeah. Um you want me to opine about that a little bit?

SPEAKER_01 7:35

Yes, yes, please. No, I want you to I want you to rub about that, yeah.

SPEAKER_00 7:38

So well, I guess to to piggyback upon the discussion we were just having about, you know, people and organizations have incentives to see things a certain way. Well, the AAMC is an organization that exists to advocate for the interests of medical schools, which are businesses that are gainfully involved in producing more doctors. It's very understandable that you would see many problems through the lens of what you produce. I mean, you're you're the you're the carpenter with a hammer, you know, every problem is going to be a nail. That's that's sort of, you know, from from sort of first pass, I think you should subject those figures to some scrutiny. But secondly, if you actually look at the numbers, I mean, it's hard to decide how many doctors we need. I mean, it's easy in some sense to analyze and say, well, here's our population, and we can predict with some certainty that the population is going to increase at this rate. And here's our training pipeline. And we can again predict with some certainty that if things continue as they have, we'll increase the number of physicians at this rate. And look, there's a mismatch between those things. I mean, you can say that, but what doctors do and how doctors practice and where doctors practice is um, you know, those are things that are not really answered by that analysis and how medicine changes over time and what medical care people are really going to need, all those things are harder to assess. And so when people talk about the physician shortage, I think the better question to ask is first of all, I mean, what what are the data that make you think, not just some number from the AMC, but I mean, what what tells you that there is a physician shortage? I mean, what are the data? What are the problems that you see that say we have a physician shortage? I mean, what would you what would you say to that? Or what would you say are the common things that people might say to that?

SPEAKER_01 9:20

It's difficult to find coverage if you're some physician out somewhere, or from a patient standpoint, you know, if you try to see a primary care doctor, you can't see a primary care doctor, psychiatric wait times are forever. So Right.

SPEAKER_00 9:31

Yeah. Access to care times are long. Yeah. Right. Yeah. Yeah. I mean, yeah, go ahead.

SPEAKER_02 9:36

No, but I was just going to echo Anisha's point that, you know, it's really patients come in and say, I have to wait six months for primary care. But we don't know what's the appropriate amount of time to wait for for primary care. Should everyone have access to a next day, same day appointment for primary care or next day, same day appointment for interventional cardiology or neurosurgery?

SPEAKER_00 9:53

Yeah, right. Yeah. So I think that gets at one, you know, a common thing that doesn't get considered, which is that, you know, there is an element, I think, and I think this is established empirically, of supply-induced demand. You know, if you, if the more accessible doctors are, the more care people get. And that that can be good or it can be bad. It's it's not, it's certainly not an unmitigated good. You know, it's like building more roads in, I don't know, the DC suburbs or Southern California. I mean, you you have just as much gluten grid lock with more people on the road. You know, you you keep increasing the capacity and it incentivizes more people to use the capacity that you've created and people are sitting in traffic just as long. And I think that that's one, that's one important thing to think about. And then I think, you know, when you when you think about any of these problems, I mean, it's true. Um, often struggle to access a physician, especially a specialty physician, or there are places that would like to hire more physicians than they have. But if you think about why those circumstances exist in the first place, I mean, why do those problems exist right now? You know, I mean, if these jobs that people are trying to fill, if they're so great, why aren't people taking them right now? You know, why do these problems exist in the first place? Why is it that the primary care physician has such a well, a long waiting list? I mean, and when you drill down those problems and you think about the incentives in play that that lead people to behave the way they do in the system in which we currently have, it's not clear to me that just training more doctors is going to fix any of these problems. Or if in fact people who come to practice in this same system will see the same incentives and will practice the exact same way as existing doctors do. And we haven't really fixed the problems that we sought to fix in the beginning.

SPEAKER_02 11:31

Can I uh just push back on a little point there about this supply this idea of supplier-induced demand? I think it's something that comes up a lot and I think it's worth drilling down to because I think medicine unfortunately gets a little bit fairly unfairly diagnosed with this idea that more doctors create more demand. You know, we don't talk about restaurants creating more hungry people, we don't talk about lawyers creating more lawsuits, even though I think there's probably a bigger argument that there's supplier-induced demand from the lawyers than you know, from the neurosurgeons. I might be a sociopath, but I'm not out there tripping old ladies to give them subdurals so I can get myself more business. You know, my wife's an RDGYN, she doesn't have any control over whether or not her patients get pregnant. Right. So it this idea of supplier-induced demand, I don't know what why do you think medicine gets labeled with that, but a lot of these other professions don't.

SPEAKER_00 12:17

Well, I think um I think there is a lot of discretionary uh, you know, work in medicine to sort of use my own practice. I mean, so I'm a pediatric nephrologist. I mean there's a certain number of children who develop end-stage kidney disease. That number is generally fixed. I mean, you could train more pediatric nephrologists and you're gonna have the same number of dialysis patients. That that's true. But what's less clear, I think, is sort of on the edges of my practice. I mean, when I see someone who has whatever condition, I mean, you name it an outpatient condition in pediatric neprology, there's not really any evidence to say, oh, I need to see this patient back in a week. Or actually they can wait a month, or maybe they could wait three months or four months. And there's not really any data or any good evidence on that. And my decision about when they should follow up probably does involve some amount of how easy it is to get them scheduled at that time. And I think that's probably true. And there's lots of other examples, you know, we could we could go through, but I think lots of people, if you made it, I mean, if you had a physician stationed outside your door, so that as soon as you walked outside your home, you were confronted by a physician that was available to assess anything that you might want assessed. You can imagine that many days when you walk out your door, I don't know, my neck kind of hurts. You know, maybe I've slept wrong, but I don't know. Maybe it's maybe I should get this checked out, you know. I mean, and this is an extreme example, but I think the the premise here is true that a lot of things people maybe don't need to be seen for. I mean, not to sound cruel, but you know, some things, some patients that wait, as you suggest, it may be because they can wait. When someone needs to be seen in my clinic, it doesn't matter if I have to double book them. It doesn't matter if I have to stay late. If they need to be seen, they're gonna get seen. But someone who's being referred for a more marginal indication and they get an appointment that's a couple months down the line and they're feeling better and they decide not to come, I'm not sure that that's a bad thing. And and I mean, of course, the problem is that patients aren't always in a position to know. But um, but yeah, I do think that medicine is a little bit different than some of the other industries that you you that you mentioned.

SPEAKER_02 14:16

I mean, I I think I'd argue that's more of a byproduct of our third party payment system, right? For the example of the doctor outside the door, if he says you, your neck hurts, you want to get it checked out, it's gonna be 50 bucks or, you know, and then if you want the MRI, it's gonna be a thousand dollars, right? The the fact that so much of what we provide is actually free, right? I have a discussion with my patients. You want to come back in two weeks and four weeks, you know, to get your repeat scan, price doesn't come up, right? And I think again to your point, a lot of those marginal, and we have data from this, the RAN study, the Oregon study, a lot of those marginal cases will seek more care if the cost to that person is driven down. So I agree. I think I think it's complex, it's nuanced, not to get too far off topic, but to me it seems like a lot of it is is this third-party payment system that drives that. Right. No, I agree with you. You know, Anisha and I are are contractually obligated to go and disparage the third-party payment system at least once per podcast here.

SPEAKER_01 15:05

Aaron Powell, but the original sin, third party payment. No, but Ryan, such a great point you made about uh the fact that, you know, we talk all the time about how medicine, how how the practice of medicine in some ways is broken, how we're all becoming cogs in this large wheel and there's burnout, and you know, Fridays are spent doing epic inboxes messages. And it's like, okay, why exactly tell me why we had the we had the survey that I know I I was to you know, it's one of those it's confirmation advice, right? I saw the survey about like p physician nutrition and stuff, and I really didn't read the permanente study. Yeah, yeah. I didn't really read the thing.

SPEAKER_02 15:36

It's a little overstated, I agree. Yeah, fine, fine, it's overstated.

SPEAKER_01 15:39

I really didn't read the thing. I was just it just it just it just, you know, you feel it in your bones when you hear when you see like physician attrition. It's like, oh yeah, absolutely. And it's of course the denominator's wrong and blah, blah, blah. But but but the point is real. The point is still is still a good one. I still I still kept my retweet, okay? Because the point is still a good one in the sense that, you know, there's there is there are a lot of physicians that are that are that are leaving, that aren't practicing for as long as they thought they would, that are, that are going from full-time to part-time, right? So what exactly and it's such a great point you make, Brian, what exactly is the point of feeding more people into this machine that's basically, you know, not working, right? So maybe if you if you want to it may make a lot more sense rather than running around talking about uh increasing the physician labor pool, it may make a heck of a lot more sense to fix some of the drivers of what makes medicine less appealing for the folks that are here. So you so you they can they can work more. Or hey, how about we make things more efficient so the physicians that are practicing can see more patients, right? I mean, Dr. Sorry, some unnamed neurosurgeon in California is always telling me about the inefficient system that that that makes it difficult for him to like you know turn over an operating room, right? He he should be able to see like seven or eight people instead he's doing like two cases, right? And and that, you know, is something that's not considered. We just keep banging this drumbeat about we need more doctors, we need more doctors.

SPEAKER_03 17:02

Right.

SPEAKER_01 17:03

So it's great. What so what what so what does one who exactly benefits? Would you say there are a bunch of players that benefit from this you know idea that there is a physician shortage?

SPEAKER_00 17:12

Oh, yeah, sure. So uh for one thing, I mean, medical schools benefit. If you if you want to play a parlor game at home, uh I double dare you to Google up from your local area or any new medical school um and read the news article about it. And I can just about promise you, it will it will say the reason that they're making this school is to fix the physician shortage. That's why they're doing it. Of course, medical schools are also nice things to have for universities. You know, they're um, you know, they're they're a prestige marker. They um, you know, bring in students that have good access to federal loans, even under the new policies. Um, you know, a common thing that we hear is we need to have this medical school in this rural area because it's an underserved area. Uh, you know, and then the physician shortage, remember we have this physician shortage. And surely if we train people here, they won't respond to the same incentives that other doctors do and make them want to, you know, do direct primary care or become a cash-only dermatologist in Miami. They'll want to stay here, surely. But of course, also putting schools in those areas, you have much lower structural costs to put a school in such an area. And it's a much bigger sort of relative improvement to the local economy. It's much easier to get politicians on board. It's jobs, it's good jobs. You know, it's it's people coming in and spending money. It's a nice thing to have. So, you know, everybody wants a new medical school. And of course, we've seen a lot of new medical schools being developed. I think people, as you, as you highlighted, I mean, physicians aren't by and large are not out there, you know, making their own practices anymore. I mean, they're employees. And, you know, in desirable areas, I think we already see the the effects of, you know, it's harder to get a good job as a psychiatrist or emergency room physician or something in in some areas that are already sort of saturated with good jobs, you know, that's that benefits the employers, though, you know, for sure. So um, you know, talking about the physician shortage certainly, you know, helps that.

SPEAKER_01 19:15

Do you think I've made the point that we have too many residency spots? And I made that point in response to the idea that we have so many residency spots, we don't have enough medical schools. It's like the reason, you know, we need more medical school students graduating is to fill these residency spots that we have. Do you do you agree with me that we may have too many residency spots that the res the number of residency spots we have has little correlation to demand that actually exists for doctors in the population?

SPEAKER_00 19:41

Yeah. Well, I'll tell you, you'd be hard-pressed to think that we have the right number of residency positions, because if we did, it would just be totally by chance. It's not as if there's any sort of central planning authority that's reviewing data and saying, you know, we need this many doctors and this many specialties and we got to build up the pipeline because we anticipate this need. That of course doesn't occur. The decision about whether to add a residency position or create a new residency, it's entirely driven by hyperlocal factors at that hospital. If you have the patience, if um, you know, it would be more efficient for the way that your hospital is set up to have a resident rather than a physician extender of some kind. If having residence would enable a generous federal subsidy, well, then you have a resident. Nobody, you know, you don't have to, like I say, there's no central planning. So if your position was, yes, we have exactly the right number of residency positions, we would have really had to arrive at that just by chance because there's not any mechanism other than individual hospitals pursuing their own incentives to give us the system that we do have.

SPEAKER_01 20:47

But I mean, uh do you really think a central planner would be able to get this correct?

SPEAKER_00 20:50

No, I don't think so. I think it'd be hard. I I wouldn't say it would be worse, but it would be hard. But I still think it's probably worth doing because we spend so much subsidizing graduate medical education. And I think that the public, I think the taxpayer gets so little return from that investment. I mean, really, much of our GME funding functionally serves to subsidize indigent care. It's not a clean or direct way of doing that. And I don't think it's the best way of doing that. And I think that, you know, if you were looking for an opportunity for reform, reworking the GME payment system to, you know, to try to inspire hospitals to train residents in the way that we want or in a more innovative or rigorous way, or in a specialty that we need, or to to train doctors that are actually going to work in areas that we say we want them to work. You know, the the funding system is a powerful lever that you could do a lot of stuff with. And really we don't. I mean, hospitals get money for training residents by virtue of being a hospital that trains residents.

SPEAKER_01 21:48

What you don't think that there would be a lot of influence brought to bear that had to to the person making those decisions? Yeah. I mean, it'd be you know, forty five billion dollars large health systems.

SPEAKER_02 22:00

Yeah, jockeying for more.

SPEAKER_00 22:01

We're talking about, yeah, I mean, but but but you're we're you know we're talking about yeah, I want to say it was like fifteen billion dollars or something like that was the last time that I looked at it. Um but I mean we yeah, we're talking about a lot of money. We're talking about a I mean a big lever, and you're right. Lots of people are very interested in it.

SPEAKER_01 22:14

And picking up 45 billion, right, a year?

SPEAKER_00 22:17

It maybe more than that. Yeah.

SPEAKER_01 22:19

So uh what about what about eliminating what about what if we eliminated GME funding?

SPEAKER_00 22:23

Aaron Powell It would hurt hospitals that um you know really come to have come to depend on it, especially the indirect medical education payments, the the the add-on uh money that gets tacked on to um you know to the expend you know, the the discharge payment for Medicare patients. Um, like I say, those payments in particular seem functionally to subsidize indigent care, especially at hospitals in the Rust Belt and Northeast, you know, that have trained residents for years and years, those hospitals would be very harmed by the removal of those payments, and I'm sure would advocate very strongly against it. But if you chose not to subsidize graduate medical education, if you said we're not going to do this at all, most hospitals would still train residents because training residents provides direct benefits to the hospitals even without a federal subsidy. Now, not all positions necessarily would be as profitable or even profitable at all, but I think most of them certainly would be. And it's a pretty pretty good deal for the hospital, even without, you know, a federal uh, you know, tapping into a federal stream of funding, you know, on top of what you get from the resident themselves.

SPEAKER_02 23:29

Yeah, the point I I the example I like to bring up is uh certain neurosurgery residency program lost its accreditation and had to close. And uh their eight neurosurgery residents had to go elsewhere. And in response, the hospital, I think, hired like 23 advanced practice providers, right? And those residents are costing that hospital very little because they're getting GME funding on top of that. And meanwhile, those APPs are all making probably mid-six figures, right? So just the just the labor cost alone that you're saving by being able to have residents, you know, they don't you don't have to pay them overtime, you pay them a salary. There's a lot of things that go into make that labor extremely inexpensive. And so they act almost as de facto physician extenders, right?

SPEAKER_00 24:07

Right. And and as much, you know, whining as you hear from hospitals saying we, you know, we've got this physician shortage, we need more federal subsidies. It's it's also true that even when Congress has done things to curtail the availability of those subsidies, hospitals continue to increase the number of residents that they train, which which they would not do if it weren't a money winner without the subsidy.

SPEAKER_02 24:30

Yeah. One of my biggest pet peeves is when people say there's a cap on residency funding, and I point to the graph that shows that the the residency slots, uh, the rate of of production of residency slots has outpaced population growth since the cap was instituted. So the cap, to my view, essentially had no actual function on slowing the rate of residency growth.

SPEAKER_00 24:49

Yeah, right. There's I you know I want to say there's a JAMA paper about that. And for maybe like four years in the early 2000s, there was sort of flat growth, and then hospitals adjusted and there's been steady growth ever since. So yeah, it's it's it it's it's strong empirical evidence that the federal subsidy is not necessary to make residency positions a money winner for hospitals.

SPEAKER_01 25:10

All right. Tell me about the leave physician shortages and talk about uh something that's been near and dear to your heart for a long time and what we talked to you about uh in the past when you came on the Akkad and Coca show uh was the the match. Tell me about the match and application fever. So for for someone who matched for someone who matched 15 years ago, actually, why don't you, as Anthony was saying earlier, uh could would you mind just giving us giving a brief explainer on what the match is?

SPEAKER_02 25:34

Yeah, there's a lot there's a lot of acronyms I think our our audience doesn't know, like ERAS and RMP and AAMC. I I think it's it's interesting how they're all connected.

SPEAKER_00 25:43

Yes. Um all right, well, let me give you a I guess a a brief primer on the match. So time was back in the in the old days that if you wanted to do a residency, and I guess I should say that you didn't even necessarily have to do that, but if you wanted to, you just applied to a hospital that had the kind of training program that you wanted, and they reviewed your application and they might interview you or might offer you a job. And it was kind of just like what you'd have in any other marketplace in America. But because of the hospital benefits from having residents and from training residents, you know, that 24-7 coverage, you know, that that cheap availability to staff your hospital, hospitals developed quite an appetite for having residents. And by the 90s and 1940s, um, the number of residency positions that was available was almost double the number of graduating American medical students. And in that era, unlike now, we didn't have policies that made it easy to fill positions with international medical graduates or people from different training backgrounds. If you didn't get a person from an American medical school in the match, you were probably going to have your position unfilled. So hospitals began to aggressively headhunt medical students and provide these high pressure offers, you know, these take it or leave it offers, which put position, you know, students in a tough spot because if you got an offer from a decent training program, it's kind of hard to turn that down. I mean, you got to be a little bit ballsy to say, you know, why I don't want that. I mean, that's a decent program, but I I'm hoping I'm gonna hear from a better program. But what if you don't?

SPEAKER_02 27:11

Does that not give the med students some leverage? I understand it's the take it or leave it aspect could make people I think eventually could med students figure out, like, hey, I've got leverage. Maybe I could ask for a better salary or better call protection or better amenities, things that they don't get now.

SPEAKER_00 27:26

Right. And that's a fascinating part of the story to me is that, you know, in the modern era, there's no question that that would occur. That medical students would say, Well, you know, I I might come to your program, but you're gonna need to pay me. You know, I'm not just gonna take an offer as a second-year medical student and commit myself to where I'm gonna go as a fourth year, you know, pay me more. And what's funny is that that never, and I and I've actually looked at the history of this pretty thoroughly. Um, that never comes up. And I think that that has to do with a couple of things. You know, one is the culture of medicine. I mean, uh, you know, we tend to defer to our mentors, and you know, this is how, this is how we do it. You know, you you are a resident, you live in the hospital, or you live on the hospital grounds. The the term you don't even get paid a salary. They don't even talk about salaries. You get a stipend. You get a, you know, a little book allowance and something to cover your room and board if you weren't living in the hospital. Um, so the so the mentality was sort of that we aren't getting paid. It's not that that's something that you could negotiate it. It but it's so funny to think about that today because what you said, I mean, whenever I, whenever I think about this, I have the same feeling that you did is why, if there were ever time to increase resident salaries, that was the time to do it. And yet it didn't occur. And it never seems to have really come up. Instead, what students did is um and faculty is work on this idea um, you know, called the match, which was that, you know, instead of these high pressure offers, what we're gonna do is we're gonna let res, you know, students apply to whatever programs they want and programs can interview whomever they want. But then at the end of that process, instead of directly making offers, what they're gonna do is they're gonna submit a rank list of their preferences and they're gonna submit those things to a neutral clearinghouse where the, you know, we'll use an algorithm to optimally pair up the you know applicants and programs. And so that's what happened in the 1950s, is and it's what we call now the match. So um, and it and it persists to this day, even though the market is very different, where now, if you take all comers, we have many more applicants than we have positions available if you take people from all you know educational backgrounds. So, so that's the match.

SPEAKER_02 29:34

The now the um, so the the pieces that go into it, right? So there's ERAS, which is the application service, right? And then there's the NRMP. Who controls those? Who owns them? They cost a lot of money, both of them, right? And I I think they generate a fair amount of revenue for their parent bodies. Am I wrong?

SPEAKER_00 29:52

Well, yeah, so um NRMP is the National Resident Matching Program. So that is the central clearinghouse that that works to take these rank order lists and and pair them up. So NRMP these days is is really a pretty independent organization. And they don't, and they do charge fees and and they have increased some of their fees and are continuing to increase some of their fees. But there have been times actually where I've applauded them because they had excess revenue. And unlike almost any other nonprofit, they actually lowered their prices, which which is, I think, a noble thing to do. And overall, their revenue is a fraction of what um ARES is. So ARES is the electronic residency application service. That is a product of the Association of American Medical Colleges. And for many years, this was, you know, by far the dominant application platform that applicants use to apply to residency. And this is sort of an interesting story, too, I guess. I'll I'll try to be brief. Um, you know, again, in the old days, when you applied to a residency program, there was sort of a a bit of sweat equity involved in that. I mean, you had to, you know, collect your paper applications from different programs. And the and the applications might be different. And you had to, you know, go on the scavenger hunt to get all the documents you needed from your school and give things to the dean. And all this was done through the mail. And so, you know, there was sort of these constraints sort of placed a limit on the number of programs that applicants might reasonably want to apply to. I mean, you can only lick, but so many stamps and envelopes and things. So then that changed in the 90s when we started to have computers more available. This became something that got done on a computer and eventually something that got done on the internet. And that lowered a lot of the friction in applying to residency programs. Now you could apply to any number of programs if you had a computer and a mouse and a credit card number. And again, highlighting what we talked about before, how the markets changed, we now have more applicants overall than we have positions. And in highly competitive fields, we have significantly more applicants than we have positions. And so one way that applicants can gain a relative advantage over each other is by applying to more programs than their competitors. And so what we saw is year after year, applicants began to submit a little bit more, a few more applications and a few more applications. And this is really not a good thing for the system as a whole because it meant that programs get overwhelmed with the number of applications that they receive. You know, many programs receive a hundred applications for every position that they're trying to fill. And program directors, unlike, you know, admissions staff at a university or something, they have many other jobs beyond reviewing applications. They actually have to take care of the residents that are in the program. They actually have to do all their other work. They're not just full-time admissions officers that can pour over thousands of applications in the context of a regular work week. And so that leads to, you know, uh a natural tendency to, you know, to use screening metrics, which has externalities on medical education as a whole. And for students, you know, you're trapped in this arms race where the reason you got to apply to 80 programs is because you, you know, everybody else is applying to 75. And uh, you know, this cycle continues on. And really the only beneficiary was the AAMC, because as the sponsor of ARES, they configured the pricing system such that you pay by the application. Because, you know, that's how computers work. You know, you put your stuff in there, but every time that something comes out of the computer, it costs you an extra $15, right? I mean, that's we all know computers to work in this man. Maybe we don't, but that's how they configured it. And so it became the single biggest source of their operating revenue, the electronic residency application service, more than the MCAT, which the AMC sponsors, more than AMCATS, which is the software that applicants use to apply to medical schools, more than those things combined, more than membership dues, more than everything else all put together, you know, ARS revenues accounted for more than that. And so it gives you an incentive, you know, to see things a certain way and throw your hands up and say, well, you know, we can't do anything about this application fever. I mean, you got to let people apply. Um, you know, fortunately, that's gotten curtailed a little bit naturally by preference signaling. But um, but there for a while, I mean, we we had a steady upward trend in applications and and and you know, that's the only thing that's paired it back so far. What's preference?

SPEAKER_02 34:12

So preference signaling is uh it's been so long since you've participated in the match process, Dr. Coca.

SPEAKER_00 34:18

So, well, yeah, preference signaling is a uh is is the new metagame for applicants. So starting in 2020 uh with a couple of specialties and now moving out to every specialty, applicants get to identify programs that they're really interested in by sending a preference signal. And this came from economics, actually, where historically in economics, the market for PhDs every year, the people who were finishing their PhDs and looking for faculty jobs, they would go to one big economics meeting and they would try to interview with people who had jobs. And this was a problem because, you know, universities wanted to interview the people who actually were likely to take that job. But people who want a job are incentivized to apply to all the jobs available. So they came up with this preference signaling among the economics PhDs to say, these are the jobs I'm actually really the very most interested in. So you know, that so the university doesn't waste their time and so that, you know, they they thoroughly evaluate my application. And when you get a signaled application, you know that the applicant has has given you something of value because they have a finite number of these signals. So it's meaningful to programs to know this person really wants to be here. And so if I don't want to be interviewing, you know, 10 people for every spot I'm trying to fill, I should pay attention to people that I really want, you know, that really want to be here. So anyway, uh, and signaling started in 2020, and I don't, I really don't think the AAMC realized what was going to happen. And I many people did, and and I'll be honest, I was one of them, and I tried to be a little bit quiet about it because uh I I really do think they sort of got Trojan worst with this. That's not what they say. They say that, you know, of course, they they welcome the you know decrease in application numbers. But what happened is that Odolaryngology, you know, the specialty that that pioneered preference signaling, increased their signaling number. I think they had five signals initially, and then they increased it to 25. Now, when you give applicants 25 signals, if you submit a 26th application, or if you're a program director who receives the applicant's 26th application, I mean, you can try to interview them and recruit them if you want, but you know that your program at best is their 26th favorite program. And so maybe you might just want to spend your time focusing on the giant stack of applications from people who signal to you. So when you have these high signaling numbers, they function as an application cap. Now, they don't, they're not a restraint of trade. I mean, if you want to apply to more than, I mean, you could apply to many programs as you want. I mean, you can just give AMC a direct deposit, you know, slip from your bank account. You can apply to every program, you know, but hit them all. You're you're entitled to do that. And programs certainly are not restricted. And programs certainly are free to look at unsignaled applications if program directors are bored or, you know, just need something to do. But what these things do is it now that we're several years into this, you can see, like in a special view like orthopedic surgery, they have a cat they have a signal limit of 30. If you submit applications over 30, it's like a 1% or less chance of getting an interview at the medium program, which is what you'd expect. I mean, you don't need program directors, don't need to be, you know, searching for applicants when they've got a stack of applications this big. So that has paired back um some of the application fever.

SPEAKER_02 37:29

Now, the uh go ahead, Anisha.

SPEAKER_01 37:30

No, no, go ahead, go ahead, sorry.

SPEAKER_02 37:32

I was gonna talk about uh some of the controversies around the match. It's been it's been uh targeted as monopolistic, anti-competitive, right? Again, so you've gone from this situation where theoretically, it may not have panned out, but theoretically, med students had some say over their job conditions, right? They were given an offer, they had the ability to say no, go to other places, and negotiate. With the match, if you match at an institution, you have no negotiating leverage. You have to go to that institution. Maybe there's some ways around it. But essentially, you are committed to that institution. There's no negotiation. They write a number on a piece of paper, that's your salary, those are your working conditions. That's been challenged as monopolistic, right? I believe there was a case that was going through the courts, and then that case got superseded by some legislation. Um, but then also as recently as this year, uh the House Judiciary Committee released a report on the match, causing calling it a residency hiring monopoly. And they even emphasized this sort of all-in policy, loss of negotiation, low wages, and poor working conditions. Uh, you want to comment on that?

SPEAKER_00 38:28

Yeah, there's much I could say about this. Actually, for anybody who's interested, uh, you know, one of my last videos, I think, is about an hour-long analysis of a congressional report. So it's sort of hard for me to even decide where to begin. I um I think that the NRMP um and the matching algorithm work pretty well. I think there are trade-offs to the system. There are applicants who would do better if we didn't have a match. But if you think about who would do better without a match and why they would do better, I think it may become clear to many people that maybe we wouldn't be better off as a society or as a, you know, you know, uh, a country interested in training physicians. Remember, like I said, for competitive specialties, there's a strong excess of applicants who are very qualified, who never get the chance to, you know, in your surgery, in your specialty of neurosurgery. I mean, there's many people who would love the opportunity to become a neurosurgeon and they won't get it, you know? So I'm very, very skeptical that individual applicants without a match would have any leverage over programs. You know, if if you want to be a neurosurgeon and you go to the program director and you're like, well, you know, I don't know if you saw my step 2 CK score, but if you want me to come to your program, I'm gonna need an extra $5,000. Program director is gonna, you know, do a spit take and move on to the next applicant, like in about, you know, five seconds. And they'll be almost just as good as you. They'll be indistinguishable. And they'll have, you know, maybe they'll have 15 research papers instead of 16, or, you know, uh 273 on step 2 CK instead of a 275, you know, but they'll be very qualified. And this is true really, not just at the, you know, the most competitive specialties, but if you consider the market as a whole, it's true everywhere. You can go to the, you know, whatever, whatever the you know, least desirable residency programs are. There are applicants who go unmatched, chronically unmatched, who would be desperate to work there. You know, would they take a 10% pay cut? I bet they would. Would they take a 50% pay cut? Some will. Would some offer to work for free? I bet there would be. You know, if they're if they could legally do it, they would say, please let me work here for free. You know, I so I don't think that resident salaries would improve at all. Now, is the match anti-competitive? Yeah, I mean, it's true. The existence of this matching market, it prevents certain applicants from being able to compete in ways that they would like to compete with other applicants. They would like to compete on price and they're not allowed to. But say what you will about the current system. Residents are selected, you know, when when all residents toss the same to the program, programs gravitate toward quality or whatever their definition of quality is. And we can debate about whether what a program considers quality is truly what they should consider. But you can't debate that, you know, they're they're choosing who they think is the best. And they're not influenced by the fact that someone would be willing to sign a contract right this very second, or someone would be willing to take a pay cut or say they're going to take no vacation or something. And and I just don't know that we want a system where we want people competing on those dimensions. When we have a system where, like I say, we can debate about the measures that we use for quality, but that's what when when all residents cost the same, that's what program directors use to choose their residents.

SPEAKER_01 41:44

Yeah. The the um No, it it it does seem to be a more a system that that seems to work in a in a broader way. And I agree with you. I think it works for residents in some ways, because of course there are a lot of folks that'll work for very little amounts of money at elite programs, right? And so it introduces another variable. So I mean there's it's definitely not market-based for sure. But uh does it does it protect residents? Yeah, absolutely. And I I think you're absolutely right. I think the counterfactual that you present is probably the realistic one, which in which you know people think, oh yeah, you know, we're gonna be able to negotiate and get higher. But the but the issue is is that there's a lot more, you know, residents that are vying for uh many, many specialties. And I think those folks are the ones that are that are gonna be really, really hurt. So yeah, maybe some psychiatrists, you know, some of the lesser, you know, uh sought after specialties. Let's not call anyone out directly. Yeah. Right. Some of the lesser sought after specialties may maybe Yeah, anyway. Yeah, I I think you're right. But that doesn't change the fact that I mean the question is what are we going for? We're going for a system that is best for whom? Like what is the point? Like, how would you design a system? Like w what should we be designing for the program? Should we be designing for the for the applicants? Like, what are the biases we should be considering, especially given the fact that the people that are paying for it are the taxpayers?

SPEAKER_00 43:03

Yeah. No, right. Yeah, like I say, I mean, uh yeah, GME funding and and the subsidies that go to medical schools, you know, directly and indirectly are, you know, large, and it's hard to say what the return on those investments are. Yeah, but I don't think that medical practice would be better without a match. I think it's a system that does have anti-competitive features, but leads overall to, you know, by channeling that competition elsewhere to pro-competitive benefits.

SPEAKER_01 43:29

All right. We could stay on this for a while, but I I I want to I want to talk about some other other uh things that you've brought up. Uh step one. Moving step one from a score to pass fail. Walk me through uh well, first just explain step one briefly in a few sentences and then in terms of how and how it used to be used and then what what happened.

SPEAKER_00 43:52

Well, so step one is step one of the United States Medical Licensing Exam or US M L E, which is a three part exam that's That's been in place in more or less the same form for many years. And it's been called different things, but at least since the early 90s, we've had the USMLE in its current form with three steps. And if you want to be a physician in the United States, you have to pass all three of those exams. Now, step one is the test that's taken at the end of the, well, was traditionally taken at the end of the first two years of medical school when students have spent all their time learning basic science material, you know, anatomy and physiology and pathology and biochemistry. And I'm going to test those concepts. And then students go into their clerkships and rotate through the specialties and then take step two. And then usually an internship or in residency, you take step three and can become eligible for a full medical license at that point. So the primary purpose of these tests is to inform a decision about licensure that's made by state licensing authorities. And in that sense, all that's necessary is a pass-fail determination. The National Board of Medical Examiners works to set a standard of, you know, for each of these tests that a competent physician must exceed. And if you exceed that standard, then you're eligible for licensure. But going back to the early days of the NBME, they made a decision that examinees deserved to know their score. They needed to be deserved the knowledge of not just whether they passed or failed, but by far, by how far they'd exceeded that standard. And so these tests, even though the inference that's drawn from them is binary, a score was given to applicants. And so over time, especially with the advent of the USMLE in the early 90s and this, you know, upswing in applications, program directors wanted a single metric that you could use across the board to rapidly stratify this giant pool of applications that you'd received and get it down to something meaningful or that you could look at closer. And the USMLE score fit that bill because everybody took it by the time they'd applied to residency. Step 2CK, which tests material that would be more relevant probably to almost any person's practice in residency, not everybody had taken that by the time they submitted their applications in the fall of their fourth year. So the only universal measuring stick that you had available was the USMLE score. And so because of its importance in residency selection as a screen-out metric, it it dominated the medical curriculum for the first two years. The first two years of medical school became basically a you know an exercise in test prep. Um, you know, as students worked harder and harder to get higher and higher scores on this test so they could surpass those initial screens by residency programs. So that resulted in externalities to applicants and medical schools. Um by 2020, in early 2020, the NBME announced that they were going to move score reporting for step one to pass fail. Um and so that's occurred. And so since 2022, applicants who take step one, you get a pass or a fail. And residency program directors that are interested in using that in their selection processes, that's all they see also.

SPEAKER_01 47:06

Um did you did you feel what what were your did you feel like this was going to be a good thing? You did you support it?

SPEAKER_00 47:13

Yeah. No, I I thought it was going to be a good thing, and it's been sort of heartbreaking to see it not reach any of that potential. You know, part of it, I think, was timing. You know, the the step one pass fail announcement occurred in February of 2020. And man, I'll tell you, for a couple of weeks there, um, I had lots of people messaging me and um, you know, wanting to meet and talk about how they were going to redesign this or that at their medical school to evaluate their students. And then, of course, you know, by March, we were in the midst of COVID and, you know, every dean was preoccupied with making things virtual and, you know, everything else that went along with that. And so I I think in a different time, there might have been a little bit more momentum to assessing students in better ways. But in the absence of any real effort to do that, um, the path of least resistance becomes to use the other metrics that we have and just value them slightly more.

SPEAKER_01 48:09

So now instead of instead of go ahead, Denisha. Step one, let me just use step two.

SPEAKER_00 48:15

Yeah, step two has largely taken on that role um, you know, that step one used to have as far as um, you know, uh uh unidirectional screening metric. As I said, I mean, step two is in some ways better than step one, at least insofar as the material that's tested is more directly clinically relevant. I mean, it's something that can you can easily imagine could benefit a human patient than some of the stuff that people had to learn, especially to achieve higher and higher step one scores.

SPEAKER_01 48:44

But it's been many, many years now since COVID hit. Why have folks not redesigned and is it possible given the resources that are given to program directors? I mean, this is not the NFL. You're not sending scouts out to like screen tape of how you've never seen the neurosurgery combine, Anish. So, I mean, what what are the resources available to these poor program directors that are like doing clinical medicine, taking care of their residents, dealing with whatever crap ICG me is asking them to do all the time? How the heck are they going to design something else that's more that's better?

SPEAKER_00 49:15

How many papers they've published? So all programs would directors would have to do is speak it into existence and then stick to their guns. Program directors, you know, every medical school applicant wants to imagine what the program directors want. If program directors want any number, any change. I mean, you can imagine something simple to something bigger by saying we want this, and we're gonna prioritize applicants who have this and schools who provide this to their applicants and evaluate them, you know, in a rigorous way, you're gonna be getting priority at our program. Change would occur. Part of the trouble is that program directors have not consistently articulated that. Or I think in some cases, they don't really know what they want. And and that's a hard thing to answer. I mean, we we we have sort of a criterion problem in medicine about deciding even what a good doctor is or looks like or does. Aaron Powell, Jr.

SPEAKER_01 50:09

So how would you do it? I mean, you're an assistant program director. What would you what would you speak into existence if you could?

SPEAKER_00 50:14

Well, what what is it that you want?

SPEAKER_01 50:15

Well, what is what is it that you can do? Tell me a change you want, and I'll tell you what. You're a program director. So I mean, what in in terms of the you know pediatric.

SPEAKER_00 50:22

Well, I mean, take take something simple. I mean, um, you know, an example I sometimes will say is um uh um No, it's a great question though.

SPEAKER_01 50:30

What what what do we want? Because that is a great question. You know, Greg Murphy, who's the uh who who is constantly poking the bear of residency, you know, he he's a congressman, I don't know if I forget from where. And he's constantly his big point that he makes not infrequently is that you know, we're not screening he's talking about medical school, I guess, but whatever. It it applies down down the road, and you can talk about that as well. Like he says, look, we're not screening to get medical students who want to practice long term, right? Because it gets back to this whole physician shortage issue, and he he thinks that a large part of the problem is that most folks are not practicing full time uh at a certain point. Of course, the congressman is not practicing full time either. Right. Anyway, yeah. Whatever. So so are we should we be prioritizing what should we be prioritizing for? If the idea is, hey, we have this GME funding that we're getting, maybe we should be prioritizing for people that are going to be in practice for a long term. Can we is okay, maybe that's maybe that's what we should do.

SPEAKER_00 51:27

Yeah, no, I'll say a few things. So one thing, if you think about medical education, another trend that's been consistent is that the length of training continues to increase. And it's true, medicine is becoming more complex. And so, I mean, we have some fellowships and subfellowships that exist now because we didn't used to have those capabilities or those technologies. But it's also because residency training is more compressed. We have tighter work hour restrictions and we have different workflows. And I think many people have concerns about the rigor of residency and the experience that residents get and whether they're ready to practice at the end of residency. And, you know, one solution to that is to simply add more and more training on the back end. Just keep adding more and more fellowships. And maybe you'll get to practice independently when you're 50 or something. But you know, the other solution to that would be to make medical school itself more relevant and um and helpful to you in residency. So let's just say you're a program director and you want um your residents to come in with some particular skill. And what that skill is, I mean, it may vary by um, you know, by different disciplines. Um, I don't know, if you wanted to be a pediatrics resident, you said, you know, I would really like people to know how to do the newborn exam. I mean, it's it's one of the places in medicine where the physical exam really matters because you're not gonna be able to assess many of these things another way. The things that you might find that are important are rare and you got to be sharp to find them. So I want uh, you know, my residents to start doing that earlier. And I'm gonna prioritize applicants whose schools have given them extra training in this area and rigorously evaluated their competency in this area. And if you say that, some schools, some applicants will go to their schools and they'll say, gosh, you know, I mean, in my heart, I want to work with the sheriff of sodium. So I need my school to do this. Now, I mean, of course, that's not gonna maybe happen for me, but imagine that you were at an Ivory Tower surgical specialty program and you wanted your residents to come in with some, I don't know, some skill reading a radiograph in your area of expertise or something with suit. I mean, suit you, you can put whatever you want to say. Applicants will demand that their schools assess them in this way. You know, if you stick to your guns, if you say, look, I mean, and that's not to say you can't match here if you know your school is hard-headed about it, but I'll tell you what, we're gonna review applications first from schools who have provided this, that, and the other thing. You'll see that change happen, you know? And and really, I think that's the only way that we're gonna get it because otherwise, schools have an incentive often to just get out of their students' way, to just get out of their way more and more and let the test do the assessment. Then they don't have to be the bad guy about assessing anything. Let the test do the work, let the students come to class or don't. Um, just get out of their way. Let them write their research papers, give them the degree. You're getting the same amount of tuition dollars, you know, fewer lawsuits, fewer headaches, fewer unhappy students. Just, you know, just side to the side and let the students do all this other stuff. And that that gravity is hard to overcome unless you have some countervailing force.

SPEAKER_02 54:35

Yeah. So I, you know, I took the easy route to neurosurgery, going to a DO school that had no home neurosurgery program and no real research infrastructure. But I think part of the way I was able to stand out was really having a decent step one score. And so for me, that was almost an equalizer that I think a lot of people could lean on if they didn't have those connections to what these other sort of nebulous program requirements might be, right? If you want to have, you know, have heavy-duty research application, you need to be at a at a med school that has that infrastructure. Um, if you want, you know, some of these other qualities you have, you need to have some infrastructure in place. And so I looked at step one as almost an equalizer where someone could, you know, come from a place that doesn't have all that, but still make an application that stands out with an outstanding step one score that, you know, yeah, it reflects your ability to do test taking, but also a commitment and um, you know, ability to to study and stick with it and and you know, eke out that good score. So I'm wondering, since we've gone to PassFail, uh has there been a change like in people like Dio's going into neurosurgery, these sort of like more fringy type applicants getting into these highly competitive specialties?

SPEAKER_00 55:44

That's a good question. Yeah. Because um, you know, there's there, I think there are many people who who feel the way that you do and have similar stories to yours. Um and so one thing that you often hear is that I mean, in the absence of a scores step one, now, you know, program prestige matters more, you know, school prestige matters more. And I think uh it's an appealing idea in a certain way, but I don't know of any data that supports that. And and you know, when you look back at the scored step one era, I mean, certainly school prestige mattered quite a bit then. And then the other thing to think about is that certainly there are people like you who, you know, you did well on step one and benefited by doing well. But you probably wouldn't be surprised to know the medical schools that had the highest step one scores were the most prestigious medical schools. You know, it was Harvard, it was Mayo, you know. So sometimes I think this gets conflated into uh, you know, an idea that, you know, step one was benefiting all these hardworking people at lower, less well-known schools, and then all these lazy students at Ivy League schools were getting beaten out by them. And they weren't. I mean, they were matching very well and doing very well on that test. And if you were an alien from outer space and you stumbled upon this system and you just looked at it and you said, All right, I see the MCAT scores, I've see where people sort into for medical schools, I see the USMLE scores, I see where people sort into for residencies. I don't think that you would, if you, if you had no personal experience, if you just saw it, you know, like I say, with fresh eyes as an extraterrestrial life form, I think you would probably look at it as a way of reproducing the existing hierarchy. You would, you would say these scores actually are the are the means through which we're reproducing the same hierarchy that we had, you know, because the best schools have the best the applicants and they score the best by and large. And sure, individuals, you know, maybe on other places do. But I don't know that um so you so I would say I don't know of data that that show that that's more or less likely now. I think, I think it's just different things. I mean, you you would have to, if you wanted to match in their neurosurgery now from an osteopathic school, you would certainly need to signal wisely, blow step two CK out of the water, get your name on as much research as possible, strategically configure your weigh rotations, you know, play some softball and be good at it. You'd have to do all those things, you know. But you've heard about the tournament surgery.

SPEAKER_01 58:10

Yeah. Right. Brian, so what are we not doing well when it comes to selecting residents today?

SPEAKER_00 58:17

Well, like I said, I think that program directors continue to miss the opportunity to elevate standards in undergraduate medical education, to continue to accept the limitations of metrics and say, well, you know, everybody's letters of recommendation look the same instead of insisting on standardized letters of recommendations, which don't eliminate those problems, but do improve them. And some specialties are using those now. Or saying, you know, um, you know, all these curricula are past failed. And I don't want that. I really want grades or I want class rank to be in the MSPE, but they they won't sort of put their money where their mouth is and say, make a statement, put it on your website and say, look, we think ranking program or ranking, you know, uh people's class rank is important. And if your MSPE, your dean's letter does not include that, you're going to the back of the line at our program. And and we'll see how long those those policies persist. Anyway, I'll I'll catch up.

SPEAKER_01 59:14

So, okay, so is this getting this is so so this is getting towards trying to have better prepared residents. Meaning, we want better prepared residents, and you see residencies as someone who can use the lever to better prepare residents. Yeah?

SPEAKER_00 59:29

Right. And it's not just and I think there's another point about this that's important and that sometimes gets overlooked. You know, whatever we use to select residents, some people are gonna get what they want on match day and some people are not. You know, not everybody is gonna get the chance to be a dermatologist or a neurosurgeon or, you know, train at a prestigious program. Some people are gonna win and some people are gonna lose. Um and we can use step one scores, or we can use step two scores, or we can use research, or we can use, you know, any other thing that you can come up with. And at the end of the day, you're gonna have the same number of winners and losers on match day. But that doesn't mean that every selection system is equivalent. You know, you could have the same number of winners and losers by having just a pure lottery, for instance, but that wouldn't incentivize students to learn the way that we might hope that they would in medical school. We have an opportunity to make our selection systems require of applicants that they compete on dimensions that are going to make them a better doctor. We have the ability to use things that are practical or um might benefit a real human in some way, even for people who lose. You know, just a couple a day or two ago, I was talking with um Will Flannery, uh, Dr. Gluckenfleck on his podcast. And, you know, one example that I used then was suppose that um we had a course that medical schools had to offer or could offer for applicants that wanted to go to a competitive specialty, let's say. And what it was was handling emergencies on an airplane. And you go in, you get a case, kind of like the old step two CS, except you're on an airplane, you have the resources that you have, you have that environment. Um, let's see what you can do. And we're gonna rigorously evaluate that and maybe use that in residency selection. You're still gonna have the same number of winners and losers. But now the people who won and the people who lost, maybe they're in a position where they can help someone later where they wouldn't have been if they just wrote some crummy paper that's sitting in the corner of the internet somewhere that nobody ever reads, or some factoid for step one or what, you know, whatever else. Maybe now we've instilled in them a practical skill that, you know, is there a doctor on the plane? Yeah. You know, and you can tend to the patient's hypoglycemia or, you know, coronary event or whatever in a way that you might not have had you not been inspired by the selection process to acquire those skills. And you could do that with any number of things. I mean, you could do it with ACLS, you could do it with um anything that you thought might benefit the quality of the average physician. You have the ability to elevate the winners and the losers with our selection system or not.

SPEAKER_01 1:02:00

None of that. So that that's great. Uh so you know you're defining these other metrics that are beyond just what we do now, which is primarily very test-based, right?

SPEAKER_03 1:02:10

Right.

SPEAKER_01 1:02:11

Test-based letter letters, and as you're saying, some research and papers that uh maybe no one's read, no one will read. You haven't mentioned anything about you know target demographics for classes. Is that something that's relevant? Relevant. Is that something that should be relevant in terms of like, you know, the demographic of a of a class? Should should a should a residency class have a certain demographic makeup? Is that something that is, in your opinion, given too much weight right now, too little weight? Is that something that matters? Because that is like beyond evaluating the individual, right? That's saying, okay, the class cannot be made up of just this because it's too then homogenous.

SPEAKER_00 1:02:48

Is that is No, I I don't think, I mean, I I I don't think you can pre-specify um, you know, the I I don't think we can know, you know, the ideal demographics. I think we should acknowledge that much. I do think it's appropriate and right to consider that people have different backgrounds that may provide different values in training or in medical practice, but I don't think you can conclude by any sort of demographic that there's a right number of, you know, fill in the blank for this program or specialty.

SPEAKER_01 1:03:21

Yeah. It's interesting. I I was thinking about it a little bit, partly because of this Greg Murphy tweet from a few days ago, because you were talking about the return on investment, right, for the GME and how much money we spend. And you know it's forty-five billion dollars a year or something like that. And um, you know, yet we have a fair number of trainees that stop working or go from part-time to full-time. And there's a pretty significant demographic breakdown for that. It would be but th that that would be like w would it be right for somebody like a Greg Murphy, who's a representative of taxpayers or whatnot, to demand that residents take that into account? And uh, how would they take that into account even? I mean, that yes, there's a there's a case to be made for certain demographics that do end up staying in practice and certain demographics that don't end up staying in practice, but it seems wrong to take what a group is doing and try to apply it to that individual because they are a member of that group, you know. Everything breaks down when you try to get like everything like to me, it just on principle, it just seems wrong to ever make a decision on someone because they belong to some group, whether or not there's some net utility that we think is gonna come to society or not from it.

SPEAKER_00 1:04:28

Yeah, no, I I think as you said, I mean, it's hard to pre-specify, you know, the the the right makeup of of whatever is this or that or the other thing. You know, the attrition stuff, I do, I do want to say, I guess uh it's a little hard, I think, also to say that, you know, I mean, there's good attrition and there's bad attrition. You know, I mean, the the attrition in the U.S. Marines is not, you know, zero percent. And medical school admissions committees are not infallible. If people decide, really, for whatever reason, at any point in their medical training or career, that they don't want to do this, I'm I'm okay with that. I mean, I think, and really I think that patients in society are probably better off by having those people apply their talents and interests elsewhere. And and it leaves patients instead with a slightly smaller workforce of people who maybe are more likely to want to take care of them. And that may be on balance a better thing. So, you know, there's there's attrition that's good attrition, and there's attrition that's that's not good attrition. And if you just look at the number, it can be hard to tell which is which.

SPEAKER_01 1:05:32

What if the attrition rate is predictable in a significant degree based on score? Like, for instance, attrition rate, we know attrition rates for um medical school, for instance, are highly dependent on how well you do on your MCAT. Meaning below a certain number, the attrition rate really skies. It is that is is that I mean, is that a reasonable proxy to use?

SPEAKER_00 1:05:55

Well, yeah, I think medical schools do that. I mean, like you said, I mean, a high MCAT may not tell you much more than a you know pretty high MCAT, but below a certain point, yeah, you you're probably not doing the applicant favors by letting them in. Um, there may be a gray area where if somebody has a very high upside that you perceive for one reason or another, then maybe it's worth taking a slightly higher attrition risk. But but yeah, if you if you're if you're too far below that threshold, I don't think and I and I think in matter of fact, that's how most admissions committees look at it is you know, if you don't exceed the threshold, it's probably not the right thing for, you know, for for any of us.

SPEAKER_01 1:06:36

Yeah. So it seems like we're never ever gonna get away from objective tests ever, because of you know, tests generally do seem to predict basic things like attrition rate, not necessarily who's a good doctor or not. Uh, but but yeah, I mean, uh, it seems like we need to be doing a lot better job. And that's that's a great point you're making about uh how residency programs should really be driving that uh as being an engine for change and you know, for for for that for that pipeline that's coming up. So all right.

SPEAKER_02 1:06:59

Well I've taken up to say it really sounds like you're endorsing my idea of a neurosurgery combine. So we're gonna we're gonna roll this out next year. There there will be there will be batting practice, a home run derby and suturing.

SPEAKER_01 1:07:14

And suturing. I love it, I love it. You get the suture of baseball in between. All right. Well, uh Brian, thanks so much. Sorry we ran a little bit over, but uh super interesting conversation. And of course, you you bring just a great uh level of balance and a high level of critical thinking uh to these topics. So thanks, thanks so much for uh coming on.

SPEAKER_00 1:07:32

No, thank you for your kind words. It was a pleasure.

SPEAKER_01 1:07:34

This was great, thank you.

SPEAKER_00 1:07:35

Stay on.