In today’s KERCast, Dr. Victor Montori interviews Dr. Peter Noseworthy, Professor of Medicine and cardiac electrophysiologist at Mayo Clinic. Dr. Noseworthy maintains a federally funded research program centered on improving the care of patients with cardiac arrhythmias and works extensively with the Optum Labs Data Warehouse dataset to examine outcomes in patients with various heart rhythm disorders, particularly the prevention of stroke related to atrial fibrillation. In this KERCast episode, Dr. Noseworthy breaks down the hot topic of artificial intelligence (AI) in clinical research, including how big data can inform patient-centered care, the role of clinicians in translating these findings into actionable recommendations for individual patients, and the importance of collaborative relationships to interrogate, strengthen, and responsibly apply this developing methodology.
In today’s KERCast, Dr. Victor Montori interviews Dr. Peter Noseworthy, Professor of Medicine and cardiac electrophysiologist at Mayo Clinic. Dr. Noseworthy maintains a federally funded research program centered on improving the care of patients with cardiac arrhythmias and works extensively with the Optum Labs Data Warehouse dataset to examine outcomes in patients with various heart rhythm disorders, particularly the prevention of stroke related to atrial fibrillation. In this KERCast episode, Dr. Noseworthy breaks down the hot topic of artificial intelligence (AI) in clinical research, including how big data can inform patient-centered care, the role of clinicians in translating these findings into actionable recommendations for individual patients, and the importance of collaborative relationships to interrogate, strengthen, and responsibly apply this developing methodology.
It's time to start the KERcast brought to you by the Knowledge and Evaluation Research unit at Mayo Clinic. I'm Victor Montori, your host for the KERcast. And today, my very dear friend, Peter Noseworthy is our guest. Peter is a professor of medicine a cardiologists, a cardiac electrophysiologist. He does ablations to take care of cardiac rhythms that cause disease. And he's also a federally funded researcher centered on improving the care of patients with cardiac arrhythmias. Peter is one of the pioneers if I may, of using artificial intelligence for practical applications, both in the care of patients with cardiac problems, but also in, in research itself. So I hope that in the next hour, we're going to spend some time figuring out, figuring out what this is about. Peter, welcome to the KERcast.
Peter Noseworthy:Well, thanks so much for having me. It's great to be here Victor, I always enjoy the opportunity to talk to you. And thanks for the invitation.
Victor Montori:Delighted. Um, how does one get to be Peter Noseworthy?
Peter Noseworthy:Well, that sounds like a memoir question. I have to admit. At the age of 41 I'm always a little turned off when somebody starts writing memoirs, I feel like I'm just at the beginning of my career and trajectory and just figuring it out. But I'm happy to tell you how I got where I am. Even if it means I, you know, hit a bunt and I'm on first base, but I'm happy to take you there.
Victor Montori:That was a baseball analogy.
Peter Noseworthy:Yeah, I don't know, I don't, I don't actually know anything about baseball. So that's gonna be where my sports analogies end for today. Well as you said, a fair amount in the introduction about who I am and what I do. But at my core, I'm a clinician, I'm a cardiologist, I take care of patients who have problems with heart rhythm disorders, but I like to try to keep a mix in terms of the way I spend my time. And I like to have efforts in research, and administration and so forth, that all sort of center around the key problem of dealing with patients who have cardiac arrhythmias. And there's some interesting problems there are things like risk of stroke and risk of sudden death, and also management of symptoms. So, you know, I've, I've only been at Mayo now for about eight years. So as you, so let me see if I can unpack that. So you show up But I feel like I've already had the opportunity to wear a lot of different hats, and put my energies into various things. But it all kind of comes around, built around the idea of trying to figure out what to do with patients with cardiac arrhythmias. So when I first got to Mayo, Mayo was forming this relationship with OptumLabs. And we had access to this large administrative claims data through United Healthcare. And I realized that we had an opportunity to study what was actually happening in real life. And in my field, and EP, we were used to studies that maybe had 50 people or 100 people in them, and people were presenting single center experiences. And a lot of what we were doing was based on relatively flimsy evidence compared to other parts of cardiology, where we had large, randomized clinical trials. And I thought that was an opportunity to use real world data to inform what we do at the at the point of care. It wa n't something that I was an ex ert in, but I created or forged some collaborations with peop e in our Center for the Scien e of Healthcare Delivery, an was able to learn a lot about h w to apply these th to the place you see there's an opportunity, it's something you're not particularly trained or familiar with, you find some people that are and then you collaborate with them to make that happen. That's gonna be a trend that you're gonna, you're gonna see me talking about.
Victor Montori:So where did you acquire the inclination to do that. I mean, most people will like to stay wherever they're, wherever they personally find comfort, right? So this idea of going into a, an area that's new, and finding others that can help you...Did you learn that in kindergarten, everybody learns everything in kindergarten.
Peter Noseworthy:There was a, there's an opportunity at Mayo to do the Scholar Program that allowed me to get some formal training and connect to people who were truly experts in the use of big data and leveraging it for clinical questions. And I could see that there was an opening being created by this new relationship with United and OptumLabs. So I took that opportunity applied for that, was able to do some formal training for myself, but also just being able to understand the limitations and strengths of the data and then leverage the existing expertise within the institution. So I think when I first got to Mayo, I was a full time clinician, and I enjoyed it. But I had a relatively narrow scope. Every day I went to work, I was wearing lead, I was in a cinderblock room doing procedures. And I didn't feel connected to this large ecosystem. So I was able to find that there was an opportunity to sort of move outside of my relatively narrow scope, and broaden my horizons and have access to resources and expertise. And it paid dividends, because all of a sudden, I had this different perspective, I had days of the week where I was spending in different parts of the institution with entirely different people with non overlapping expertise. And I found that to be extremely exciting and rewarding. And it's, it's something that I think I'm trying to reproduce by going out of my comfort zone over and over again. And it's something that I tell other people who we've recruited, junior faculty to have some comfort in doing take that risk, even if you're a little bit outside of your own expertise. That's where the really interesting insights and when things become very exciting.
Victor Montori:I have to say that the, that was the most gangsta thing I've ever heard you say, I wear lead and I go, if feels like that should be a line on some sort of rap song.
Peter Noseworthy:Well, nobody's ever, nobody's ever given me that compliment before. Yeah. But I'll take it.
Victor Montori:Yeah. So and so variety. So I've now noticed that one thing that seems to be driving you is, you know, the interest in taking advantage of opportunities, but also curiosity, but also working with other people from different disciplines. That seems to be, that seems to add a lot to your day.
Peter Noseworthy:Yeah, without a doubt. You know, I think that burnout is a big problem with clinical medicine. And we're all trying to figure out how to, to create a career that is satisfying and sustaining, and where we feel like we make contributions every day, and we go home, exhausted in a good way, but recharged and wanting to come back. And I think that it's very easy for people, it's probably true in research, but it's definitely true in clinical practice that you can get in a rut, where day after day starts to feel the same and you feel like you're not making progress, and you're no longer excited by the questions and you feel like you're you're in a routine. And we have to fight that I think because we have a lot to give as researchers and clinicians, if we can show up every day energized and ready to give the most of ourselves and leverage the resources that we have. So for me that that requires having some variety. So I like it when I go to work on Monday, and I do procedures, and I'm working with my hands, and I'm doing something that's very tangible. The next day, I'm in clinic, and I'm sitting across from patients, and I'm realizing what is bringing them in and what their concerns are. And I'm developing relationships. And I'm figuring out what the key questions are that we need to answer. And then the next day, I come in, and I'm talking to researchers, and we're going through data, and we're figuring out what we can learn from the available data at our fingertips. And then the next day, maybe we're trying to design healthcare delivery options for our practice, from an administrative perspective, we're trying to motivate and build teams of expertise that can that can deliver care more effectively. I think if you can wear on each of those hats, I think you can see the same problem from multiple sides. And it's much more exciting and much more fulfilling.
Victor Montori:So that, that is an orientation towards, you know, helping, feeding yourself in keeping you sane, excited, open, generous. Is there another, what's the other side of that? In other words, is there value and also what you're able to do when you show up? In other words, the, do you feed on impact?
Peter Noseworthy:I think so I think you have to remember that it's always centered around that patient that you're trying to take care of. So even though on a day to day, or a week may go by and I may wear many hats, it all funnels down to a relatively small number of patients who I really feel like I have clinical expertise in and whose voice and experience I try to represent in the research and in the way we design our healthcare delivery models and so forth. So it always has to be grounded around the patient. And I think that, you know, if the clinician is in the clinical lane and only in that lane, and then there are administrators trying to make impact on the way that clinician exercises his practice or her practice it doesn't work because the solutions don't, don't meet the needs of that patient. Similarly, if there are researchers who are in a silo, and are not grounded in clinical practice, they may be answering questions that may not be relevant, or may not take into account the insights that a clinician at the, at the, at the frontlines can, can pose so I think it's important for us as clinicians and researchers to make sure that we're, we're kind of always stroking in the same direction, all of our efforts are, come to come together around a singular purpose.
Victor Montori:Before we leave the Big Data space, I always, I'm always struck by the fact that, you know, some people are able to see the value in the methods of research that they have, but they're also in a in a very privileged position to also see the potential dangers or challenges associated with using that data. Have you had any, any aha moments in relation to big data? Where, on the one hand, you've been, you know, excited about its potential, but on the other hand, quite worried about its potential downsides?
Peter Noseworthy:Yeah, there are many examples, without a doubt. So, you know, I got into this because I was interested in understanding the impact of an, of ablation procedures on long term outcomes, on mortality and quality of life and stroke risk and things like that. And the only way to really get at that question is to randomize patients, and then to look at how they, how they do. And whenever you're using an observational data set trying to overcome the potential for confounding is very challenging to do. And as clinicians, we're very good at picking up the patients who are going to do well with an ablation procedure. And even though if we match, or we do a propensity score on, we often use 90 variables, or even more to try to minimize that confounding. There's a certain intangible gestalt that a clinician can get across the table or from the foot of the bed looking at a patient. And it's very hard to understand how to completely eliminate that risk. So the other thing we have a big data is we can, we can have very small effect estimates that are highly statistically significant. So we don't want to be, we don't want to look at statistical significance as our barometer for whether something is valuable or not, we have to look at whether these are clinically meaningful impacts and whether we have confidence in the data. So we've developed a research program around taking clinical trial data and trying to replicate it in observational data. And then having this sort of reciprocal learning from what are the limitations and advantages of a clinical trial versus observational data. And those two things can go together very nicely. So there may often with the trial, it may not be particularly generalizable to real world practice, the complexities of everyday practice are not reflected in a very cherry picked, prospective clinical trial that's performed at very high volume centers. So with real real life data, we get a real sense of generalizability. But we have to, we have to understand that there's a potential for confounding and so forth.
Victor Montori:Do you, so you don't, you don't necessarily have to fall on the side of preferring chance, like in a randomized trial, to choice like in an observational study in terms of allocating of treatments, but you actually feel that these are complimentary sources of evidence to try to get closer and closer, if not to truth, at least to clarity.
Peter Noseworthy:Yeah, I think that's very well put. And that's exactly what we tried to do. You know, I would say, one of many investigators on our big randomized prospective clinical trial for catheter ablation called CABANA, and, you know, had a budget of, I think, around$60 million, it was enrolled over nine years, international trial from dozens of countries, but it only enrolled 2000 patients randomized to ablation, or, or medical therapy, but over the same time without trial, there were hundreds of thousands of patients who underwent ablation. So to to waste all of that observational data is a shame, but also to take that observational data and now put it in the context of what we can learn from random treatment allocation is also a shame. So I think we have to put those two things in context of each other and use them complimentary evidence. We have
Victor Montori:So, just because we have the large data it would be a shame to not do the trials. Right? So it has both, just for those who are not medically trained in our, in our, in o r conversation, do you have a layperson summary of wh t ablation is?
Peter Noseworthy:Oh ablation, so this is a invasive procedure that is done to control a cardiac arrhythmia of one type or another. So often these arrhythmias arise when there's some sort of abnormal electrical circuit in the heart. And just like any other electrical circuit, if you can break it, if you can identify the loop, find a vulnerable part within that circuit and break it, you can break the arrhythmia. So we use various energy techniques delivered through catheters, often through vascular access into the heart to cauterize essentially, either by heating or freezing the tissue and eliminating those abnormal circuits. So that's, you know, we're talking about big data. And then sometimes, sometimes I come to work and I'm dealing with an N of 1, you know, one circuit within one heart, and in a very sort of tangible way, and I enjoy that spectrum.
Victor Montori:That is phenomenal. So how do you, how do you go from big data to AI?
Peter Noseworthy:Well, so I think I realized that there's great power in data through my work in OptumLabs, and that was very fulfilling. But then we started asking over and over again, similar comparative effectiveness questions. But I realized that there's a lot of richness of clinical data beyond simply outcomes, claims, billing codes, things that can be extracted from the electronic health record. And it was actually a fortuitous turn of events that brought me to AI. My Chairman, Paul Friedman, asked me to run our ECG and physiologic monitoring lab, which is a part of our practice that deals with reading ECGs, an ECG is that electrical test where you put on the probes and you get a little electrical readout of heart rhythm. And to be frank, I wasn't terribly enthusiastic about taking over that role. It felt like a very mundane part of our practice, the ECG lab had been around for decades, it hadn't changed much. We were just reading the studies for the clinical practice. And I thought it would be a distraction from my research and clinical practice. But...
Victor Montori:Hold on, so your boss says, go run this thing. It feels like an administrative job on a mundane part of your practice. So you say no, right?
Peter Noseworthy:So I said, Give me a day to think about it. And he, and I think he planted the seed, which was, if you assume this role, you'll have access to a whole new set of data. And in the current environment, we can leverage that data to answer all kinds of interesting questions. So at Mayo Clinic, everything is centralized. And many healthcare institutions are like this, but particularly Mayo Clinic. So we read all of the ECG data for all of Mayo Clinic, which is, you know, somewhere between 250,000 and 400,000 ECGs a year, and we've archived all those as digital files. And of course, they're linked to our electronic health record. And they've been sitting there basically untapped for decades now. But in parallel with this, Paul Friedman had this idea of building out a core within cardiology of expertise in data science, artificial intelligence. And I saw this as a potential sandbox to play with these technologies or sort of an incubator for the application of AI to clinical medicine. And that was really how we got into this. So we started to do some studies on that incredible trove of data that was at our fingertips, and a whole world opened up. And to be frank, this is now what I'm spending most of my time when I'm not mucking around in somebody's heart, trying to find one of those circuits. This is what I'm doing. And it's...
Victor Montori:I hope you don't describe the procedures to patients as mucking about in their heart.
Peter Noseworthy:Right! Yeah, I think there's a more technical word, but I'm trying...
Victor Montori:So, so this, this AI opportunity now comes up not because you, you sought it, but because you were needed in a sort of leadership position. And you were able to see behind, at least your leader was able to convince you that there was something there for you, and you bought it.
Peter Noseworthy:Yeah, exactly. But it's just like, you know, there's sort of like a bit of an opening, a bit of an opportunity. It's a little bit similar to the way I felt when I saw that relationship with OptumLabs and access to data and how I could move out of my comfort zone in EP into big data. This was another, another pivot and a whole 'nother world....
Victor Montori:When people get started in science or in their careers, or even in medicine, they they often get the impression that the line towards their realization is going to be a straight line. You're describing a story of opportunities that you were able to recognize, but seem not to have been in your plans.
Peter Noseworthy:Yeah, exactly. I mean, if you went back three years ago, I really had no idea what AI was, and I had no particular ambition to start to learn it or to apply it to medicine. But I, you know, part of being a part of a rich ecosystem, like we have in Mayo Clinic and many other academic institutions are these, the breadth of talent and skill set. And if you can tap into that, it can be phenomenally exciting. So maybe I'll, can I take you through maybe our first project in AI for ECG...
Victor Montori:Before you do that, now you boasted that three years ago, you had no idea what AI was, well, now you can tell us what AI is.
Peter Noseworthy:Right? Well, what do you want me, how do you want me to describe this? I could describe how how we're applying it, the kinds of insights we're learning...
Victor Montori:What is it? Yeah, what is it? When people say oh, AI, what are they talking about?
Peter Noseworthy:Yeah, well, at its, at its core, It's data science, but it's just it's applied with, there are really, there are a couple factors that made AI, that have facilitated the explosion of AI and its application to medicine and other modalities, because the, the core concepts have been around since the 1970s. And the idea of machine learning and unsupervised learning, and taking a data set, pairing it with an outcome, training a dataset, training a computer system to pick up patterns in the data and predict an outcome. That's not a new idea. But now we have much better data at our, at our fingertips, we have it archived in ways that are usable, and we now have the computer capabilities to run increasingly complex models, which are now finally being able to deliver results that are useful at the point of care. So, you know, I'll give you an example, we, an ECG is a relatively straightforward electrical recording of the heart. And it's 100 year old technology Einthoven, you know, put it, you put your foot in a bucket of salt and made an inscription of, of the electrical activity in the heart won the Nobel Prize, but it really hadn't changed much for a long period of time. And if we use what's called a convolutional neural network, which is a way of taking that signal, breaking it down into convolutions or little features of the signal, they may be curves and slopes and inflection points, and these things that are not necessarily recognizable to the human eye as salient features. But if that's repeated hundreds of thousands of times into each of these features, and then a computer system learns the relationship between each of those features, and is trying to learn the relationship between each of those features and each other and then an outcome of interest, it can start to see patterns that are very informative. It's similar to the technology that's used in facial recognition, or any kind of learning of various images. And we're all convinced that that power when our iPhone sorts our photos into various family members, and regardless of the view, the lighting, it's it can be black and white, you can be wearing a Halloween costume, it can still tell Victor from Claudia from, you know Alonso very well. And I think that's a, that's a really powerful technology. So that same technology, applied to ECG, we can learn all kinds of features that even a very skilled cardiologist cannot discern based on the ECG. So we typically use ECG to say, what is the heart rhythm are they in normal rhythm or something else. But we had this idea that maybe we could use it to determine heart function. So we paired our ECG database with our echo database. Echo is a way ultrasound test of the heart to look at heart function. And we trained a convolutional neural network to look at the squiggly lines, and predict whether the heart was likely to be weak or not. And to our somewhat surprise it performed exceptionally well. So like any diagnostic test, we measure the area under the receiver operating curve. And if it's a guess, we get an AUC of 0.5, like a coin flip 50% of time, it'll say one versus the other. If it was perfect, it'd be 1.0, or it would always get it right, this test would get it right, this convolutional neural network on the ECG with an AUC of 0.93. So for comparison, tests that we use every day in routine practice, like a pap smear, or a mammogram or, might have an AUC of 0.7, to about 0.8, so we can beat things that we've already accepted into our practice just by taking an old technology, that ECG applying a new form of data science, and breathing sort of new light into the, into the technology.
Victor Montori:Do you know how that, do you know how this network figures that out?
Peter Noseworthy:That's a area of investigation and it's very interesting to try to do that. People often complain about AI, because it's a bit of a black box, we basically, it spits out a number between zero and one and we have to find a cut point, and then turn it into a binary result and take some action on it. But trying to see in that black box is a big challenge. So we can do things like, I'll go into some of these details just for fun, but you can do a saliency map, where you look at an ECG, and it'll highlight areas that the model is paying particular attention to, you could take an ECG, and you could blind the model to various parts of the ECG, and see when it can no longer recognize it. So it's sort of a deductive approach. You can blind it to various features that you might think are important and see how it performs. And then you can even use another model to instead of, to basically turn it upside down and one of our data scientists has worked on this as part of his PhD, but it's called this generative adversarial network where you have, have one network that instead of creating the output, it actually creates a fake ECG and then the other network looks at the fake ECG and tries to interpret it. And as soon as one network can fool the other, you've created an output or an ECG that the model can't tell from real from fake. And if you're training it to be of the, of the outcome of interest, it basically creates like a caricature of an ECG that represents the disease state of of interest. So people have done this for, you know, you might have a model that says, is this a horse or a zebra. And then if you make one of the generative adversarial networks, you can create a caricature of a zebra. And it's interesting, it doesn't really look like a zebra, what it looks like, is a mess of manes and stripes and a bit of the fence post at the zoo and, you know, these all these things, but you would look at it, any human would look at and say, well, that's sort of a Cubist zebra, but to the model that really looks like, looks very convincingly like a zebra. So we create these ECGs that look sort of funny, but they have exaggerated features, that might give us some clue as to what they're looking at.
Victor Montori:As you were coming into the field, was there a moment when you, when you, when you went oh, my God, this is good stuff.
Peter Noseworthy:Oh, yeah, for sure. Well, I'll tell you one little anecdote that I thought was especially interesting. When we developed this model that performed so well for detection of low ejection fraction with an AUC of 0.93. That was based only on the ECG. And we thought, what we could very easily just augment the model by giving it a couple of clinical data points that are readily available. And we started by just putting the age and the sex into the model. And we thought, if we went from 0.93 to 0.98, we're done, we're gonna have a great model. But in fact, it didn't improve the model performance at all to several decimal points. And we looked at each other and thought what's, what's going on here. But the data scientists knew right away the answer, and they said, the model already knows the age and the sex, looking at the ECG, that's the only explanation for why giving it additional information doesn't inform it. So we then thought, well, that would be amazing. Because really, we don't think we have any way to tell age and sex just based on the electrical inscription of the heart rhythm. But in fact, then we train models to pick up age and sex, and it performs exceptionally well. And that actually was the explanation. We're giving the model no new information by saying age and sex. So now we have a separate model that can tell the sex of an individual just based on their ECG with an AUC of 0.98 or 0.97, you know, that's just about as good as a human looking across a room at another person, it's really exceptionally easy for the model to tell sex on the ECG. And, you know, I read ECGs for a living and my colleagues do, I can't do that. So this is an example where the AI can see beyond human capability. And I, even though we don't need an ECG to tell the sex of a patient, it's proof that there's information there that's hidden in plain sight. We can't see it as clinician, but it's there. And it tells us that we might as well start looking for medically interesting patterns in this data, because there's likely all kinds of rich, clinically informative data hidden there, within the ECG, so that's catalyzed a whole number of projects within our group.
Victor Montori:Somebody for the first time looked in, in a microscope and saw a bacteria and realized that was causing disease or put on a couple of goggles to look at infrared or ultraviolet light and saw that there was more light than one could see. I mean, using your description that we're looking at a new, a new way of enhancing our perceptions or handling our senses.
Peter Noseworthy:Yeah, exactly. So you know, people are uncomfortable with this as a black box, because we don't always know what we're looking at. But there are so many examples of that. And this is a tool, we have to learn the limitations of it, the downfalls and apply it carefully and judiciously. But we shouldn't throw this out just because we don't necessarily always understand. And there's some really fun examples early on. We, we started, because we had now a measure of ECG age, we started to look through all of our patients, and we found some outliers. And one of my colleagues Suraj Kapa, I share an office on the clinical side with him, was looking at a case, patient looked older than his chronological age, he was aging appropriately. And then overnight, he became about 20 years younger. So he delved into the, into the, into the clinical record to say what was going on in this patient. And on that day, that he appeared to get younger based on his ECG, he had actually received a heart transplant and received a heart from a younger patient. And it was just some of those, you know, it's N of 1, it's anecdotal, but it kind of gives you goosebumps and it's, it's proof of this concept that there's stuff there. And since then, I can't tell you how many times I've gone through and used this data and seen these sorts of things. really fascinating clinical insights that we wouldn't have had if we didn't have the ability to to apply these technologies.
Victor Montori:I have to ask you the same question I asked you with big data. So now that you've sort of felt its power, what are, what are the caveats? What are the, what are the, the difficulties that you see in AI? What what makes you afraid?
Peter Noseworthy:Well, there are, when we learn from our past experience, and our practice patterns and the outcomes that are in the, in our recent history, and then we train models to predict that we have to be aware that there are inequities in the way we practice medicine, there are disparities in outcomes. There are biases that are sort of baked in institutionally and into these sorts of things. And we don't want to simply create systems that promote and potentiate that kind of, those sort of fundamental problems of our existing healthcare system, especially if we don't understand the models, and then they become baked into our practice and it sort of ensures potentiation and promotes these kinds of inequities. So we have to be very, very cautious. So, you know, for example, if, if we're not as good at diagnosing low ejection fraction or heart failure in one group of patients, but we create a model that's based on our historical data, then as we apply that model, maybe we will continue to miss those patients and that underserved or, you know, vulnerable population will continue to suffer those sorts of consequences. So we have to test these things rigorously in externally validated populations, we need to do subgroup analyses, we need to understand where these models work and where they don't. And we've been doing a lot, trying to do a lot of that work, before we deploy these models clinically.
Victor Montori:Which brings us to values, right, because you wouldn't be worried about those things if, if you were not, for instance, interested in injustice, and what would you think, what would you say is, has been a primary value driving, driving your work thus far?
Peter Noseworthy:Well, I think that, I like the fact that every day that I go to work I have, I wear a slightly different hat. And I like the fact that I can then test what we're doing in one realm, yeah, across discipline or across from a different perspective. So when, on a Thursday, if I'm working with these AI algorithms, and we're trying to figure out how to apply them, by the next Tuesday, when I'm looking a patient in the eye in the clinic, I can think about, does deploying this algorithm help this individual patient? And does it make sense? And is it bringing value to this interaction? Or is it a distraction? Or is it, does it open up some sort of risk for this individual patient? So they're sort of baked into this as a bit of a reality check? Each, each, each day. So I, I think having an opportunity to look a patient in the eye, have that conversation, and doing that regularly brings you back to the patient I think that's, that's sort of a critical thing. Mayo Clinic broadly, always talks about the needs of the patient. And they talk about research as addressing the unmet needs of the patient. And I think that's a, that's a perspective that most Mayo Clinic clinicians and researchers sort of live by. And it's a very simple way of articulating the fact that, fundamentally, we're here to take care of patients. And everything we do has to come back to that. And we have to have confidence that we're actually doing this for the good of our patients.
Victor Montori:Our research unit, as you know, has three values, you know, patient centeredness, integrity and generosity. From what you just said now, he sounds like the patient centeredness will be the thing of the three values that will be closer to you. Am I interpreting that correctly?
Peter Noseworthy:Well, I think as a clinician that really is the, the fundamental orientation I have to my daily work and my goals in life. Of course, integrity is absolutely essential. And that goes to things like you know, if we develop a model and it, and it ends up wreaking havoc on the way we apply medicine, that's a major problem. If there's problems with integrity of our application of big data to clinical practice, or we find some spurious association that we start acting on. That's an incredible disservice. And I actually like the idea of generosity as well. And I think that that's a fundamental value, I think, for building teams of people who work together for growing a research team, and so forth. So I think that, when I think about my mentors over the years, probably the most important attribute, in my opinion of a good mentor is generosity. And I can think of many examples. And you know, even the example of my chair asking me to take over this role in the ECG lab, is an act of generosity in a way, sharing this, this access to data, this opening up an entirely new line of investigation. And I think I have to be aware of that sort of generosity that people have offered to me and I try to offer it to junior faculty, collaborators, so forth, that I've worked with, and, and there are many aspects to that. So if you, if you share opportunities, if you can share responsibility, that's important. It's not just, and if you can share the credit, then I think you create an environment where things are synergistic. And there's more work to be done than one individual can do. But if you can share that responsibility, hold people accountable, in sort of a generous way where they can then benefit from it, you can leverage your work to be much more powerful and impactful.
Victor Montori:So we got a question from one of our audience members here was wondering about how you see competition. So some people see competition as a way of keeping your axe sharpened, and, you know, trying to beat the next guy to the next big idea, and so forth. And some see competition as inhibiting or impairing the kind of collaboration you've been speaking of. Where do you, where do you land?
Peter Noseworthy:I don't, I don't particularly feel a lot of competition. But part of the reason is the kinds of collaborations I think that I've formed. And I think it's easiest for us to think about collaborating with people who are very near to what we're doing. So people with the same expertise, the same worldview, the same experience, and the same skill set. And there you have two people, and it would be very easy for that to become competitive. But I think if you formed teams of people with completely non overlapping expertise with very different worldviews, then these are not people you're in any way competing with. But you have a true collaborative spirit. And you can feel very confident in what you bring to the table. And you can feel confident that, that, that adds value to the interaction. So my world is full of people who have much more expertise than me in almost everything that we're doing. But I know what I have. And that's the clinical perspective, the clinical experience, and sort of the ability to triangulate these different perspectives, and leverage our efforts to the good of patients. So that's a pretty narrow contribution to that collaboration. But it means that we're not in we're not in opposition, and we're not competitive. Now, if you look around the country, there may be other groups that are doing similar things, and I might have a inkling of competition with them. But that's just motivating. And, and that's sort of fun.
Victor Montori:I agree with you. I wouldn't work with other cardiologists. So on that note, what are your favorite collaborations?
Peter Noseworthy:Well, I think I've outlined a few of them. I love, I think I love to collaborate. There are people within the Center for the Science of Health Care Delivery that I've learned an enormous amount from with the big data analytics. And they also have expertise in how to, how to translate our practices, our learnings to clinical practice. So I've done a lot of work with Xiaoxi Yao. And I've learned a lot from her and her perspectives. Nilay Shah has, has a lot of experience in how to get these things into practice and sort of the regulatory aspects and, and that's been, I've learned a lot from that. And then the data scientists who I have almost nothing in common with in terms of our training, but we have everything in common in terms of our ambition to unlock these hidden patterns in the ECG. That's been a phenomenally exciting....
Victor Montori:What is a data scientist?
Peter Noseworthy:You'll have to invite a data scientist to tell you. I feel like I could be, if I misspeak, I don't want to be quoted amongst them. But they are people who, who understand how to how to leverage technologies to unlock the patterns and insights within data, and, and that can be through various techniques. But that's where the magic happens.
Victor Montori:Yeah, I just heard the other day a guy at Google Health was talking about how they're, they're swimming in, you know, just massive amounts of data that we all kindly, naively, you choose your word, give to them in the process of using their "free" tools. And, and he was saying that the biggest challenge that they have in using data, particularly related to health is, is the ability to ask questions, because the data is there, like you said, like it was the, with this, with the squiggly lines of the ECG were sitting there for decades, you say was untapped. And so here, they have massive amount of data that might have insights related to health, and a lot of data scientists that could actually unlock those insights. But their biggest limitation appears to be the ability to be able to ask the right questions, in ways that the answers will have meaning, in other words to convert the data into evidence, it seems to me that you play a significant role in asking those questions.
Peter Noseworthy:Yes, yeah, I think that's right. Data on its own is not particularly valuable. And it's the data scientists can turn data into, into information, you know, that's informative, and then you can turn information into evidence and could turn evidence into knowledge and knowledge into, you know, true insights at the point of care. And I think it's important that we, that we, that we harvest all that information that's at our fingertips, and we don't stop at turning data into information, but we bring it all the way into really informative insights that are valuable to our patients. That's really, a clinician has a role in that. And there are many stops along the way in the evolution of turning data into, into really important insights. And ultimately, it has to be tested in trials, we have to hold it to the same standard that we hold almost any new therapeutic or drug or diagnostic or device, we have to hold our data insights to that same sort of standard.
Victor Montori:There's another thread going through your work. In addition to take advantage of opportunity and be curious and be collaborative with folks from multiple disciplines and be ready to learn new skills. And that is something you were mentioned earlier when you were talking about the CABANA trial how it was a multimillion dollar investment in causal inference. And that you, how you run, what I would assume was a much less expensive effort in using data that had already been collected as a matter of taking care of patients and and billing them I guess. There's a recycling nature to your work, isn't it, an element that adds sustainability to research?
Peter Noseworthy:I like that formulation a lot. I actually haven't thought of it in that, in that term exactly. But you're right. I think that if we go back to that CABANA example, the CABANA trial cost tens of millions of dollars, and it took about a decade. And if you think about all the study coordinators, across all the sites, all the investigators, all the follow up all the research forms that got generated, all the IRBs, just the incredible amount of work to study those 2000 patients. And then with relatively small amount of investment, we had an R21 grant from the NHLBI for a couple hundred thousand dollars we were able to extend our learning from the CABANA trial, in my opinion, by using that available data. And it's much more efficient. Now that we're talking about AI and trying to test those sorts of applications prospectively, we're developing a whole set of prospective pragmatic practice embedded and EHR enabled clinical trials that I think will extend what we've learned from the AI and test it in a rigorous way. So I told you about how we develop the algorithm to detect low ejection fraction from the ECG, but stopping there is just a curiosity or it's information or it's a single research finding, but it's not necessarily clear how to apply that to clinical practice. So we did a prospective cluster randomized trial which we called EAGLE and cute name for like ECG AI guided detection of Low Ejection fraction in practice. So we created a IT infrastructure to take all ECGs that were being done for any reason, and silently in the background run this AI on all the ECGs that were done, and then we reported it to frontline clinicians across the upper Midwest health system, including in Wisconsin, and rural sites in Minnesota, to primary care doctors, gave them their result, along with a, a recommendation either to consider getting an echocardiogram or not. And then we just let it go in a very pragmatic hands off way. And it was up to the clinicians and the patients to decide whether they wanted to get the echo based on that new piece of data. And then we looked at were we able to diagnose more people with previously unrecognized low ejection fraction out in the community in practices across the upper Midwest. And we just finished that study, I just presented it at the American Heart Association, the paper is now under review, but we were actually able to demonstrate that we were able to increase the diagnosis of previously unrecognized low ejection fraction, just by taking this AI, running it through our electronic health record, making the results available along with a recommendation at the point of care, and essentially, execute a large prospective, randomized, clinical trial to do that. And we enrolled, you know, several hundred clinicians as the as the subjects in the study, and performed the AI on over 22,000 patients over a relatively short period of time, relatively inexpensive, certainly, if you think about in comparison to other prospective randomized trials. And it only took us about eight months between conception, execution, completion and submission of the paper. So we accelerate that timeline, we make it much more pragmatic, it's more generalizable, because it's taking advantage of real world practice and it's much less expensive.
Victor Montori:Now you need to figure out if it is the same to have...In other words, you have to figure out if the patients are better off because of this uncovering of their problem, right, because they may represent a population that is different from the population in which the problem is diagnosed, without the assistance of the AI or not.
Peter Noseworthy:Right. Yeah, and there are many layers to that. So there are subsequent studies as a part of EAGLE that we're actually currently doing. So we did look at whether or not they were treated appropriately with medications that seem to, and 97% of patients who are found to have low ejection fraction in our study, were started on either an ACE inhibitor or a beta blocker, which is the guideline recommended treatment. Now, we don't know these may be very low risk patients, we don't know their risk of progression, we have to look at long term outcomes, we have to look at things like there are certainly people who we flagged as potentially having low ejection fraction, but who had a normal EF during the study, and what do we do with those patients do they need in the, in the primary study, we found that those people were four or five times more likely to develop low ejection fraction over time. So we have to learn if that bears out to be true in practice, and do we need to then create some sort of surveillance infrastructure for looking for incident low ejection fraction. We need to understand the outcomes, there's still a lot of work to be done. But I think what we're able to show is a couple things. One, we can embed AI in clinical practice, by leveraging the electronic health record, we can use that to the benefit of early diagnosis of previously unrecognized, but medically important disease like heart failure and low ejection fraction, we can do it at a relatively low cost. And we can do it in a way that we think is minimally disruptive. And paired with this, we're doing a qualitative assessment of the patient experience with the algorithm as well as the clinician experience. And it's possible, the clinicians on the, on this, listening to this podcast will be well aware that there's nothing more annoying than the constant alerts that come across the dashboard as we're seeing patients and caring for them. And these sort of computers telling us what to do. And one more alert could be overwhelming, it can contribute to burnout. And if it's not actually beneficial to you, and your patients, eventually it will be it'll be rejected.
Victor Montori:Yeah, I am also concerned about the patients who, all of a sudden you're minding your own business doing their thing, and then they get this alert that say they have heart failure. It seems a pretty loaded thing to be told. And then if you were to do a quick search about the term, and you don't know, you don't know necessarily about severity, you may find that oh, it says that 50% of people like me are gonna be dead in two years. Well, thank you very much algorithm.
Peter Noseworthy:Right. Yeah, no, exactly. But that's part of the reason that the consented subjects in our trial are actually the clinicians, not the patients. So the patients didn't get these results. The, their clinicians got the results, and it was up to the clinician to put that into clinical context and figure out whether it was worth looking for low ejection fraction in that patient. So let me give you two examples. There may be a patient who is dealing with end stage metastatic cancer who's getting potentially cardiotoxic doses of chemotherapy, and who is getting supportive care, or something like that, in that patient, it may be very plausible that the algorithm picked up low ejection fraction, but it may not impact the trajectory of that patient's health care. And the clinician can look at that, put it in context and say, I'm gonna leave that to the side, and we're going to deal with the issues at hand. Whereas there may be a stoic Minnesota farmer, who has been noticing that he's becoming increasingly short of breath, working hard, but has not brought it up. Is not complaining to his doctor, but in only a very subtle way and in this situation, the alert that this patient may have a low ejection fraction may just cause that clinician to ask a couple more questions, uncover this sort of previously unrecognized, decline in functional capacity or something for that individual patient and warrant an echocardiogram. So even though the AI runs as a machine in the background, it's man and machine coming together for the benefit of the patient, where it really brings value. So that's why we give the result to a clinician who can put it into context in an intelligent and patient centered way.
Victor Montori:Yeah, one looks at the early days of the Mayo Clinic and as the Mayo brothers were recruiting colleagues, you know, all of a sudden, you know, colleagues have been, we're coming in trained, for instance, again, in the use of the microscope, and all of a sudden, we need to start analyzing urine and see if that's going to give an insight as to what our patients have. And why would urine tell us that and all of a sudden, urine becomes, analysis of urine becomes a black box until we start understanding exactly how it helps us, but the use of it to help was already quite in place. And in Peru, there is a there is a story, in Peru where I was born there's a story of, of Dr. Barton, who had come from, I think it was France with the ability to use a microscope and put it to use to understand that Oroya fever was caused by bacteria. And he you know, now we call it bartonella because of Alberto Barton but nobody would believe him. And so he left medicine, he left medicine and used his knowledge of microbiology and pasteurization to develop the first soft drink in Peru that was fully pasteurized, and it was called Pasteurina, which lasted long enough for me to actually have tried it. Yeah. So this, this series is about care that fits, it's about making health care and the care of patients fit in the, in the complicated lives that people have, do you see a role for the kinds of technologies that you're working with now to facilitate, to make care fit in the lives of patients?
Peter Noseworthy:Yes, um, there are, you know, I could go through some specific examples. One is just is, is to take this, these sorts of insights, actually apply them to clinical practice, and try to do that in a way that is to the benefit of patients. Another is to leverage these technologies in novel ways in how we study various interventions. The follow up study to EAGLE is called BEAGLE and it's a, it's using a different algorithm to look for atrial fibrillation, and like low ejection fraction, this is a problem that can lurk, it can be asymptomatic. But we have a similar algorithm to run on the ECG that we think might indicate an individual's likelihood of having atrial fibrillation. And the main problem there is that if you have atrial fibrillation, you're at risk of stroke. And if you don't know you have atrial fibrillation, you don't know you're at risk of stroke. So some portion of people who suffer a stroke are found after the stroke to have a. fib. If you have a. fib, you could anticoagulate you could prevent that stroke, but making that diagnosis is key. And since it's fleeting, and often asymptomatic, there's a lot of a. fib out there in the population that is missed. And I, as somebody who takes care of these patients, I've always thought how do we go out there and find these asymptomatic episodes. So we have this algorithm now that can run on an ECG and identify patients at risk. And then there are separate algorithms that can scour a medical record and say, did we already know that this patient had atrial fibrillation, and then ask other questions like if we were to find that they have atrial fibrillation do they have enough other risk factors that they'd be at sufficient risk of stroke that we'd actually want to treat them with an anticoagulant? So we can use all this AI, natural language processing, and leveraging the data that's available to us in the background, to look at the entire ecosystem of Mayo Clinic patients, hundreds of thousands of people at once, if not millions, to find the people who we think are out there with an unrecognized lurking risk of stroke related atrial fibrillation. So we just started this study, a couple weeks ago, we now enrolled about 150 patients. We identify them and without even bringing them into the hospital, or bringing them into a clinic, without having to have a face to face conversation, we reach out to them through electronic means through the patient portal. And we say, we think you might be eligible for a clinical trial, would you be interested in participating, they can enroll, they can learn about the study remotely from the comfort of their home, regardless of where they live. And then we're just sending them in the mail, a heart monitor that they can wear and look for this arrhythmia. And over the first week that we started enrolling, we had only enrolled about a dozen patients, we picked up two patients with previously unrecognized atrial fibrillation without having them to ever even come into Mayo Clinic to talk to anybody in person. So in that sense, we're taking these technologies to patients where they are in a way that they can receive this care within their home environment at at their leisure and their convenience. They're not coming in, they're not parking the car, they're not spending money, they're not taking a day out of work, we're really trying to reach them where they are, and study them with the trial, but also offer them technologies that we think might be to the betterment of their health. So that's an example where we can leverage these technologies to meet patients where they are.
Victor Montori:And you just gave an example of a chronic condition. And I've always believed that in order for, for that to work out, you have to develop relationships with patients, personal relationships with patients. Convenience in this fashion seems to put a distance, potentially, between patients and clinicians. Is that a concern?
Peter Noseworthy:Yeah, that's a that's very astute. And you know, you've made this point before, but I think what we're doing in this case, these are Mayo Clinic patients, even though we're not talking to them, we have a relationship with them, or they have a relationship with us as a health ecosystem. And they've put their trust in us by creating a portal account, by sharing their data with us by having that open line of communication. So in that case, we're using technology to reach them. And we're using our existing trust and goodwill that they have for us and us for them to leverage that kind of technology. So it reminds us that we have to protect that trust and that integrity to get back to one of your key guiding principles. Because if we lose that, we lose the ability to reach patients in novel and technology enabled ways.
Victor Montori:Well, during our conversation, your pager has gone on several times and I wish you had some sort of algorithm to know which of those actually required you to pick it up.
Peter Noseworthy:I know I'm getting increasingly nervous.
Victor Montori:Yes, interrupt the thing. But I, you know, we'll, we'll come to, we're coming to an end, Peter, and it's been absolutely I think not only very informative, but a joy to, to review your extremely brief and early account of your career. And obviously, the adventure continues and we would love to, to hear more about that. So that's my last question. What's next for Peter Noseworthy?
Peter Noseworthy:Well, we've set up a bunch of these trials, we're testing these sorts of algorithms in novel ways, and each one is slightly different. And then we're also trying to figure out how we can reach beyond the Mayo Clinic patient. So we think we have these really valuable technologies. But right now we're offering them only to people who are within our ecosystem. But you know, Mayo has an ambition to reach tens of millions of people, if not hundreds of millions of people around the world. And these kinds of digital technologies, I think can bring value to people at relatively low cost, regardless of where you are. So we're trying to develop new ways to extend Mayo Clinic's reach beyond its walls and its existing capacity. And I think that's going to be an incredibly challenging thing to do, but also very exciting. And we want to try to bring the best of what we know in the best of Mayo Clinic to those regardless of where they might live.
Victor Montori:Thank you. Thank you, Peter. This has been the KERcast, brought to you by Knowledge and Evaluation Research unit. I hope to see you next time and please take care and Peter Noseworthy thank you very much.
Peter Noseworthy:Thank you, Victor.