Carol Vassar, host/producer:
Welcome to Well Beyond Medicine, the Nemours Children's Health Podcast. Each week, we'll explore anything and everything related to the 80% of child health impacts that occur outside the doctor's office. I'm your host, Carol Vassar, and now that you are here, let's go.
Music:
Let's go well beyond medicine.
Dr. Mark Friedberg, Hospital for Sick Kids, Toronto:
AI definitely cannot replace the human who is standing at the bedside, but definitely, they can help us be better at what we do.
Dr. Pei-Ni Jone:
I don't think that AI will replace the human being. I think it can help us become more efficient in our work.
Dr. Babar Hasan, Aga Khan University, Pakistan:
I still always want to talk to my physician, and you put trust in a person, and there's the art and science of medicine, and I think it's a lot to do with the art. I think it's a technological tool. It's a very powerful one. We're going to have to learn how to use it, but it will never replace the physician.
Carol Vassar, host/producer:
Doctors Mark Friedberg, Penny Joan, and Babar Hassan expressing the consensus of the pediatric cardiac AI experts we spoke with recently at the World Congress of Pediatric Cardiology and Cardiac Surgery. That is: AI will not replace the work of a skilled, trained, experienced human clinician. Yet AI, in all of its formulations and manifestations, has the potential to bring revolutionary change and improved outcomes on a global scale.
So, what's next for artificial intelligence in pediatric cardiology? It lies in part with doctors and training and newly minted MDs like Scott Anjewierden, a third-year resident at Mayo Clinic in Rochester, Minnesota. He's a fellow in the competitive and prestigious NIH Star Fellowship program that trains residents in research. His academic project focuses on artificial intelligence-enabled electrocardiograms.
Dr. Scott Anjewierden, Mayo Clinic:
So traditionally, when a patient needs an ECG, they will go to the clinic, they'll get the leads, the stickers placed on their chest, and then that will go to a physician, an electrophysiologist typically who will read the ECG, look at it, examine it for patterns, and see if there's anything concerning in that ECG. What we've found is that a computer enabled with artificial intelligence can find patterns in these ECGs that even humans aren't detecting. So, it's broadening the ability of these ECGs to detect and screen for different diseases. And so that's what we're looking at. We're a little bit behind in pediatrics. But the adults have started to find that quite a few additional opportunities for using these AI-enabled ECGs, and my work is to try and bring that into pediatrics and pediatric-specific diseases.
We don't have any of these in the clinical setting, but I envision a day in the near future, hopefully, where you would get an ECG, and on the spot, it would come back with an initial reading from this AI algorithm that would flag for several concerning possibilities, maybe low ejection fraction or poor heart function that wouldn't need to wait for that cardiologist interpretation and would be finding additional things that a cardiologist typically wouldn't find.
So, we started simple. We've developed an algorithm. You can feed in an ECG, and it will tell you what age the patient is, and it's pretty accurate. Typically, it's within about a year and a half. I can talk more about that, but big picture, our idea is to create a whole host of algorithms that detect different things, and we've got a few in the wings ready to get published. We're going to present a few at the American Heart Association meeting in November, but those are finding things like low ejection fraction or poor heart function. There's one for detecting sex, patient-reported sex, which we've found that can monitor pubertal status and correlates with hormones. And then, we have a few looking at atrial septal defects, ventricular septal defects, and other congenital heart diseases specifically. So, we're finding now that we have a robust data set, and we've got the code in place to run these algorithms where we have so many questions, and the answer is, which one do we do first? Because there's so many opportunities for us to build these different algorithms.
Carol Vassar, host/producer:
As an early career physician and researcher seeking a fellowship in pediatric cardiology, I asked Dr. Anjewierden to share his thoughts on the potential of AI in both the near future, three to five years, and across the totality of his career over many decades.
Dr. Scott Anjewierden, Mayo Clinic:
I really see the adoption of AI to go with physician interpretation. So, in my work specifically, we work on screening, so there's a limited number of pediatric cardiologists. If we can enable AI to extend our reach to do a first pass at interpreting different studies, we can reach more people, we can make it more cost-effective, and we can bring better care to patients that way. There's tons of other opportunities for us to implement AI. I think another one that's been presented at this conference was the use of AI and real-time data. As physiologic data from the ICU comes in at just huge volumes, can AI process that and warn us about things to come? I think AI can help us dictate our notes for clinic. It can draft a letter to respond to an insurance company about approving certain tests. I think if we work together with the AI, we'll become more cost-effective, more efficient, and we'll be able to reach more patients.
Carol Vassar, host/producer:
Just as the future of artificial intelligence lies with physicians across the spectrum of experience, it requires collaboration between specialties, fields, functions, institutions, and global boundaries.
Dr. Shubika Srivastava, Nemours Children's Health:
I think it's a long journey.
Carol Vassar, host/producer:
Dr. Shubika Srivastava, Co-director of the Nemours Children's Health Cardiac Center and Chief of Cardiology for Nemours Children's Health, Delaware.
Dr. Shubika Srivastava, Nemours Children's Health:
We need data. We need rules and guidelines to help us move forward as a community and really, most importantly, collaborate. We cannot individually single-handedly take on this task without collaborating across the fields.
Dr. Rima Arnaout, UCSF:
Obviously, you need computer scientists, data scientists, and clinicians.
Carol Vassar, host/producer:
Dr. Rima Arnaout, Assistant Professor of Medicine and a member of the Baker Computational Health Sciences Institute at the University of California San Francisco.
Dr. Rima Arnaout, UCSF:
I find that if you just put a classically trained computer scientist in a room with a classically trained clinician, there's something that's lost. You need new people who are glue people. The clinician who knows enough about computation that they can think of problems in that way. The computer scientist who's interested enough in medicine that they've taken some initiative to learn a little bit about how doctors think. So those kinds of people form the core of the best possible team.
In addition to that, and this comes to our collaborators, we have clinical collaborators who do this every day. Once you're out of the clinic, you lose your finger on the pulse of what exactly are the problems that patients and doctors need solving. So that's really important to us. We're increasingly trying to crowdsource some of our clinicians over reads or validation of what the model does. It's really interesting because it also serves as somewhat of an educational experience for the clinicians as well, so we have the infrastructure for that.
The last piece is when we designed this model, we really had the world in mind.
Ultrasound is a funny thing, right? There are cutting-edge latest machines that can do all kinds of fancy tricks and pictures, but not everyone has those machines. Then you got to think that most clinics have an ultrasound machine that might be five years old, ten years old and is really working on standard two-dimensional imaging.
We're trying to design our solution for that standard case. We don't want to design a solution that only works if you have the latest, fanciest machine because then you say that you're building AI. You say you're democratizing a solution, but you're not. You're just solving something for the elites already if you will. So, with the Netherlands community study and other studies, we're trying to test our model out in the communities. Those small clinics may or may not have their data organized in a way where it's easy for us to get at it. We work with those folks. We help them figure out where their data was stored; figure out how to send it. All of those things are not part of the daily operations really of hospitals in general, but certainly not in small clinics. So that's a piece where more support, more understanding that it's time for multiple reasons, not just for AI research. It's time for hospitals and clinics to take hold of understanding where their data is stored, how it's transferred, and so forth. I think that can help a lot of research efforts.
Carol Vassar, host/producer:
At Cincinnati Children's, Dr. Michael Gaies is medical director of the acute care cardiology unit. When he was at the University of Michigan in 2016, he and his colleagues were working to determine whether the implementation of a cardiac arrest prevention practice bundle could decrease pediatric cardiac arrest rates. That's when a natural opportunity arose to help them with their efforts, which in turn provided a concrete use case for bringing AI in the form of predictive analytics to the bedside.
Dr. Michael Gaies, Cincinnati Children's Hospital:
I was a cardiac intensivist at the University of Michigan, and we implemented a software program that is commercially available and used by many congenital heart centers, which collates and displays data in a way that we never had access to before, goes beyond what the electronic health record can display to us, much more real-time integrated and customizable. After implementation, we recognized an opportunity to determine how this software platform impacted our patient's outcomes and the care that we provided.
So, this study really compared our patient outcomes before and after implementation of the software and compared those data as well to a group of control hospitals from our Quality Improvement Collaborative, PCCC, the Pediatric Cardiac Critical Care Consortium, and we found that this software platform led to improved outcomes in cardiac arrest prevention, length of stay and readmission to the ICU. Now, what's interesting about that platform is that it was very advanced for us in terms of data aggregation and display, but the predictive algorithms that were built into this platform when we first took it on actually lagged far behind.
And it underscored, I think, what's really necessary as we think about the application of artificial intelligence and predictive algorithms to impact clinical decision-making in that it has to contain data that the clinicians can't see for themselves. The initial versions of the predictive algorithm called the Index of Oxygen Delivery or IDO2 didn't include key monitoring components that we have available to us all the time in the ICU, and therefore, we felt that we had access to more information than the algorithm did. And so, as we progressed through the many stages of adoption of artificial intelligence and predictive analytics, its ability to show us the things that we can't see for ourselves is truly where we will see an elevation in our clinical practice.
If you have algorithms that reveal patterns that an experienced clinician can integrate with their experience and judgment, that is where you have the true human-computer interaction, if you will. And even that part of it, teaching experienced clinicians to trust the data that come from an algorithm and teaching them how to apply their experience on top of that is even a whole other task beyond just making the math work well and making the data capture work well and getting access to data that are so subtle that you can't see it with your own eyes. It's not trivial to look at how humans interact with new technologies and how to gain adoption and not just psychological or emotional buy-in, but how to actually use it so that it's most advantageous. And there's a whole literature on this, which I've only scratched the surface in my time researching it, but clearly, we need experts from outside of medicine to come in and tell us how we, as clinicians, can best utilize the tools at our disposal.
Carol Vassar, host/producer:
For Dr. Mark Friedberg with the Hospital for Sick Kids in Toronto, the AI revolution means coupling a more powerful and accurate cardiac ECG with inputs from other patient-specific parameters to create the clearest picture yet of a patient's overall health so that he, as a physician, can decide the best next steps for that child's treatment.
Dr. Mark Friedberg, Hospital for Sick Kids, Toronto:
There's going to be automatic quantification, automatic detection of borders, automatic reporting, and already, there are commercial companies that will take the echo from the beginning to the end and generate a report all through AI. Most of the parameters that we take a long time to measure today. And so all of that's going to happen. So it's going to improve image quality, it's going to improve the reproducibility through that, both through the image quality enhancement and through the automated measurement, and it's going to allow us to take many more parameters into account or many less parameters into account because so many algorithms can recognize from many, many parameters, just those two or three or four that are really important in determining the outcome. Much like facial recognition can recognize those features of a face that ultimately identify me versus you. There's certain features. So it's the same in AI, so that's what's going to become more powerful.
But, ultimately, what I think it's going to be is the ability, as I said, to take the imaging data together with many other parameters, even in the acute settings such as in the intensive care unit. All that data we get from monitoring a patient's respiration and gases and everything else right through to the cath lab or just to the ordinary unit or to an ambulatory setting, and something simple like imputing the blood pressure, but imputing maybe the lab test together with that or the physical exam, or automatically recognizing the words from the medical chart and incorporating that together with various imaging and the ECG and the CT, all of that together. That's where I think it's going to go, and that's already what's happening.
Carol Vassar, host/producer:
One real-world scenario of AI's problem-solving potential that has great implications for the future of pediatrics and pediatric cardiology involves using ensemble learning to better detect fetal congenital heart disease via ultrasound, and it comes from the lab of Dr. Rima Arnaout at UCSF.
Dr. Rima Arnaout, UCSF:
Ensemble Learning Models means that we used a couple different learning models, and we stacked them together like Legos.
So, the real problem that we were trying to solve is how can we get better at detecting fetal congenital heart disease from ultrasound? So worldwide, ultrasound is recommended for every pregnancy at about 18 to 24 weeks to look at a number of things, to look at organ anomalies. That's when you find out if you've got a boy or a girl, but importantly, you're also looking for congenital heart diseases. Congenital heart disease is a paradox because it's both the most common birth defect and also it's still relatively rare. Complex CHDs are still less than 1% of the population. So, it's really hard for physicians to build and maintain their skills at CHD detection, but the stakes are very high.
This might be only 1% of the people you see, but those people need delivery at special hospitals. They may need surgery in the first year of life. Increasingly, there's research going on where they will try to intervene with catheters, with surgeries, with medicines during pregnancy to try to improve the outcome of the CHD. You can't do that unless you're detecting those CHDs during fetal life. So, it's extremely important. Affects every pregnancy, and yet, if you're not in an expert center, worldwide detection rates, so sensitivity specificity can be as low as 30 to 50%. So, there's a huge gap between where we need to be and what detection can be in practice worldwide. That's the problem we were trying to solve.
There had been been important clinical research which showed that when you really train people, clinicians on acquiring good images, interpreting images, when you really put them through the paces, they get much better at detecting CHD. The problem is then you go back to your day job, you're still not seeing these more than 1% of the time, you lose that training.
So, we were looking to see whether computer algorithms - machine learning - whether we could train the computer to interpret fetal ultrasound imaging and detect CHD because computers, they don't get tired. They don't forget quite in the same way. And to see whether we could use that to improve CHD detection with the goal of making that a tool that clinicians can use around the world.
So, we trained our model on imaging from UCSF. Interestingly, we trained it not just on the fetal survey, which is that everybody gets a 20-week ultrasound I was telling you about. We also trained it on a more detailed type of art ultrasound done by experts focused on the heart fetal echocardiograms as well. So, we trained on the specialty imaging in order to improve performance on the screening imaging, and that seemed to work pretty well.
So, we tested in a number of different test sets. It's like the final exam for the model, and one of them, we did over 4,000 fetal surveys - fetal ultrasounds - with a real-world prevalence of CHD, and the model got an AUC, which is like accuracy of 0.99, which is pretty high. That said, we're still testing on UCSF data. One could say that data was still acquired in an expert center. Maybe it was easier than what you might find around the world. So, we've continued to work with collaborators. Now, we have a nice team of centers across the country and across the world where we're working with them to continue to validate our algorithm from different clinics around the world.
So, we have started testing in more and more external data sets so different from the ones that the model trained on. We have a pre-printout in MedArchive where we worked with a group in the Netherlands who've been amazing collaborators. So, the Netherlands actually does something great where they have a national protocol for these fetal ultrasounds that's always the same. And when there's a CHD detected, they put it in their registry. So, they know all the CHD cases in their region, and they know which ones were diagnosed at the time of the ultrasound and which ones were missed during the initial clinical assessment but were picked up later when the baby was born.
So, we've tested on that dataset. Now, it's interesting. It was a real test of generalizability of our model. Generalizability just how well does the model works on data sets different from the one it was trained on. Because the data sets, the fetal ultrasounds from UCSF they record a lot of images, 3,000 to 5,000 images per study. So, there's a lot for the model to look at when it's doing its thing. In the Netherlands, in the clinics, in the rural centers, and so forth, the average number of images per study that they store is like 41 images. Very, very different. We trained our model to work based on the international screening protocol. The Netherlands uses their own national protocol. That's a difference. So, despite all of those differences, actually, we tested in this Netherlands community dataset, we have a sensitivity of 91%.
So, baby steps, no pun intended, but we're trying to validate this piece by piece.
Dr. Scott Anjewierden, Mayo Clinic:
When we talk about artificial intelligence, we're pushing the boundaries of medicine.
Carol Vassar, host/producer:
Dr. Scott Anjewierden.
Dr. Scott Anjewierden, Mayo Clinic:
We're well beyond the traditional. We're going into the future and trying to see what we can develop. But also has this meaning that our patients need to be well and experience wellness beyond the walls of the clinic room. How can we bring that type of wellness outside of our medicines and our patient exam rooms, and how can we bring that to their homes? I also think AI has some potential to bring that closer to them.
Music:
Well beyond Medicine.
Carol Vassar, host/producer:
Thanks for listening to part three of our series on artificial intelligence and pediatric healthcare. With me, Carol Vassar, and our guests, Dr. Scott Anjewierden, Dr. Michael Gaies, Dr. Rima Arnaout, Dr. Mark Friedberg, Dr. Pei-Ni Jone, Dr. Shubika Srivastava, and Dr. Babar Hasan.
This concludes our series on artificial intelligence, but not the conversation. Join us at nemourswellbeyond.org to tell us your thoughts and ideas on AI and our AI series. That's nemourswellbeyond.org, where you can also leave your ideas for future podcast episodes or series, hear earlier episodes of the AI series and all previous episodes of the podcast. You can also subscribe to the podcast there as well.
Thanks to the team working behind the scenes to make this series possible. Che Parker, Cheryl Munn, Susan Masucci, and Allison Micich, and thank you for listening. Join us next time as we talk with former Health and Human Services Secretary Alex Azar and Nemours president and CEO Dr. Larry Moss about value based pediatric care, and the first ever pediatric track presented by Nemours at HLTH in Las Vegas. Until then, remember, we can change children's health for good - well beyond medicine.
Music:
Let's go…well beyond medicine.