AI Clinical Trial Screening: Evidence, Recruitment and Workflow with Dr. AJ Blood

Clinical trial recruitment starts with finding eligible patients, but the work does not end with a match. Cardiologist and AIwithCare co-founder Dr. AJ Blood joins Chris Hutchins to discuss AI-assisted clinical trial screening, the supporting research and the workflow around it.

They examine AI chart review, RECTIFIER, studies in NEJM AI and JAMA, access to trials, bias monitoring and implementation across sites. AJ explains why research staff and frontline clinicians need to be involved in design, and why a pilot needs clear measures before it starts.

The JAMA study discussed was a single-center heart-failure trial with time to eligibility determination as its primary outcome; enrollment was exploratory. The episode considers what that evidence supports, what still needs validation and how to evaluate useful change without relying on promises.

IN THIS CONVERSATION
- AI chart review and eligibility screening for clinical trials
- What the NEJM AI and JAMA studies measured
- Patient access and the difference between screening and enrollment
- Bias monitoring and validation across research sites
- Workflow adoption, frontline design and measurable AI pilots

CHAPTERS
00:00 Welcome and episode introduction
00:19 Evidence over promises
00:41 From clinical research to implementation
03:42 Why effective care struggles to reach patients
09:04 AI-assisted chart review
12:28 Who gets access to clinical trials?
16:55 Building and testing the screening tool
23:10 What the evidence shows
27:35 Faster enrollment and shorter timelines
30:51 Monitoring bias across sites
34:57 Workflow determines adoption
42:41 Set clear measures for a pilot
47:25 Design beside the people doing the work
51:46 Connect with AJ

GUEST & RESOURCES
AIwithCare: https://aiwithcare.com/
AJ Blood on LinkedIn: https://www.linkedin.com/in/alexander-blood-md-msc-facc-abom-a8654a23/
2025 JAMA randomized prescreening study: https://jamanetwork.com/journals/jama/fullarticle/2830514
2024 NEJM AI study: https://ai.nejm.org/doi/full/10.1056/AIoa2400181

THE SIGNAL ROOM
Hosted by Chris Hutchins
https://signalroompodcast.com/

Transcript

Copy Transcript Copied

Chris Hutchins: 0:05

Quick thing before we start. I wrote a book. It's called Beneath the Signal. And it's about the human work behind trusted data and responsible AI in healthcare. Search Beneath the Signal on Amazon now and pick up your copy. Okay, here's our episode. Most of the time, AI in healthcare, it's being sold on a demo and a promise. But I guess it's something almost nobody does. He put his own AI tool through the same evidence bar that we hold medicine to He's a practicing cardiologist and intensivist at Brigham and Women's Hospital up in Boston. AJ, welcome to The Signal Room.

Dr. AJ Blood: 0:38

Thanks so much for having me, Chris.

Chris Hutchins: 0:39

I've been looking forward to the conversation. Love to hear a little bit about your background, so you know what kind of led you to practicing medicine, you know, kind of the progression that got you really Clearly excited about using AI and working in the research space.

Dr. AJ Blood: 0:53

Yeah, thanks so much, Chris, for the opportunity. Yep, so I'm really fortunate to run a research group here at Brigham and Women's Hospital today that came from many years of training and learning. Know, I started as an undergraduate at Johns Hopkins and worked at the medical school at Johns Hopkins Hospital. Really very basic science at the time. Really learning about transcription factor regulation as I was doing my undergraduate studies in neuroscience. I was then recruited to the National Institutes of Health down in Washington, DC. I was able to spend a year there, again, kind of furthering those very basic science academic pursuits, publishing in genes and gene transcription. What we then, I then had the opportunity was to join actually a brand new medical school at the Hofstra Northwell School of Medicine in New York. So decided to kind of step out of the laboratory and to the bedside, which was a really great experience for me and really helped me fall in love with patient care. Did my undergraduate medical studies there at Hofstra Northwell. So really fortunate to do my internal medicine training at Duke University and Duke University Hospital, work with the Duke Clinical Research Institute. And was really, again, flabbergasted about this decades long gap between medical discovery and something that would affect a patient at the bedside. And so

Chris Hutchins: 2:11

Right.

Dr. AJ Blood: 2:12

I really started to learn that process of clinical trials and how essential it is to show that as we come up with these new innovations, that they're safe and effective. And the way to do that is in prospective blinded randomized controlled trials. And we can oftentimes be surprised by the results of those trials. Of recent data that's come out even on the cardiology space where I work as a cardiologist has really kind of challenged the notions that we came into these, phase one, phase two, into these phase three trials to show that these drugs really should be made available to patients with good safety and efficacy profiles. But that's a major time and effort sink that goes into these. And so as I joined at Brigham and Women's

Chris Hutchins: 2:50

Right.

Dr. AJ Blood: 2:52

I did my fellowship focusing on a lot of these tools and artificial intelligence, machine learning, as well as epidemiology and my master's at the School of Public Health here at Harvard, really just like fascinated by this process and how challenging it is to go from an idea to a safe, effective medicine being delivered to a patient at the bedside.

Chris Hutchins: 3:13

Right. No, it's not something that everyone tackles. So I mean I most of the people I've talked to over the years that do research, I mean, they do a fair amount of publishing, and of course the discovery part is really exciting. But you know, being a practicing physician, what made it such an attractive thing for you to start to work on fixing this the slow and you know inconsistent screening for clinical trials? I mean, that's not a small task that you're tackling.

Dr. AJ Blood: 3:42

Yeah, it's not always the most glamorous or sexy aspect. So my research group at Brigham and Women's works in what we call implementation science. And so I was really fortunate to train under one of my mentors, Ben Scirica, the TIMI study group here at Mass General Brigham. Which really does these big trials in partnership with many of these large pharmaceutical and medical device companies to show that these medicines are safe and effective. One of the things we were surprised by when we were looking at our own kind of internal quality metrics at Mass General Brigham, you know, there's, know about this evidence gap. And even when the major societies publish guidelines on what medicines should be used, what targets should be, you know, really... Aimed for in terms of achieving blood pressure or cholesterol control, or a new medicine comes out and here's the patient population that's going to benefit from that. We were really surprised that even in those situations where we were designing and running many of these trials here as the leaders at Mass General Brigham, our patients still had gaps in terms of their optimal care delivery and care quality. And so that was really surprising to me. I went... I sought to learn, well, why is that? What's the challenge? We know this medicine is safe. We know it's effective. We know these patients should be on them. And even here where we're running the studies and writing the guidelines, we still have these opportunities for improvement. And so this field of implementation science, how do you kind of overcome many of the challenges from the patient, the provider, and the system side that make it more difficult to actually get that evidence-based medicine delivered to patients? I think what we really found was there's not one single problem. Every snowflake in an avalanche pleads innocent, it's really kind of this... System-wide inertia, both from the provider, the patient, and the system standpoint, that makes it really challenging to get that right medicine to the right patient at the right time. And so what we really focused on is a remote team-based asynchronous approach. And I had the fortune of doing this as a research leader to understand how can we make this easier, both for patients and providers, to deliver and receive healthcare in a more efficient way. If you take a step back and think about first principles and saying, hey, if I was a patient, how do I see my doctor at Mass General Brigham? I drive 30 minutes, I wait 15 minutes to park, I pay for my parking, I walk into the building, I get directed to the right clinic or lab. I then wait another 15 or 20 minutes while I'm filling out a questionnaire or two, bless you. And then eventually get the opportunity to sit with my care provider for somewhere around five to 15 minutes. At which point I follow that entire sequence in reverse. Go back to the waiting room, I fill out some more forms, I maybe get some lab work done, go back to my car, leave the parking lot, drive another 30 minutes home. And then I'm stuck in that process where after all that time and effort, maybe a half day, I had five or 10 minutes with my clinician to really understand what I need to do, what's important to me and how I need to change my health or behavior to optimize the quantity and quality of my life. And so we said that's... In today's day and age, there's a lot of opportunities to make that experience better, both for patients and for providers. And so we took a team-based remote asynchronous approach to everything from across the cardiometabolic spectrum. So overweight and obesity, hypertension, hyperlipidemia, type two diabetes, heart failure, and really making this a process where every patient who qualifies has the opportunity to do this with a physician always in charge, but really with a team-based approach so that they can have a high touch model. That didn't break the financial system. And so we were really fortunate to publish a lot of that work, circulation, JAMA cardiology, again, kind of these nerdy academic journals where we show this is safe, effective, and getting patients on these medicines to optimize their health doesn't have to require as much of the four walls, as much of the processes and the many steps that we sometimes put in place, but ends up getting in the way. Of delivering that really high quality care. And so that was a really exciting process. And we got to study this as a field of science called implementation science. Know, publish that it's safe, it's effective, it helps us deliver better care, patients like it, providers like it. And what we found was the biggest challenge and even getting these studies done, let alone getting this done at a population health level to really optimize care and care delivery for many numbers of patients. Was finding those right patients. And what we found was you really had to take somebody who knew the clinical medicine, who knew how to read a chart and say, okay, here's our criteria for this study or this trial, or for people who even qualify for this medicine. And Chris, as you know well, the electronic health record, unfortunately, is not always the easiest set of technology to work with. And so, you know, there's structured data, that's a component, it's important, but unstructured data, oftentimes really hides the diamonds. And that's where you really need to understand what has this patient tried before? What is the reason they qualify for X, Y, or Z medication or procedure or diagnostic? And what's been discussed with them in the past? And how do you make this a seamless experience where you're not inundating the system with best practice alerts or other ways of nudging

Chris Hutchins: 9:00

Right.

Dr. AJ Blood: 9:00

Patients or providers to consider things when you can say, look, The technology has advanced to a state where we can say, I can have a co-pilot or an assistant go through the chart, find the relevant information, surface that to a study staff. And what we showed in those journals that we mentioned was that in the New England Journal, we can safely, accurately, and cost effectively do this with the assistance of an AI tool to make it a much more efficient process. The JAMA randomized control trial that we did showing that using this process, we can actually get patients enrolled in clinical trials faster. And really at the end of the day, what we can do by enrolling in those trials faster is show that these medicines are safe and effective sooner, get these medicines into the hands of patients with the collaboration of their providers, and really accelerate the scientific process. And I think that's what we're really excited about at AIwithCare.

Chris Hutchins: 9:55

This is a really difficult space, I think, sometimes. And I maybe kind of pause on this for just a second. Talk to me a little bit about some of the dynamics. When you mentioned, you know, the patient maybe just has a few minutes with their clinician, they gotta drive home and all that. So there's a couple different areas that I think really are significant in terms of helping people understand just what it was that you guys had to deal with in order to do this work because I think most organizations that are going through any kind of turn you know acquisition or things they know there's a lot of problems and challenges when you're consolidating electronic health records and deal with out all that kind of stuff. But when you're trying to optimize a workflow like this, one of the barriers has to be drive time and the inconvenience for a patient to get in and out of a city. Not everyone has that dynamic, but that's another layer on top of what you guys We're dealing with the most anyone else would have to deal with as well, right?

Dr. AJ Blood: 10:53

Yeah, I couldn't agree more that the workflow is essential and that's both from the clinical team, the research team and from the patient. So anything that we can do to make this less onerous, I think we all just know this all too well. There's way too much friction in the system. It's not easy to be a patient. It's also, you I think we see from some of the burnout metrics and others, it's not easy to be a provider in today's world. And so I think as opposed to kind of remain as the status quo and kind of rage against the machine in many ways, I think we decided we're going to try to create a better mousetrap here and understand if we can take something that will work in the workflows that we've already kind of established that people are used to, but use some of these tools and technology to transform it in a way that accelerates efficiency and makes it a lot faster and easier, both for the patients and for the providers.

Chris Hutchins: 11:47

Yeah, I think it's a great illustration of what's really what it really takes in order to move things forward in a way that's actually gonna be effective. This clearly you guys didn't just go off and, you know, create something in a silo and toss it over the fence and tell everybody to use it. You really had to have the right experts in the room to work through all the nuance and make sure that you're accounting for the things that are really critical, not the least of which is how long is a patient gonna have to travel to get there. How do we make it interesting to them so if they live two hours away, they may not be inclined to join the clinical trial if you don't have things in place that's attractive to them. Are you were there other things that are considerations for patients that you've experienced?

Dr. AJ Blood: 12:28

Yeah, I think that when you think about a clinical trial, I think the most important thing is, this right for me? And does this really apply to me? And really understanding. And that's what's happened over the course of the past few decades is clinical trial protocols have gotten increasingly complex. 20, 30 years ago, maybe there was five or 10 what are called inclusion and exclusion criteria to make sure that this new drug or device or innovation is tested. In the right population to show that it has the best chance of working and being effective. In the past 10 or 15 years, what we've really seen is these trials and trial protocols as medicine has only become more complex, have also gotten more complex. And so what was five or 10 inclusion exclusion is oftentimes 40 or 50. And so when we reach out to a patient who might be eligible, we have to kind of go through each one of those questions one at a time, only to find it criteria number 47. Darn, you actually don't qualify. So I think one of the major things that is something that patients don't like is to go through this whole process, be offered this great opportunity and only to find that, well, you're not really a candidate at the end of the day. And so trying to make sure that every time that we reach out to patients and interview them and really... That they're the right patient for those trials. And that's what a lot of these, what this tool can really solve for is, you know, again, quickly, accurately and cost effectively taking that protocol criteria, matching it to their clinical data and both structured and unstructured elements to make sure that they are a perfect fit for the trial. Now there's a lot of other elements of, you're right, absolutely. How far do they have to drive? How many study visits are there? What is involved in participating in the trial in terms of... Am I getting blood draws? Am I getting transfusions? Is this a pill I'm taking? And I think that what condition that trial is for very much makes a difference

Chris Hutchins: 14:20

There is my participants.

Dr. AJ Blood: 14:23

in terms of their appetite for participating. If this is Stage four cancer where this is a drug that might extend the quality or quantity of your life, people are more apt to kind of give that a whirl if every other therapy has not been effective. If this is something that's might lower your cholesterol, which might eventually lead to a lower risk of heart attack or stroke. That is important too as your local friendly cardiologist, very important, especially

Chris Hutchins: 14:50

Right.

Dr. AJ Blood: 14:51

At a population health level. I think the impetus and the urgency for that, we have not done as good of a job in the cardiology community talking about the fight against heart disease as they have the fight against cancer. And when your mortality is kind of staring you in the eyes, I think there's... A much higher uptake rate in the clinical trials in the oncology space than we see in some of the preventative medicine trials that we do in cardiology. So there's many different elements

Chris Hutchins: 15:17

Okay.

Dr. AJ Blood: 15:17

That go into that decision, both for a patient or provider to make the call that a clinical trial is the right step. But I would say in terms of advancing

Chris Hutchins: 15:23

Okay.

Dr. AJ Blood: 15:25

science Advancing medicine, really understanding to the best scientific way that we can, how can we make healthier, happier, longer lives for people? The evidence-based

Chris Hutchins: 15:36

Right.

Dr. AJ Blood: 15:36

Medicine process is essential. And any way that we can take tools and technologies to accelerate that is really only gonna lead to a higher quality and quantity of human health. And that's kind of what we're aiming at in expanding access to this tool and technology.

Chris Hutchins: 15:51

No, excellent. Then it just it strikes me just how complicated things are. And I don't know that in healthcare as a rule we do a great job helping people to understand the complexities of what we're doing. Because what you're describing is just it's a workflow inside of a particular specialty, inside of an organization, inside of a practice. I mean there's so many other nuances to it. We're just talking about a sliver of it. I just think I think it's worth noting for people just how complicated this is and how serious you better be about doing research because it's a lot of work if you want to get this right. So have you talk a little bit about the tools that you're using? So what does RECTIFIER do? Walk me through what happens when it reads a clinical note to judge trial eligibility. And where is it confident and where does it still have to hand it back to the to a human? And I think the other interesting question would be. Is there something is there a point in that process that you automatically are building in that pause to make sure that you're getting human judgment engaged?

Dr. AJ Blood: 16:55

Really important question. And so to step back and tell the story, we built this tool for my clinical research group. I'm a principal investigator of clinical trials. I was looking at our kind of internal metrics on how much time and effort we were spending looking at charts of patients who ended up being ineligible in addition to talking to and working with my study staff who were spending. Hours and hours a week sifting through these charts kind of one at a time, you know, the unfortunate reality is 90% of the charts that we open are for patients who end up not being eligible for the trial for one of those inclusion or exclusion criteria or another. And we only really can know that when we have a highly trained clinical study staff go through that process of assessing their eligibility and reading that chart. Now that takes time, it takes expertise. And was the biggest bucket of costs that I had as a researcher and a principal investigator on the trials that I ran. So, you know, in addition to their feedback that, hey, you know, Dr. Blood, this is really not fun, you know, spending hours and only finding out, you know, after many hours of work that nine out of 10 of these patients, I just wasted my time and I can't even reach out to them to solicit their interest in participating in study because they're not eligible. But I only knew that after I spent all this time and effort. Checking for it. I said, there's got to be a better way here. And I really was very fortunate to have the sponsorship and mentorship of some of my colleagues at the TIMI study group to really say, you know, let's see if we can do this better. And we built this tool called RECTIFIER. And what RECTIFIER does is it really looks at both the structured and unstructured data and the electronic health record. And it assesses based on those inclusion and exclusion protocol criteria, not only is this patient eligible or are they not eligible. But why? And show me the evidence and cite your source. So what it can actually do is then tag back in the electronic health record why

Chris Hutchins: 18:48

Right.

Dr. AJ Blood: 18:48

a Decision on eligibility or ineligibility was made by the artificial intelligence model. And for really importantly, for each and every one of those decisions,

Chris Hutchins: 18:54

Yep.

Dr. AJ Blood: 18:57

these are all human in a loop, right? So there is never an autonomous AI process where we're letting the model run wild to make decisions or medical decision or even give medical advice in any way, shape or form. This is surfacing and sourcing relevant information to a protocol question about a clinical trial and allowing that clinical study staff to confirm or deny. And really just making that process much more efficient has shown that we can save enormous amounts of time and effort going through. The ineligible charts, but also surfacing those eligible charts and confirming eligibility much more quickly and accurately. So that's been where we've saved an enormous amount of time and effort on the clinical research front. This was built initially, again, in my research group at Brigham and Women's, but it's scaled across, Mass General Brigham, across 30 or 40 different use cases within the organization.

Chris Hutchins: 19:43

Right.

Dr. AJ Blood: 19:44

And clinical research is a really great one. It's showed an enormous amount of promise there and some really great results and even the randomized clinical trials that we can accelerate the scientific process by using this tool and technology. What we've then done is take that and license that out from Mass General Brigham and into a new company where I am happy to serve as the co-founder and CEO. That company is AIwithCare and we're now working with many different hospitals, healthcare systems, research organizations, and pharmaceutical companies, CROs, and sponsors to help accelerate the pace of scientific progress. Really excitingly, as a clinician, some of the things you never know what's going to happen when you bring a new tool

Chris Hutchins: 20:24

Yeah.

Dr. AJ Blood: 20:24

To a bunch of really smart, innovative people. And within Mass General Brigham, we've seen them start to use RECTIFIER, which is the tool, powers this technology, for many other use cases. So we've seen clinical population

Chris Hutchins: 20:35

Okay

Dr. AJ Blood: 20:38

Health, safety and quality metrics and reporting really start to be enabled by this registry work that's been published in some of the work that we've been able to accelerate the pace

Chris Hutchins: 20:48

Great.

Dr. AJ Blood: 20:49

Of registry creation, upkeep. With some of these AI tools. And really excitingly, in addition to other research groups and different therapeutic areas outside of my realm of expertise as a cardiologist, but things like ALS, a really fantastic work by Dr. Morganroth at Mass General Hospital in terms of screening every single patient she sees in clinic for every trial she's running to make sure that they get that opportunity when they see her, if they qualify. To participate in a trial, ALS is one of those disease states, time is really essential. And so making sure that you have that right in time opportunity to get people engaged and participate in clinical trials is really exciting. Another Dr. Chris Moran, a pediatric gastroenterologist, actually using this for clinical operations. And this is, Mass General Brigham put a press release about this, but a really exciting utilization of the use case where he's taking all the referrals that he's getting from the community. Running that through a customized version of RECTIFIER so that he can assess for any concerning findings or red flags that would warrant that patient being seen earlier. And again, these are kids. And so being able to then use that tool to flag these charts for his oversight and confirmation that yes, there is something in this referral that we need to see them faster and sooner. Making that offer to patients of getting in sooner, getting a procedure, a diagnostic procedure done on the earlier side actually makes a really big difference. Again, time in these disease states, especially in kids, is essential. Getting their disease under control, disease modifying therapies so that this is inflammatory bowel disease and others, and really understanding what's happening there and kind of nipping it in the bud before it blossoms into a much bigger problem. It's been really fascinating and incredible to see this tool being used in clinical operations to help Diagnose and treat disease earlier. So I think it's really just been an honor to be a part of this process and really exciting to see this used in so many different facets of our clinical operations and clinical research here at Mass General.

Chris Hutchins: 22:48

Well that's the stuff that gets me excited. Cause I mean you're doing the work, but you're starting to see some measurable results. Let's talk a little bit about that too. You know, would you what is the bar that you have to clear to satisfy the research community that yes, this is actually this is the right way to measure it. We're seeing results that we can trust.

Dr. AJ Blood: 23:10

Yeah, thanks so much for the opportunity. Yeah, so we were fortunate that the research studies that we published in the New England Journal, you're showing that our AI powered tool was actually more accurate. So we designed the study to show that it was non inferior. So it wasn't worse than our study staff, because again, our study staff are fantastic. They're essential. But like many in clinical practice and clinical medicine, there's always more work to do. Right? Like there's many patients out there's many patients waiting. So we really wanted to be able to give them a boost of time and efficiency. And this AI tool has been able to do that. And again, it does it by surfacing the relevant information, finding the eligible folks first, and making sure that they're much more efficient in their patient identification and screening. So. All of these processes are human in the loop, decisions driven by humans, clinicians, anytime there's an eligibility determination, but making that process more efficient is where there's a gap. And I think Dr. Moran is an example in his pediatric GI clinic used to have a nurse practitioner, you know, have to kind of go through all these referrals, sometimes getting a hundred or more referrals a week and kind of sifting

Chris Hutchins: 24:19

Well you can do that.

Dr. AJ Blood: 24:20

through and quickly understanding, is there an emergency Here? Is this somebody who really needs to be? Seen sooner, assessed sooner. And what we saw in some research he's done on this effort is that the referral from the pediatrician, whether it was dictated as emergent, urgent, or routine, actually made no difference in terms of the actual urgency of the procedure that the child needed. And so, and using the AI tool, that actually was not true. The AI tool did make a difference. It was actually significantly more efficient in identifying red flag signs or symptoms from the patient's chart and that referral. Now where the real lever up came, and this is how we're designing all of our workflow and the implementation of this technology is the flag identified by the AI and confirmed by a clinician. So allowing that human in the loop to really be an amplifier and the validator. That says that yes, the AI flag says there's something there that's confirmed by a clinician. And then what they did was offer these earlier appointments to these patients who were flagged as potentially high risk. And what that did was it reduced their no show rate. Decreased the time to which they had the diagnostic procedure and therapy. And so it got these kids seen sooner. Less likely to no-show to their appointments and on treatment sooner, really getting their disease to control sooner, which was really exciting. And so for all these things, I think our philosophy around this, both in my academic hat as a researcher and clinical implementation scientist, as well as CEO of AIwithCare in this company that's now taking this tool and technology to many other hospitals and health systems is to make sure that we're really building systems and workflows really importantly in place that keep that human clinician With their hand on the wheel, with the decision in their hands, but empowering them with this data so that they can actually do that much more efficiently. And the ways that they're spending their time is actually in gratifying aspects of the job. And I'll tell you, as a clinician, as somebody who works in clinical research, these people go into it because they wanna make a difference, right? Like they wanna have impact, they wanna improve scientific discovery, they wanna improve the quality and quantity of patients' lives. And really where we have the tangible feedback in doing that is when we have the

Chris Hutchins: 26:33

Right.

Dr. AJ Blood: 26:36

opportunity To engage with patients, when we have the opportunity to shepherd them through a clinical trial. So it's really exciting to be able

Chris Hutchins: 26:41

Yeah.

Dr. AJ Blood: 26:42

To insert technology at a point in the process that is a major bottleneck, a major inefficiency, and where AI is really well suited to deliver massive returns on that investment. And that's what we're seeing in our partners, both here at Mass General Brigham and through the lens of AIwithCare with many other hospitals, health systems, and research groups.

Chris Hutchins: 27:02

Now you from a trial standpoint, if the results were essentially you doubled the enrollment across the roughly forty five hundred patients that were that you were looking at to you know, to determine eligibility. From a scale standpoint, what is typical in terms of what how big that cohort has to be for in order for you to get statistical significance enough that you can actually you know you can validate that the what you've come up with is the right approach.

Dr. AJ Blood: 27:35

Great question. And so we were fortunate to have this study powered. Our p-value is 0.04. You very kindly noted it was an 84% increase in enrollment. So nearly a doubling of enrollments, prospectively, across the trial that we published in JAMA, the Journal of the American Medical Association. And so when you look at metrics like that, it's really transformative. So when you look at clinical trial enrollment, to see any site or principal investigator enroll two or three or four patients is really, it depends upon the trial, but in general, seeing is a pretty good performance. You really, you can imagine cancer trials where the numbers are small and the timelines are smaller is very different than a primary prevention trial and hypertension or cholesterol management, which can have many, sometimes tens of thousands of patients and last much longer, years or even three to five years sometimes. Dynamics of the trial can be very different based on what the question you're asking and the statistical significance that you're trying to show. In terms of trial enrollment acceleration, that's actually a much simpler question to answer because these timelines oftentimes stretch out months to years. And so what we see in being able to show, we were fortunate to have statistical significance in our published results, but being able to show a 20, 30% increase in enrollments across any one individual site. Let alone across many sites across the trial, that's where you actually get to shrink those timelines of the trial. And again, when they're essential, you have to do these trials right to really prove that these medicines or innovations are safe and effective. But being able to get through that trial faster by enrolling people faster, by getting them on the therapy faster, by demonstrating that safety and efficacy sooner. That's where you can actually compress that trial timeline and get those innovations, those medicines, those devices to patients sooner. And that's really what we're aiming

Chris Hutchins: 29:28

Right.

Dr. AJ Blood: 29:29

For here. Those trials can oftentimes again, last sometimes two to four years before our trial is completed. And so if you can imagine enrolling those people faster actually can compress that timeline, but. On that order

Chris Hutchins: 29:42

Great.

Dr. AJ Blood: 29:42

Of magnitude. And that not only takes an enormous amount of time and effort away from the study operations, safety, quality, keeping the study coordinators, the PIs and the patients engaged and part of the process and getting the data flowing to make sure that you have what you need to file that regulatory filing for eventual approval. But also, you know, get it. That's an extremely expensive process. Mean, something out of Deloitte a few years ago, it's about $2.2 billion per clinical trial for the top 20 biopharmaceutical companies who run clinical trials. Extraordinarily expensive and a huge amount of that cost goes into clinical study operations, clinical research staff, but also the time that they waste in terms of, you know, keeping that medicine under patent protections and making sure that's an ROI for the company that's really spent all this time and effort developing this new device drug or therapeutic. And so I think that's where we see an enormous amount of efficiency opportunity, both on behalf of our pharma partners, but also the health systems that run these trials, and then eventually to the service of patients.

Chris Hutchins: 30:51

W once it's running across multiple sites, how do you catch drift or bias before it has detrimental impact at a scale that no one wants?

Dr. AJ Blood: 31:02

Yeah, so Chris, this is essential. So I think you're right to call this out. And we, you know, we address this head on and upfront. So both in the New England Journal article, a peer reviewed publication, we looked at potential bias in incorporating this tool as well as in the Prospective Blinded RCT that we publish in JAMA. And so, Coming at it with an intentionality, I think is a first step, right? And making sure that you're aware that you need to be watching for this, right? You need to have systems in place so that you don't look back a month or a year later and say, whoops, you know what, we should have. And so being thoughtful, intentional and strategic and planning for that, I think is really the most important first step. So when we publish our New England Journal article, when we published our JAMA article, both of those studies showed no bias across age, sex, race, ethnicity, primary language. And so we wanted to make sure that we kind of built that monitoring capability into the platform, into the functionality of the tool and technology. So that is something that we have. Both to ask all those questions, not only as a whole system, but to also do that across kind of question by question. And not only in the screening aspect, which is really important, but also when you get to enrolling those patients. So our platform allows not only for the pre-screening, but also as you flow those patients into the clinical trial, as you reach out, engage and enroll those patients, actually monitoring that in live, real time. Not only across one site, but many sites is really important for that trial and the trial demographics. And really understanding that this medicine is not only gonna be safe and effective in this one very narrow slice of the population, but much more broadly valuable. You have to make sure that these trials are representative to ensure that they're safe and effective in all populations that might benefit from it and not just the one that happens to enroll in the trial. So I completely agree with you, building structural ways to make that. Easy and make that part of the process for the local study teams as part of the platform that we've built is part of the way that we've made it part of the workflow. It's part of the process is ingrained from day one so that you're not way behind the ball looking back and saying, whoops, I should have, but instead saying, great, we're on track, we're on metric, or we need to tweak. We need to let's dive in here and make sure that we're, you know, make it. Activating or reaching out to some patients of this population who haven't decided to participate as much. And I think being able to be aware of that is the first step and making sure that you're making it easy for those study teams and for those patients to participate in the trials is a really important part of the scientific process.

Chris Hutchins: 33:44

I think there's a the whole idea of dealing with this kind of scale is interesting and I hear a lot about pilots these days and this I would say the majority at the moment seems to be really not so happy with the whole idea of pilots for a variety of reasons. Usually it's the lack of a an operational game plan because you talked about you know baking things into the operations. It seems to me there's probably some things, maybe even a blueprint that lives inside of the clinical trial processes that have been developed over decades that could maybe help organizations to think differently about what they're implementing. And this is not just a clinical trial for one particular condition. There's a number of different aspects of what you're doing that benefit other parts of the care delivery that goes on in an organization. Mm I'm definitely interested in your perspective on it, because I think that there's probably some things to learn just from the workflows that you guys have in place. So when you're yes, it's a clinical trial. It may work, it may not work, but you have the gears and the foundational elements that are undergirding it right out of the gate. Otherwise you can't trust the data that you get out of it. Is that fair?

Dr. AJ Blood: 34:57

I completely agree. And I think there's a little bit of a parallelism you're hearing kind of in even the tech and kind of AI world right now, right? It's not the model, it's the harness. And I would say similarly kind of in our world, it's not the platform, it's the workflow. Right? Like you can have umpteen pilots and, you know, ways of testing. And I think there's a lot of pilot fatigue. There's a lot of kind of death by pilot that does happen kind of in the healthcare system, especially in the academic medical systems that I've had the fortune of working in. But I think the way that you can solve for that is by saying, how do you make this Akin to the workflow that the team is already used to doing and only make that more efficient. If you're adding steps or completely revolutionizing and transforming what these teams of people are used to doing, change is very hard in healthcare at a baseline. And changing one thing can break the gears and, you know, explode the machine. I'm sure you've seen that before in some of the work that you've done, Chris. So I think that the more that you really have teams the people who understand those workflows, who understand those team members, the problems they're dealing with and the things they're trying to solve for. I think that actually becomes really essential. And I feel very fortunate to be in the position that I'm in having both a clinical background and a clinical research background to say that I know have lived and worked with all the people that actually make these challenges and overcoming them their day job for decades at this point. So, being really not just like Knowing it, learning it, but living it, I think is an essential component of designing any solution such that the end user is going to get benefited at the end of the day. And I think, you know, there's a couple of truisms that I like to come back to, right? Like, you know, AI at the end of the day is just a data model, right? Data in, data out, garbage in, garbage out. Know, so AI cannot revolutionize or transform or make magic. You really still have to start with the data and the quality of the data that you have as an input. Another truism is, before you think about revolution and transformation, in healthcare it's rare that we see massive shifts in the way that our workflow changes. It's oftentimes stepwise. And the steps that actually move us forward are ones that actually make lives easier for the people who actually have to take the steps. So that is true at intake, as it is at triage, as it is at vitals, note taking. I think some of these ambience have made things, have exploded in use because it hasn't changed massively what we do. There's not a whole ton of additional inputs or transitions that we have to make to actually make that a part of our usual daily process. And so I think the more that we make this easy, both for principal investigators, for clinicians, for research coordinators, and for the people that support all those processes, and ultimately for patients, the more likely we are to be successful in these efforts.

Chris Hutchins: 38:04

From your perspective, having been the bedside, but also speaking with boards and executive teams, when a tool stalls like this, is it the technology, is it the trust, or is it the workflow that you think would be the primary things that are most often being tagged?

Dr. AJ Blood: 38:24

It's a great question. I would say it ends up oftentimes being a little bit of all three. So the technology is transformative, but pointed at the wrong data sources or used in the wrong way, it doesn't help anybody. And so if the technology, and I think again, whenever there's a new innovation, it oftentimes kind of permeates the lexicon, Like how are you, if you're not using AI, you're doing something wrong. So how are you using AI? So if you're not, but the challenge is to just incorporate something quickly and say, well, we're using AI for this, but it doesn't, know, it either narrowly fixes one small thing that actually wasn't the major barrier to what, you know, getting a process completed was, that doesn't make a big Delta. At the end of the day for the team and for your outcome of interest. So I think that the technology has to be there. That's obviously table stakes. To answer your question, I think the workflow is oftentimes the biggest challenge because a lot of times in technology, what I see is a lot of technology vendors assume that I'm going to bring this new widget to the table and everybody's going to be so excited about my widget that they're going to just drop everything else they're doing and use the widget. And the truth is, that is almost impossible and clinical workflows and especially research workflows are extremely complicated involving many different systems, many different platforms, many different data sources and elements, many different stakeholders. And so I think what you really need to find is something that works well in the workflow. And I think that's actually quite hard to do because no matter what you're changing some amount of behavior and changing behavior for clinical teams, for research teams, we're a bit... Very, for good reason, risk averse. And so any sort of change comes with an enormous amount of apprehension. And I've seen the simplest workflow changes in the hospital environment sometimes take a year or longer. And I'm sure you've seen worse than that, Chris. But it's because we've done it this way for a long time. You know, we know this is safe. Know it works. Changing something introduces uncertainty. And we, you don't even know all the ramifications of, all right, how's this going to affect biomedical engineering? What about our tagging process? Is our barcode system ready for this? Know, QR is better in so many different ways, but this is going to now affect, you know, its connection to Epic or the EMR. It's going to now affect our workflows and our scanners is going to affect, you know, the... Snowball effect of a change in medicine is much more so than I've ever seen in any other industry. So understanding that and coming to the table with that knowledge and acceptance and really trying to make as narrow as you can some of the changes that you're affecting in the workflow, I see as the best way to getting clinical uptake. And that's different than getting board uptake, right? So board uptake, C-suite uptake, they want to hear transformation. They want to hear... AI enabled from beginning to end, they want to hear 20, 40, 80% increases or decreases in cost or increases in efficiency. The rank and file, the people actually implementing these tools and changes, the ones who are going to be directly working with the systems or the patients that make this possible, that is not what they want. And it is very difficult to convince those folks that revolutionizing a process that has worked well for years or decades, depending upon your place of practice, getting buy-in from those folks is essential at the end of the day, but really hard to do.

Chris Hutchins: 42:02

Right. What would you say to the executives or CIOs that you know are in a process of trying to figure out what solutions they should be moving forward with and what's that criteria look like. What would you tell them is most important to really consider and maybe it's more than one thing, but before you move into a move through a contracting exercise, thinking that you've got everything covered that you need. I think you guys are not often I say you guys, forgive me, physicians like yourself are typically not coming into the contract conversation. Which to is just the it's the natural flow that everything should follow if it's having anything to do with clinical care.

Dr. AJ Blood: 42:41

Yeah, it's a great question. And I think you're right. Are, I think in many ways, kind of entering the valley of disillusionment and kind of the cycle that we are in this new technology or disruptive innovation that we're seeing with the advent of AI. And so, you know, I think AI is a powerful tool. AI can be transformational in its capabilities. Health care forever. Has been a lagger in terms of uptake for many reasons, risk averseness, patient safety, first do no harm, and many other tenets of healthcare, being good reasons for that. I think we're also at a point of impetus where there's a strong sense both from internally and externally within healthcare that there's an enormous amount of inefficiency in the process and that we need to do better. And so when I talk to the C-suite and CIOs and other folks at hospitals, health systems that I have the fortunate to work with and partner with, a lot of this is let's not boil the ocean. Let's, you know, again, and not saying, you know, death by pilot, but picking some discrete milestones and metrics around which we're going to make a decision of go, no go. And the good news is generally with AI, those don't have to be six, nine, or 12 months out. These can generally be relatively tight timelines in terms of the results or the kind of process or outcome metrics that you're looking to track as a result of any new innovation or integration of a new tool or technology. So I think being tangible, be discrete, setting timelines and keeping to them. And you know, again, like failing fast is oftentimes, you know, something we, like to talk about a lot in tech, not nearly so much in healthcare, but I think some people are seeing some of these, you know, 85% pilot fail rates, not as a bad thing and as an opportunity to say, well, where is this really going to help and where is it going to make a difference and where are some of these process and outcome metrics really going to be, you know, as great as promised. I'll say, you know, some of the data that's coming out of the ambient space and I have many friends and great partners who work in that space. So this is not meant to cast shade, but to say, these revolutions in time back to clinicians, we're certainly not feeling and from the data, we're also not seeing, right? Like I think it's some single number of minutes that are coming back to clinicians, daily schedules with the advent

Chris Hutchins: 45:14

Yeah.

Dr. AJ Blood: 45:15

Of some of these tools and technologies. And so, there's value in that. I'm not saying it's not valuable. And I think there are other reasons that ambient adds value. But I think a lot of the promises around you're going to get your nights and weekends back and documentation after hours is no longer going to be a thing. We're not seeing that, you know, become a part of our reality as much as was hoped. And so I think when you think about these pilots and understanding, you know, what you're really looking to change and aim to revolutionize, like I think we really want to be as intentional. As rational and reasonable and as firm as we can be about what we're really aiming to achieve.

Chris Hutchins: 45:58

It's not an easy bar to clear

Dr. AJ Blood: 46:01

Yeah.

Chris Hutchins: 46:01

For sort of a thing. And I think the things that are really challenging I think are when you're trying to get something across the finish line, just to get it a green light to go forward. There's I think there's some different elements that probably are not included in the assessments that probably would be better if they were. Which really it's some of the key things it will either cause there to be a good level of adoption or they there might not be. And I just think about how for years and years when I've worked with data solutions that my chief data officer role that's that was almost all the time. We're having conversations around what's the ROI, what's the risk, going into these different conversations around what you deploy. But you talk a little bit from your perspective as we wrap up here. What are some things as a clinician that you see are often not represented in the overall structure that you by the time it gets to you and people already made decisions and they've done development work? So I think that's an area where we probably fall down more often than not when we're trying to operate that way. Instead of keeping you involved and building with you alongside of you. We go off and we'll take what we remember that she said and six months down the road we're bringing you something that doesn't match that at all.

Dr. AJ Blood: 47:25

Yeah, I think as much as you can, and we've seen this in fits and starts, but really having the developers and designers almost elbow to elbow with the end users. And that could be a clinician, that could be a nurse, it could be a PA, it could be an MD, it could be the front office staff. But really understanding, and I think people come to the table with these preconceived notions of what their day actually looks like and what they're doing and what their challenges are. And that's oftentimes informed by their managers or by somebody even more senior than that. And I think oftentimes there's a lot of little stuff that gets lost in translation or that gets lost in communication where, you know, I need to be able to pivot, you know, and if I get a slip from one of my docs, I need to stop everything and go over here and do this. But if I come back to my computer and I'm logged out, I just lost like four minutes to do my two factor authentication to log back in and I can't do it. I don't have time because I've got patients standing in front of me in line waiting to register and I can't sit here and log back into this tool. That's only saving me a minute or two a day because I've got a

Chris Hutchins: 48:27

Yeah.

Dr. AJ Blood: 48:28

I've got more pressing needs. And so I think really, as much as you can, these kind of, and it's becoming in vogue, forward-deployed engineers, really understanding before the tool is in its final form, before this product is really even getting surface to the end user in a production environment, really pressure testing this at every stage and not like. Three or four months apart, but like three or four weeks apart so that you're really, as you get into wireframe, as you get into prototype, as you get into pilot, like you really want this to be as successful as earlier on in the process as you can, because when you lose buy-in, especially clinically, it's extremely difficult to get that back. So I think that's something you want to engineer for.

Chris Hutchins: 49:09

Yeah. No, yeah, I think you'd probably agree with this, but the challenge of the funding model always gets kind of battened around and it's sometimes it just it stalls things for months at a time. And there's just this perception that well, if we add the operating tail into the budget that we're building to do the program, it's gonna inflate it and it may not be that high, but it's usually a gap and it's not even there. And so you're moving great cons, you have a successful pilot. Now everyone wants to roll it out to the enterprise, and you guys are like, we can't do that yet. We don't have everything that we need to support it. We'll we talk a little bit about that from your perspective. And I know that there's people listening out there that are decision makers and in healthcare organizations for sure. But they've heard me say stuff like that, but I think you can give it some credentials and some weight. So I'd love to hear you kinda address that one.

Dr. AJ Blood: 50:04

I think it's a Ben Franklin quote, an ounce of prevention is worth a pound of cure. We like to talk about that in preventative cardiology and medicine, but I couldn't agree more. Having this, even in small bits involved early on in the process, before you make a whole ton of architectural structural decisions that are gonna impact the end user experience, I would encourage as much as I can, that getting those stakeholders involved early can make a really big difference, but also picking those stakeholders intentionally. Oftentimes it's not the division head or section chief or executive director who's going to be the end user of a lot of these tools. And they have experience that is undoubtedly valuable and can add a lot of color. But I think the people who are in the day-to-day kind of in the trenches who are going to be the ones most impacted by this are really the ones you want to engineer around. So I think that's sometimes a mistake that can get made is I want the most senior person, the most decorated person, the professor of X, Y or Z, you know, but I think sometimes that person is. Giving talks or speeches, off running divisions or departments, off doing fundraising and doing surgery or taking care of patients with a minority of their time. But the people who you really are engineering for on this are the ones who are much more on the front lines. And those are the people that you wanna really make sure that you get buy-in from because that's the vast majority of the volume that you're gonna be interacting with.

Chris Hutchins: 51:28

And thank you so much for coming on the show. This has been wonderful. I know that people are gonna really appreciate this. If you wouldn't mind as we close, just kind of let folks know about the website.'cause I think we you'll probably you'll see I'm guessing you're gonna get some feedback from this. See probably some folks that would love to pick your brains a little bit.

Dr. AJ Blood: 51:47

Always happy to engage and Chris, thanks so much for the opportunity for all the kind words. It's been a pleasure to have this conversation with you today. aiwithcare.com is the websites. You can always reach out to me. It's AJ at aiwithcare.com and really excited to continue this conversation. And Chris, thanks so much again for the opportunity.

Chris Hutchins: 52:05

Yeah.

Dr. AJ Blood: 52:05

Appreciate it.

Chris Hutchins: 52:07

It's my pleasure. For my listeners, that is gonna do it for this episode of The Signal Room. In the meantime, stay healthy, stay curious.