Radiology AI on Trial: The Wins vs. The Warnings

AI is rapidly expanding across the medical imaging sector, but does it truly improve patient outcomes, or does it introduce unmanaged risks to clinical workflows? 

In this episode, Stephen Pech and clinical radiologist Chee Chong host the ultimate showdown—weighing the major promises of radiology AI against the practical limitations encountered in real-world medicine.

What You Will Learn:

The Power of Smarter Triage: How AI reorders radiology queues to surface life-threatening cases earlier in high-pressure emergency departments.

Workflow Beyond the Image: The role of large language models and automation in eliminating repetitive administrative tasks for clinicians.

Mitigating Global Staff Shortages: Real-world data from European screening programs (including Denmark) utilizing AI as a trusted double-reader.

Unlocking Hidden Data: How proactive disease screening extracts vital risk biomarkers from routine, existing scans.

The Clinical Risks: Why faulty training data, automation bias, and a lack of regulatory transparency can lead to dangerous diagnostic pitfalls.

Transcript

SPEAKER_02 0:00

AI is better than humans in some specific areas, but everybody's worried about it, kicking everybody out of jobs. If we can really review faster and at high quality, that's good, isn't it? Chi, what do you think? Is your job in danger?

SPEAKER_01 0:17

AI is helping us to see things that we haven't been able to see in the past. I'm very excited for this kind of revolution with AI and imaging. Anything that makes radiologists more accurate is a good thing. It's a good thing for the patient, less resources for the government to have to spend on imaging. So I think it's a great outcome for everyone. And unless AI is 100% accurate and 100% all cases are better than humans, I think humans will always have in place because together we are better than either of us on our own.

SPEAKER_00 0:52

Welcome to MIAI, the podcast about medical imaging and artificial intelligence. Sit down with the most brilliant voices in radiology and AI to discover how technology is reshaping healthcare delivery today.

SPEAKER_02 1:08

AI is everywhere in radiology, but is it actually helping? Or is it just adding new risks? In this episode, we put AI on trial, weighing the winds against the warnings. From cases where AI beats humans to moments where it falls clearly short, we break down what's real, what's hype, and what actually matters in practice. It's for and against the promise and the reality, real world examples. Chi and Stephen. Let's do it. My name's Stephen Peck. And I'm Chi. Chi and I have spent the last few years trying to understand how AI will affect healthcare in the real world. In that time, we've researched, reviewed real world examples, spoken to experts, and we've used our own perspective and expertise to cut through to the core of the value and hurdles in radiology AI.

SPEAKER_01 2:06

As a radiologist, I live and breathe AI each day in my clinical practice.

SPEAKER_02 2:11

These are the key areas we've identified. Today we're going to discuss and review each of the fors and against. So I think it's really useful to start with a summary, Gi. First of all, the practice in reality. So faster triage and prioritization of urgent care. Number one, workflow efficiency beyond image interpretation. Number two, mitigating workforce shortages and rising imaging demand. Number four is unlocking latent scan data via opportunistic, proactive disease screening. Number five is a big one. In some defined tasks, AI can outperform human readers. There's a few people out there who'll be worried about that, Gi. And the last promise that we're going to discuss is extending radiology services to underserved settings. Now we'll go through these in detail in a second. But what about the practice? What about the stuff which might not be perfect? Well, the first one there is generalization failures and data set shift across sites. Bias and fairness gaps between demographic groups is number two. Number three is false positives and downstream harm, especially in screening. Number four is automation bias and over-reliance on AI and outputs. Number five, generation AI hallucinations and safety risks. And the last one in practice where it might not be hitting the mark is governance, transparency, and regulatory safety concerns. So let's start with the first positive, the first promise of AI: faster triage and prioritization of urgent cases. The explainer for this is this is where AI can reorder radiology cues to surface time critical findings earlier, improving time to diagnosis. What does that mean in reality? Well, the way I understand it is that we pre-scan an exam and we prioritize the high-risk patients to be reviewed earlier.

SPEAKER_01 4:28

Is that right, Chi? That is correct, Stephen. You've been doing your research very well. Generally in radiology, we report cases as they come in. Now there are AI products out there which look into the studies, find out where there's any issues with them, and puts them at a higher priority. So we report those before others. So we prioritize like an emergency department. That's correct, Stephen. So in emergency departments, you've got all kinds of medical issues. You've got patients who have got who've got a cut on their finger, you've got patients who've broken bones, you've got patients with bleeds in the brain. Now, generally patients aren't being seen based on when they arrived into the emergency department. They're based on who has the most emergent issues that need tending to. So in essence, these AI products are doing exactly the same thing, but by looking through the images to work out which ones are emergent.

SPEAKER_02 5:32

Okay. Great. That makes complete sense. I think everybody understands let's treat the person who needs the help quicker first. And that's just a better, more efficient way of handling the low inside a practice. It does it happen everywhere or just in practice or just emergency department?

SPEAKER_01 5:50

So I guess from a radiologist's point of view, does this increase my workload or does it change my workload? I think at the end of the day, because all the cases have to be reported, it doesn't matter which order you report them.

unknown 6:04

Yeah.

SPEAKER_01 6:04

The total time it takes is going to be exactly the same.

SPEAKER_02 6:08

Okay. That makes sense because you're still doing 50, you're just doing them in different order. But is this useful for a normal radiology practice where we're just taking private patients walking on the street, or is this really only useful in an emergency department or both the Where it can help in the everyday clinic is if there's an unexpected finding, you can contact the doctor a bit earlier.

SPEAKER_01 6:35

It provides a better service delivery for your referrals, such that patients who have issues are being tended to earlier. They feel that you're responsive, and it could be a selling point for these practices.

SPEAKER_02 6:49

Okay, makes sense. So it's it's super powerful in a time critical environment like an emergency department, but still useful in a non-time critical environment like a normal practice. So you're right before Chi. I have been doing some research. I wanted to get your thoughts on one of these scenarios where triage is being used really effectively. Now, this example comes from Denmark, and Denmark runs a population-wide screening program for mammograms. That program uses AI for triage. So it's a different type of triage, though. So rather than speed triage, it's level of effort. So in one scenario, the AI scans all of the scans. If it looks fairly normal, the AI then is backed up or followed up by one radiologist to review it. But if it doesn't look normal, it's then reviewed by two radiologists so they can independently read and make sure that there's no mistakes made. The double reading for the ones which looked complicated, abnormal, and a single reading for ones which are pretty thoroughly certain that they're not abnormal and they're pretty clean. Where AI was implemented after it was, it detected significantly more breast cancers. So I think the numbers were 0.82% versus 0.7%. And they also had a lower false positive rate of 1.63% versus 2.39%.

SPEAKER_01 8:22

Are they big numbers? On zone, they're small numbers. But when you're talking about population size like Denmark, yes, that makes a difference. This is kind of borne on the realms of using AI as a double reader. Essentially, it's reading the mammogram. If it's abnormal, it will flag it for someone else to read further. If it's not abnormal, it doesn't get flagged. So it's in that realm between AI being a reader and AI being a triaging tool, which I think is fantastic and useful as it does save time and does save resources. Yeah. Yeah.

SPEAKER_02 9:02

And it's backing up rather than replacing. People get worried about, hey, AI is, you know, we can't trust it to replace it. It's not replacing it, it's just an extra helper and making it more efficient. So promise number two, workflow efficiency beyond image interpretation. The point of this one is this is where AI automates repetitive workflow tasks like measurement, reporting, support, protocoling, quality checks to improve throughput and consistency. To me, this one is all about the work that happens before and around the radiologist to help the whole process be more efficient. It's not necessarily about the reading, it's all the other stuff beyond the reading. I can imagine a large language model helping write better reports. It's also easy to imagine AI upscaling or taking the noise out of an image.

SPEAKER_01 9:51

I think there's a huge space in AI efficiency in radiology. A radiologist can report probably quite comfortably a hundred CTs or MRI a day. They can look at images and interpret them with that kind of speed. The issue is for radiologists is all the other tasks that come along with the interpretation. So the measurements, as you mentioned, the after synthesizing an answer, the issues getting that succinctly down to the paper, to the report. And that's where language models can help. The protocoling that the radiographers constantly bombard you with when they are uncertain, these are all little things that when added together will help radiologist output.

SPEAKER_02 10:41

Okay. There's some real legs in this radiology promise. There are some real legs, but there's also some a few pits there. Great. We're on to number three. The promise of AI. This one is about AI driving efficiency, specifically mitigating workforce shortages and rising image demand. Now, from my perspective, this is about augmenting radiologists, the productivity and the demand component, rather than replacing them. We have a big demand for radiology. There are workforce shortages, there is increased demand. The reality is that AI hasn't stopped radiologists needing to be trained. The demand is just too big. This argument for me has real eggs. And it's commonly being used today. Decision support for a radiologist keeps the existing structures in place. It just makes things faster or better. What's your take?

SPEAKER_01 11:42

I tend to agree. I think I think radiologists, as you know, there's a huge shortfall across the world. AI helping us as a support tool, helping us with diagnosis, that all has input to help us speed up and become more efficient.

SPEAKER_02 12:00

We talked before about the mammography in Denmark, where we had two people reading in the past, and now in some scenarios, we can only have one radiologist who's needed to review. That's driving efficiency and helping support the increased demand in the workflow shortages. I'm sure the radiologists who otherwise would have been reading thousands of mammography scans are now deployed in other places. But another example here, Cheat, and I'd love your take on this as well, is building AI into the workflow so you get extra information and advice while you're reviewing exams. So it's not bit read for you, and it's not triage, but it's extra information at the time of review which could help you be more efficient, but maybe also help you get a better result. What's your experience in those?

SPEAKER_01 12:50

I think there are some products which help in specific situations and some which don't. A lot of it comes down to how much reliable and trustworthy those products are. So for example, there are products out there which look for PEs. Very good products, as good as a human. Radiologists can trust these products and it saves time. Then there are products out there which are a bit more dubious where you don't trust the results. So in those situations, you have your initial interpretation, then you have to correlate with the findings of this less trustworthy products. Then you have to synthesize the information and to understand the discrepancy between your interpretation and this product's interpretation of who is correct. Am I correct? Is Proga correct? Or were we both sort of correcting in some parts and wrong in others? Those do result in time delay. So I guess in summary, I think the products which are reliable are worthy of usage, and those which are a bit more ambiguous becomes harder.

SPEAKER_02 14:08

Okay. It comes down to the the reliability of the product. And if it's reliable, it can save you time. But if you have to second guess it, you end up wasting potentially more time. Alright. I wanted to ask you one more question, a bit of research again. The joint US and European study where they reduce radiology interpretation time of serial CT and MR imaging findings with AI, specifically with deep learning identification of relevant priors, series, and finding locations. What was your take on that result?

SPEAKER_01 14:42

Yes, I did read up with that, Stephen. It's an interesting study. So what it does is it brings you back to the areas of interest which were marked on the prior studies. The concern there is yes, you do look at those areas and you hit those areas. It's like a bookmark. If you had to go through a book to find the relevant information and it's been bookmarked specific pages, sure you can find those correctly. But who's to say that there's not extra information on those other pages that you missed?

SPEAKER_02 15:14

Yeah. So it's it's using AI to go through past scans and say, oh, this looks relevant to what you're talking about now. I'll serve that to you so instead of you having to look through it all, you're created these auto bookmarks and that's saving you a lot of time in that process.

SPEAKER_01 15:29

Correct. But the risk is being too reliant on the bookmarks and not looking at the rest of the book.

SPEAKER_02 15:36

But then the other risk, G, is that you've gotten so little time you don't even look for the bookmarks. So, you know, as if from a human perspective. So if AI is pretty good at it, surely the total benefit to society is so much higher with the time saved and the effectiveness of finding them when it may have may have not done it, may have taken you longer, or you may have had human error in the past.

SPEAKER_01 16:02

You're right there, Stephen. A radiologist who's relying on AI to speed through cases is better than no radiologist at all.

SPEAKER_02 16:11

Okay. The fourth promise of radiology AI, where we're seeing things happen in practice, is proactive disease screening from latent scan data. The overview of this one, to go into more detail, is where AI can extract actionable risk biomarkers from routine scans. These scans were done for other reasons. And then enable proactive disease detection. The way to view this is this data was going to go in the trash. We look at the scan once, we never use it again. But there's stuff in there that could help us identify other diseases we weren't even looking for if we know how to look for it. And I need to give a disclaimer here. I'm involved with a company that's focusing on this particular area. If you want more information, please check out my LinkedIn profile or see the notes in this podcast. But obviously, it's an area that I'm pretty interested in. I just see this as extra. So extra valuable information with no change to systems at negligible extra cost, and it wasn't being done before. What's your take, Chi?

SPEAKER_01 17:21

I think this is a large untapped market. Who turns it down a free pack of fries when they go to McDonald's? Very few. So very few. Very few indeed.

SPEAKER_02 17:34

I probably sh I probably should more often if that just happens.

SPEAKER_01 17:38

When you go see a doctor, you want to see as much role as possible. You want to stay healthy. Similarly, when you have images done, you want as much information as you can gather out of those images that were taken. AI is helping us to see things that we haven't been able to see in the past. So I'm very excited for this kind of revolution with AI and imaging.

SPEAKER_02 18:04

Let's work through this now, because there's one or two little question marks around this that we need to follow up. An example is you take an existing exam that was being done for another reason, like I said before, and straight after the scan it's reviewed by AI for an array of diseases. If something of note is found, then can be reported. But here's the point. You're looking at uh looking for a broken bone.

unknown 18:27

Yeah.

SPEAKER_02 18:27

And while I'm looking for that broken bone, you go, oh, wait, what's this? So you didn't see it in the scan, but the AI says, hey, did you note this? And it goes, Oh, that's a good point. Yeah, this it's great. Now, you as a radiologist, you get that information. How do you include that? And what does the flow need to be for you to include that in the reporting?

SPEAKER_01 18:47

So it goes back to trustability. If you can trust the AI, then you'd be happy to include it for the patient's sake, for the health economy's sake. So if you can find something earlier, we all know early diagnosis leads to better outcomes. That's a win. That's a win for a patient, that's a win for healthcare dollars. I guess going back to it, would you include it? You would include it if you believed in the result. And part of believing in the result is either you're able to verify it with your own eyes, but part of it is also someone else has done the legwork for you. Someone has looked at the research and felt this is a believable result. And luckily, in in most countries, there is a department that's in charge of research. In Australia, we've got the TGA, Therapy Goods of Australia. In USA, we've got the FDA. So in Europe we've CE. So part of it is being able to see it with your own eyes, and part of it is understanding that regulatory bodies have approved these products, which is why it's important to make sure that your use case is appropriate and being approved.

SPEAKER_02 20:07

Okay. So if it's approved and you have trust in that approval, then as a radiologist, somebody provides that extra information while you're reviewing a different issue, it's a positive use case for you to be able to say, Oh, this is really valuable, I'm going to include it in my report. And then that report then gets utilized, hopefully, down by the patient and by their carers.

SPEAKER_01 20:30

Yes, it's not only useful, but almost by law these days, if I'm reporting a CT, for example, and I find cancer, even though they're here for some abdominal pain, do I not report it? Of course you report it. This is extra finding that you weren't expecting, but you're supposed to report all the findings on a study. If you don't, by law you are very much liable for any issues that arise from your miss.

SPEAKER_02 21:02

Okay, so a great use case as long as you can trust it.

SPEAKER_01 21:06

Correct.

SPEAKER_02 21:07

And then the the regulatory authorities are obviously core to that trust. Yes. Next step is promise five. And this one's about AI beating human readers in some scenarios, not every scenario. This is something that people get pretty worked up about. So the one liner for this is in certain well-bounded imaging tasks, especially high-volume screening and standardized patent recognition problems, AI can match or outperform radiologists. Though superiority is task and setting specific. AI is better than humans in some specific areas, but everybody's worried about it kicking everybody out of jobs. Scary stuff. But it's also exciting. If we can really review faster and at high quality,

SPEAKER_01 22:00

That's good, isn't it? I am excited about the technology, to be honest. Like I think anything that makes radiologists more accurate is a good thing. It's a good thing for the patients. Uh it means less resources for the government to have to spend on imaging. So I think it's a great outcome for everyone. And unless AI is 100% accurate and 100% all cases are better than humans, I think humans will always have in place. Because together we are better than either of us on our own. So I'm not too concerned about Rayol just being phased out completely. I also think it's also unlikely to occur because correct me if I'm wrong, Steve. AI requires quite a large data set to create models.

SPEAKER_02 22:50

You need to create it based on existing relationships, and those relationships have been built by humans and the existing processes that we've got. So we're only building on the backs of the people and the knowledge that came before it. AI will be able to do that better and be able to find links between things, but it needs to be pushed in the right direction first.

SPEAKER_01 23:10

Yes. And I think that's where I'm thinking along the lines of we're moving very fast with medicine. And we have new modalities come into radiology, not uncommonly. We've got new techniques in MRI that happen reasonably frequently, actually. So these things are the cutting edge, the innovative, and because they're new and innovative, the data for this kind of new sequences, new modalities, aren't always there such that AI can create a model for these new innovative imaging techniques. So I think AI will always be that step behind. Whereas a human is able to read an article, and although they have never seen a case before, they'll be able to interpret that article and put into practice. I think we may require less radiologists in the future. If AI do us promise of making us more efficient, I mean, if we can get through 50% more work, then naturally you think we need 50% less radiologists. And that's similar to what happened with the airline model. So in airlines, I'm not sure whether you remember, on those old planes in the old 747s, you had three people in the cockpit and one as a backup. Do you remember what they were, Steven? What their roles were? Pilot, copilot, and navigator? Yeah, probably back then even further there was Navigator, but there was the person in the back just twiddling the knobs, making sure the engines were spooled and making sure that they had enough thrust. They were in charge of the engines and the mechanicals, they're essentially the engineer. So we've done away with that. And why? Because technology has come along and has replaced that role. They're still the pilot and co-pilot. So although technology has moved with the airlines, such that a lot of it is done, is automated by the autopilot. You still have humans, just a few less humans.

SPEAKER_02 25:10

Okay, so it's a good story. We are improving the technology, we will be able to get through more of it. Does that mean there's gonna be less pilots? Like you said before, that we've now gone from three to two, we don't need the engineer. I would have thought these days there's a lot more pilots. There's a lot more planes, there's a lot more travel because the travel's got easier and quicker. Same approach from a radiology perspective.

SPEAKER_01 25:37

I hope so. I hope that technology advances so much that although we need less radiologists, well, the radiologists get through more work. I hope that the work increases to offset this such that we we see deep and deep into the human body and make better and better diagnoses and treat humans a lot better. So that's my future hope, shall we say, soon.

SPEAKER_02 26:02

Yeah. Well, I'm on pretty much on the same page, Chi. I think we do need to be cautious when we embrace the technology and in certain areas with appropriate checks. The fear isn't unfounded, but it's probably a bit amplified. We did use an abacus, then we used a calculator, then a spreadsheet, now advanced software and AI. You know, we have moved along and people have been fine because we're taking advantage of the technology. The technology is going to get better, we just need to use it appropriately. I'm not really worried about that taking people's jobs. I think radiology, there's so much demand. There's an aging population, there's enormous value that can be mined out of radiology, which probably hasn't been up until now. And it's the the whole market is ready for a boom if we can get it more efficient. And AI gives us the ability to provide more efficiency. And the radiology scenarios that I've looked at so far with an AI perspective still have humans involved, just supported by AI. And if that, I think we'll be ready for the change if it does come. Also, another key point is that when we're worried about the area, medical devices get put through pretty rigorous tests before they go to market. And AI is the same. So I think we've we can trust our people to be able to make sure that the right technology is going to market and available to people. Promise six, the last of our promises. Extending radiology services to underserved areas. So this one is about helping AI capability get through to places where it couldn't before because of the cost or access or availability of radiologists, etc. Can use triage and quantitative outpoints where specialist access is limited. That's a quick overview. When there's a choice between good but not as good and nothing, surely the not as good still beats something else. But we make trade-offs every day in our lives, and in and it definitely happens from a healthcare perspective.

SPEAKER_01 28:06

I agree with you, but I'll expand it just a little bit more. It's better to have a radiologist, which is 80% correct of the time, than no radiologists at all. But you've limited to just developing countries. I think it's happened in developed countries as well. There are public hospitals in Australia which have extended unreported lists. One hospital in Sydney has 50,000 unreported cases. Metropolitan Sydney. This is even in rural remote. 50,000 unreported cases. People have been waiting a year for a diagnosis. There's been definitive outcomes for this. So I agree. Anything that speeds up a radiologist, anything that helps us get through the list at this moment in time is a godsend. People are going to say it's not as good.

SPEAKER_02 28:54

Why can't I get the better?

SPEAKER_01 28:56

Well, unfortunately, there's uh limitations to resources. I'll be complaining why can't I have a very fancy car? Well, cheat, you just don't have enough money. There are limits to what we can fund.

SPEAKER_02 29:13

Yeah, I'm on the same perspective with you with the car, by the way. Yeah, that that's it exactly. If this helps people read more scans, or it's quicker to make radiologists more available, or if it is able to be deployed to places which couldn't otherwise get it, I don't think there is a gap in need. The demand is definitely there. If it extends the capability of radiology and the speed, I think it's a great thing for society. And it's not something you should really be feared by the radiology community because the the need I've seen is I've seen is enormous and health budgets aren't as enormous. So as a society, we need to make decisions about our priorities, and if AI allows radiology to be used in more places and get to more people and save more lives, even if it's not perfect, I can't see how we can say no. Totally agree with you there, Stephen. So Chi, sounds like we're in violent agreement about a number of those positives, about those promises for AI. Okay, Chi, now we're on to the realities of AI. Where does it not fit so well? Where does it not work? What are the negative arguments against it? First one is generalization failures and data set shift. This is about using AI in different situations based on the content that it was developed on. It's about the data set that we were that was built on. Is that going to work in every situation? Is it going to work on a different scanner according to a different protocol? If AI is reading an exam, it is working from a model built from other exams, normally hundreds of thousands, right? But those exams may not have covered every environmental scenario. And the result may be lower performance. 100% right. But we just need to be realistic about where it's used. It's not perfect yet, and it probably will never be perfect. We need to use the appropriate tool for the appropriate scenario everywhere. It's like an asterisk, noting that we should do our checks on the specific environment first. What do you think, Chi?

SPEAKER_01 31:23

I think you're onto something there, Stephen. Look, I think AI is limited by the training set. If the training set does not cover a particular scenario, then the ability of AI to interpret these findings will also be similarly limited. In the embedded imaging, we are coming up with new protocols, in particular in MRI, all the time. But even within CT, we've changed from uh arterial chest to venous abdomen to just a venous chest abdopelvis. Now that's a difference in timing of contrast, which also means you get different images as well. And in MRI, as mentioned, new protocols keep coming through almost every few months. And we incorporate those into our scans.

SPEAKER_02 32:08

In Lehman's terms, quickly, Chi, if we built a model based on one type of scan, if the protocol changes and the scan is done differently, we have to be realistic about the fact that that might not pick up to the same level of quality and the same results as it would have before the protocol changed.

SPEAKER_01 32:28

Absolutely correct there, Stephen. I think it's kind of analogous to a Formula One driver, they have been trained to race fast around the track. But I know several Formula One drivers, because I keenly read up on it, that though they might be great around the track, some of them find it hard to park a car. So horses for courses, just because you're very quick in a F1 car doesn't mean that that translates into a commuter car in the shopping centre.

SPEAKER_02 33:03

Yeah, exactly. And probably all of us have examples of that in our lives where we expect somebody who's good at one thing to be great at another, and uh it doesn't happen.

SPEAKER_01 33:12

You mentioned something about uh data sets, uh hundreds of thousands cases as a data set. There's about 10,000 known diseases which are one in a million or even more rare than that. And this might not be captured in that data set, which means that there's a deficiency there. So humans are very good at moving from experimental to clinical practice easily by reading an article. But for the moment, AI requires an imaging data set to train.

SPEAKER_02 33:44

The the imaging that we're doing is machine learning rather than creative AI. And you're right, that requires a data set. It just can't imagine what a disease would look like based on reading it, whereas a human can. That's a big tick for humans to stay involved, but it's also, I think, a big tick for AI to be used in appropriate scenarios. Question number two is really not about AI's usage in how it was trained, it's about AI's usage in different demographic groups. So, what about the gaps between demographic groups and uh how do we handle that?

SPEAKER_01 34:22

Although we are all humans, we live in different parts of the world, and different parts of the world have different diseases. Some are pandemic. We had COVID, which spread across the whole world, but there are other diseases which are more localized. I remember when I was studying for medicine, I was reading textbooks and they kept talking about differentials between TB, histoplasmosis, and stuff like that. And I was like, oh yes, histoplasmosis kept coming up over and over again as differentials for various diseases. Now, little did I know, after a little while of a bit more reading, histoplasmosis does not occur in Australia. We just don't have that bug. It's only in America, and all the textbooks I was reading, because a lot of textbooks are from America, were mentioning it. So therefore, you have to be cognizant of the differences still, and you have to apply your clinical knowledge to what the AI is representing. So, for example, there's some diseases which have cellular imaging, so TB, fungal infection, and sarcloidosis can have identical imaging features. If you were in India, we would provide a diagnosis of TB. In Australia, the diagnosis will be psychodoses. So depending on the training set and where it occurred, the answer provided by AI could be role.

SPEAKER_02 35:47

Yeah. And I think that's the reality of it. It's it's not that complicated. I know, for instance, in in different countries you get a different BMD score from a DEXA and it's population dependent. So let's just be realistic about that. It's not a showstopper for AI, it's just a small hurdle that we need to be realistic about to build a different models for different cohorts. Okay, the next one is reality number three. This is about false positives and downstream harm. So a small change in the way that a scan is reviewed could have unintended consequences downstream. For instance, the explanation here is that in low prevalence screen settings, small drops in specificity can create many false positives, increase follow-ups, costs, and anxiety. If we're changing something in a system, let's be realistic about what else that is going to change. AI might be faster, but if it creates other issues in the medical care process, it might not be worth it. So an example is false positives causing unnecessary extra work or even an extra exam or surgery for a patient. If we've got it wrong and somebody goes under a knife because of AI getting it wrong, that's a problem. What do you think, Chi? How big is this?

SPEAKER_01 37:02

Look, I think I think there is issues there about sprain diseases which are only present in small populations in particular. So if you're spraying disease and it only affects, say, 2% of the population, that does cause unnecessary work. It does add a lot to cost and it adds a lot of anxiety. Because to find that 2%, you might have 4% of the population unduly stressed by an abnormal result. So as an example, screening for TB, which is rare in Australia, would be costly and produce a lot of false negatives compared to screening in India. Conversely, screening for melanoma in Australia where it is of high incidence is useful. But in India, where the prevalence is low, it just doesn't make sense. So I think judicious use of AI is key, which will be guided by doctors and the government.

SPEAKER_02 38:00

Okay. I understand the judicious part. This question is specifically about how big is the and what are the on-flow effects of getting it wrong? If the results that come through from the way a report is written, or what is found on a report, and how that report gets communicated, I I'm not sure. Like any of the impact on what happens due to AI, have you seen how that's affected things downstream? Is this a problem?

SPEAKER_01 38:35

I think what you're getting at is if we're able to screen for all diseases or screen for more diseases, why don't we? And it is that kind of false positive. If if if the test isn't good enough, it does cause undue stress for patients, it does lead to more follow-ups, which adds more costs. There's a lot of diseases that we can screen for at the moment with just general medicine and not with AI. The costs, however, are prohibitive for us to do that. We all know that screening early saves money. And hopefully, maybe AI can unlock this screening pathway to allow us to screen a lot more at low cost, obviously, to prevent diseases earlier on. We just need to be careful that the false positives of this screening test that we use with AI are not too high.

SPEAKER_02 39:32

There's really two parts to this question though. One is about that false positives, the other one is about the mechanism and the way if AI affects one step in the process, it might affect other steps in the process just because it's not exactly the same and done exactly the same as other things. So if a report comes through and is written differently or is communicated differently, or happen in a different way, have you seen any on-flow effects of or downstream effects of that upsetting a step further down? Somebody getting extra scan, somebody taking longer, the report not being read correctly, another system in the chain being unable to handle the information that's provided.

SPEAKER_01 40:18

I think AI is still in its infancy for screening that we haven't actually delved too deeply into the downchain effects yet, and extra burden workloads for that.

SPEAKER_02 40:31

Okay. I'm not just talking about screening, I'm talking about all AI utilization, all the other things we've spoken about today.

SPEAKER_01 40:37

The closest example we have is every time we introduce a new screening program. So, for example, last year we introduced the lung cancer screening program for Australia. Now that has resulted in increased burden for radiologists. We all have to report them that we never used to do before. Increased burden on the GPs, we have to follow up the results, increase anxiety for patients with false positives and uncertain findings, so they have to come back for another year to have another CT or even shorter time frame. But overall, the reason why the lung cancer screening was introduced was because the government did the figures and worked out finding the disease earlier by screening for lung cancer will save us a lot of money long term. When by picking up early, we can treat it early, and that's a lot cheaper to treat than treating it late.

SPEAKER_02 41:28

Okay. Look, you brought up a couple of good examples there in terms of the impact of additional work from adding a screening program. Let's move on to reality for this one's about automation bias. This is when you trust the AI too much. So clinicians can overweight AI outputs, especially under fatigue, leading to errors of omission or commission. I can see this as a real issue because sometimes when you have a technology, you say, oh well, that said that. That must be right. That's a computer. That's a system that doesn't break. That's not under strain like a human. You know, that's not can't be coerced like a human. It just must be right. The computer says this. And I think we're, if we're honest to ourselves, that has happened probably to all of us, that we've just deferred it to the technology. What do you think?

SPEAKER_01 42:22

I think when humans are stressed or burdened with work or under fatigue, like you said, we do take shortcuts. I mean, you and I probably know that if I'm tired, do I break out the pan and start cooking something? No, I hit the microwave. So we we all take shortcuts if it's there. Like I I totally agree with that statement. I think the main concern with AI is overcalling something rather than undercalling something. As a radiologist very false positives rather than false positives rather than false negatives. Yeah. I think as a radiologist in Australia, our Australian listeners will understand this. Ultrasounds are performed by Sonographer who who then write a worksheet for the radiologists to interpret along with the images, and then for the radiologists to write their final report. Now, if the synographer has mentioned something and you disagree with Sonographer and you think it's an overcall, you still feel pressured to make a mention of it and maybe even agree with it, because everyone wants to perform medical legal defensive medicine. The problem comes with if they think it's a mass and you think it's not a mass, and you go, look, it's not a mass, but it turns out to be a mass. There is that worksheet there that is documented explaining what what the sonographer thinks. So that's why it's easy and safer to just Disagree with the sonographer than it is to make a different call. So why am I mentioning this? It's because it's similar to AI. If AI replace the sonographer with AI, if AI finds something, we're under fatigue, and we although we feel like we disagree with AI, if it's caught something, it's found something, you might push yourself towards agreeing with it. And that has its own implications of over-investigation. But if you don't agree with it, if it comes out in the courts, they'll bring up the AI report and ask you, sir, madam, explain yourself.

SPEAKER_02 44:43

Yeah, absolutely. Okay, it sounds like we're in pretty strong agreement here that it is an issue, Chi, this automation bias, but is it a showstopper?

SPEAKER_01 44:53

I don't think it's a showstopper as long as we understand the pitfalls and are aware how to deal with them.

SPEAKER_02 45:02

Okay. Good answer. Oh, we're agreed. Reality number five, this is about generative AI hallucinations and safety. Now, the explainer for this is generative AI may improve reporting efficiency, but can hallucinate incorrect content, creating new patient safety risks. In your experience, is this general worry about hallucinations in the general AI area, is that creating a shadow across AI usage in and radiology? That people know the difference between these two types of AI and is it going to create an issue?

SPEAKER_01 45:36

I think generative AI with these large language models used to help us create reports is still very in its infancy. I don't think it's a deal breaker per se. Some groups are using these large language models and they're seeing improvements in about 30% in efficiency. That's a huge amounts of improvement. And so I think the hallucinations will get better with time, similar to Gemini's Imagine, ChatGPT, these these are improving with time. So I don't think they're deal breakers per se. With 30% efficiency, I think you can repurpose that time to check your results more carefully to make sure that the output is safe. And I believe with the percent increasing efficiency, meaning we can do more with our time, it's better to get an output rather than no output at all. In some ways, large language models actually improve the report. So quite often, if you ever read a radiology report, there's quite often grammatical spelling errors and incorrect transcriptions. So left with right, no's no or yes being removed, and these can result in significant changes to a report. So AI is a lot more accurate in that sense. So you get some hallucinations, but you also get some benefits.

SPEAKER_02 47:02

So at the end of the day, I think not a deal breaker, it's deemed the one thing where I think there is a real point here is not in the reality, but in the perception. Because we know AI hallucinates, LLMs hallucinate, and that they can be creative, which is great about them. People think AI is is like ChatGPT, and every part of AI can hallucinate, whereas it can't. Some you know, machine learnings just do what they're told and provide past pattern recognition. So I think there is an issue there in perception, and it's one that will provide a shadow over use of AI in society in general and in radiology, and we need to be wary of it. This one is reality number six. It's about the economic value and the ROI. It's plausible, but it's context dependent. So the overview of this is AI can be cost-effective in some workflows, but financial impact depends on task, performance, specificity, and payment model. So, what does that mean? It means AI needs to save you money or help you earn more revenue, and it doesn't work in every situation. Again, it's pretty similar to some of the answers we've given before. This is an asterisk rather than a wall. This is not a showstopper. It's just a reminder to move forward cautiously with our eyes open and learning and not going gun-ho. What do you think?

SPEAKER_01 48:36

I agree with you. I think um there are many different use cases in radiology at this moment. Uh, some of them cost more money than they generate. Yeah. And that's a hard sell. So chest x-ray readers out there, they're supposed to reduce litigation because they give you a second read and they're supposed to speed you up. But doing analysis of them, the costs of the chest x-ray reader are higher than the litigation cost. So, although it reduces your litigation costs, the amount you spend on it is more than what it would have cost to pay out.

SPEAKER_02 49:12

I would imagine there's a tipping point for every technology, and AI will get to the point where it becomes so much more effective that you can't ignore it.

SPEAKER_01 49:20

I think the tipping point will come at some stage, but you also have to be judicious about what you have or or what you what you um spend your money on. We mentioned triage tools. Now they cost money. They're going to give you a triage tool for free. But at the end of the day, all those cases have to be reported anyway. It just reorders which one you report first. So it doesn't actually make you any faster. Look, some will argue that you will report faster when you're hitting the end of the list. You know that there are not this unlikely going to be so it's unlikely to have any anomalies, and therefore you report those faster. But that's confirmation bias, and that results in errors because you'll be like, oh, the AI says it's normal, I'm just going to report this as normal. And that's a negative in itself. Look, there are some specific use cases. I do see the large language models as increasing efficiency in reporting. There's other use cases such as screening tools which improve outcomes for patients and do generate income. And this goes with all screening programs. Look at the lung cancer screen program, it reduces morbidity and mortality for the patients because they find lung cancer earlier. It means more scans for the radiologists so they get extra income. And it's it's also actually a saving for the government because by treating it early, it saves the money from the expensive treatments from a late diagnosis. So there are specific use cases where AI can satisfy both your economic value and return on investment plus also help patient outcomes.

SPEAKER_02 51:00

Yeah, I agree. I think there's a lot of cases, to be honest, but the key one that we've found throughout this whole podcast is that it's not every case. We need to be, like you said, judicious about where that is. Now AI is useful here, but not useful here yet. And maybe at some stage in the future it is, but for now we use it in the place and in the scenario that it makes sense. The next one is reality seven. It's our last one: governance, transparency, and regulatory safety concerns. The one liner for this is safety and performance transparency gaps, plus recall adverse event activity, reinforce the need for strong governance and post-market monitoring. This is probably the biggest inhibitor of AI adoption. And it comes down to trust. We don't yet have enough understanding of the gaps, and we don't know the issues that might come up. Humans are risk-adverse, and understandably, even more so when it's up to them or their family's health. So governance, transparency, and regulatory controls aren't about crushing, it's about creating a framework that gives people trust to go back continuously. For instance, if I thought one in a hundred sandwiches I bought from a shop was going to make me sick, I wouldn't shop there. I'd shop somewhere else. But it's only one in a hundred. And I need to know it's not gonna happen, which is why we have health regulations around the preparation of sandwiches, right? We make sure that we're pretty darn certain that every sandwich I get from wherever it comes from is going to be a good sandwich. This is the same. We all hear about one issue with AI, and we think, oh, but that could be me. Um, and this is where governance and governments and regulation and transparency come in to give trust. That's my take on it. What do you think, Chi?

SPEAKER_01 52:56

I agree with you. We have to learn to trust technology. Like one of my analogies for this is reverse cameras in cars. Yeah. So previously we all relied on turning our heads, yeah, using the mirrors, reversing it up.

SPEAKER_02 53:12

Reverse camera, by the way, I was thinking about the other day. I think it's probably the one thing I use, and I'm like, oh my god, this is great. I can just reverse straight back and it's I think they're pretty.

SPEAKER_01 53:23

Let me ask you then, Stephen, when you first got them, did you still use your mirrors for a little white? Do you still turn your head?

SPEAKER_02 53:29

Yes. Yes. Yes. I still turn my head. I've only recently, I mean, it's been around for years, right? But I've only recently got to the point where I'm just now just I don't even look back. I just use the the camera. That's a really good point I didn't think about. I trust it completely. I'm I trust it implicitly now, and I just use it all time. But it took me a long time.

SPEAKER_01 53:48

And look, after a while, the majority of us do trust the camera. Look, some still don't. And it's all about trust, I think, in the technology. And I think it's too hard for us radiologists. We're just too snowed under with work. We don't have time. We don't understand the nuances of AI, we don't understand the nuances of the studies, that it's too hard for us to research every product to make sure that we trust it. I think in some ways we have to leave that to the proper authorities, the TGA, the FDA, the CE, whichever site and regulatory body you're at. I think we have to leave it to those experts to do that background safety, to do the research, and then slowly trust out trust it with time.

SPEAKER_02 54:36

Yep, it's about trust and time. But can we speed it up?

SPEAKER_01 54:40

I mean, you just said it's about trust and time, but is it's what can we do to make it team is actually move towards things which are convenient. I think AI would hold convenience and it's convincing them that that convenience does not come at a cost of safety. It's similar to microwaves. We all move to microwaves, and after so many years of research that microwaves are safe, we all now use a microwave probably more than what we probably should, but we we all move and gravitate towards that safety realm.

SPEAKER_02 55:13

I was just thinking about I was recently in Mexico City and I saw the food vendor on the side of the street, and I was like, my my safety antenna went up and I said, Am I sure about this? Is this a good idea? But I was like, it's just so damn convenient. I've got three minutes, I'm gonna try that taco. And I can tell you I won that. The taste and convenience won that time, but I well, I did have my fingers crossed. I wasn't sure I was gonna get out of that. Great story, Jamie. Thanks for sharing with us. Agreed, Chi, and thank you for your time. And thank you for listening. If you have any other questions for Chi or for me, please put it in the comment section. Please join us again for another episode of MIAI, covering all your AI technology for radiology topics. And as always, subscribe below and join the conversation. Thank you.