AI in Healthcare Isn’t What You Think

Hey everyone, quick message before we get into the episode. We love our listeners and would be thrilled to hear what you think of the show. It really helps us behind the scenes so leave us a review on our podcasts or Spotify and tell your friends to tune in. If you've got a science question or a topic that you want us to cover, just feel free to let us know. Thank you. We cover artificial intelligence from a lot of different angles on this show. And can you blame us? In a few short years, the tech has already upended a ton of industries. From customer service chatbots and e-commerce to fraud reporting and banking to social media content creation, AI is everywhere. Sometimes for better, sometimes for worse. One thing that most people can agree on is AI has the potential to improve elements of our healthcare industry. Here to provide some clarity on what that looks like is Dr. Nina Kotler. Before I chat with Dr. Kotler, we'll learn about how Koala populations in Australia are upending some common theories on population genetics. And later, we'll get into a study that explores how other people's opinions shape our own experiences. My name is Dr. Samantha Yameen and this is Curiosity Weekly. Let's dive in. There's a common understanding in population genetics. If you are the number of animals in a closed group, the less genetic variation in future generations. It's basic biology. Fewer parents can lead to inbreeding and more genetic overlap between kids. Back in lead to infertility and genetic damage down the family tree. So you can imagine the surprise of biologists when they recently discovered the opposite to be true for a group of Koalas living in the Australian state of Victoria. A population of Koalas in Victoria shrunk down in the last 100 years due to hunting, habitat loss, and disease. Their small population size and genetic pool put them on the brink of extinction. Starting in the 1890s, a small group of Koalas were moved to two Victorian islands. Their descendants helped repopulate the mainland and by 2020, the total population across Australia was between 300 and 500,000 Koalas, though how this succeeded was unclear. These studies showed that they were less diverse as a result, but when this team re-examined their genomes, they found signs of unexpected genetic recovery and diversity surprising the researchers. So what gives? If there were only a few Koalas to repopulate, where are they offspring getting all this genetic diversity from? Well, it turns out that it's a combination of things. The research team looked into the genomes of over 400 Koalas from 27 different groups all over the country. They honed in on trying to measure and understand essentially their genetic headcount, as opposed to just population size. They wanted to know the number of individuals in a population that actually contribute genes to the next generation in ways that impact diversity. This is called effective population size. Think of it this way. In a boardroom of 30 executives, but only six doing all the talking and deciding everything, the board is essentially behaving like a six-person group. That's the effective size. And there were gaps in the previous ways we've been measuring genetic diversity in Koalas. They weren't looking for the rare alleles in the Koala genome that paint a much more nuanced picture of genetic health. Alleles are variants of specific genes, by the way, like eye color, hair color, etc. Now second, the researchers discovered that when populations grow rapidly, you get more shuffling of genes in each generation, deluding the frequency of harmful mutations. For time, that continued mating diversifies the gene pool. It's an exciting discovery, not only for the cute little Aussie Koalas, but for a lot of other vulnerable species that face potential extinction. It shows that even when numbers are low, there's still an opportunity to bounce back and thrive. But those gains are not guaranteed. Another crash could erase them, so ongoing monitoring and support are still needed. As healthcare systems continue to be overburdened, it's not surprising people are optimistic about how AI might be able to help. In fact, 80% of physicians reported that they use AI in a professional context according to a survey published in March 2026 by the American Medical Association. That's up more than double from 2023. But when people's lives are at stake, of course we have to be super careful about accuracy and maintaining trust between patients and providers. It was a really careful walk along the line between hype and hope. To help us understand where things are headed, we're chatting with Dr. Nina Kotler, a radiologist and chief medical AI officer from Wazeak Clinical Technologies. She's also an associate fellow at the Stanford Center for Artificial Intelligence in Medicine and Imaging. Welcome to the show, Nina. Thanks Sam. Great to be here. I'm wondering if you can start us off with the broad categories of AI use in healthcare. And what kind of problems they're trying to address in UAs? At a high level, AI in healthcare is using data and in radiology that data is medical images, like CT scans and X-rays and ultrasounds, takes them also reports and patient information to help clinicians make better decisions. That's just very high level. In radiology, I don't know how many people in the audience know specifically what a radiologist does because I think there's a lot of confusion. And sometimes I ask people, "Hey, what do you think a radiologist does?" And they tell me, "Oh, you're the one that takes the images when I go to get my hand X-rayed." And that's not actually the case. That's the rad attack. A radiologist is the doctor that interprets those images. And in general, what we tend to do in the reason why I went into radiology is because we're called the clinicians clinicians. We're the doctors doctor. And that needs, we provide a consultation to the other physicians to help them understand what's going on with their patients. And to do that, we need information. And that information, generally for us, at least to start is a lot of the information in the medical images. And as we start applying AI, we're applying AI to that data to start to understand more from those images in combination with the patient history and all of the other data that is available about the patient within all of our electronic medical records. Radiology is one of the earliest spaces, actually, the earliest space that AI came out in healthcare. And that was back in 2016. And that's because radiology is so very digital, meaning all of our information is not in those physical paper images anymore. Right? They used to be printed out images. And they're all digital, so we can send them to different places and that enables us to use the AI a lot more easily. And in that American Medical Association survey from this year, 2026, there was a line saying that the physicians were expecting the greatest benefits of AI and healthcare to be related to diagnostic ability and also work efficiency. And then there's like hospital admin and all the different healthcare worker workflows. Is that kind of all the different areas where we would probably see AI touching? Yeah, I think we started off, it's evolved. We started with AI in healthcare in 2016. And back then, it was mostly diagnostics. It was a very narrow system that would help tell you if one thing was present or absent in a whole imaging exam. And now we're moving to another phase where we're trying to move more toward workload. Work flow is a little different. There's so many things that we do that are not diagnostics. And that part is actually way harder. So when we concentrate on those workflow items, we're actually improving the efficiency from which we can get things done. And right now, the biggest problem in healthcare, number one problem, is actually not the quality. The number one problem is having enough physicians to manage all of the people and the exams that are needed to be interpreted. Because if you can't get to that, it doesn't matter how good your quality is. So right now, workflow is the biggest thing that can enable additional capacity in the system. But the deal end all like the benefit in the end is figuring out what new opportunities are there that we can identify with AI. So instead of identifying disease after it's happened, how can we predict disease before it occurs? How can we, once we do identify disease, how can we determine what the best treatment and exactly what that disease is and how it's going to react to the different treatments we can give? How can we provide this information to physicians and to patients so that they could actually see what their body is doing? And as they make changes in their lifestyle, how that's actually affecting them, those are future things that we need to move to. We just have to get through this capacity piece in order to manage all of the patient care that we need. But it's not the endpoint. The endpoint is the improvement in healthcare. And we have going for the most urgent thing, the most immediate need and the lowest hanging fruit is solving the capacity issue that we get into all the other more innovative stuff once we've caught up. Yeah, I don't know if people realize just how far behind we are with our capacity. This is physician capacity and I'll give you the numbers for radiology since I know that. But it is not limited to radiology. We are probably about 15% understaffed in radiology in terms of the imaging volume compared to the number of radiologists that there are. And I was in London a couple of weeks ago, there are 30% double what we are. And the reason behind that is every year in the US, we are ordering about 4% more imaging exams every single year. Now, each of them exam is also getting more complex. Like, there could be, and I don't know if people realize this, there could be a thousand images in a single imaging exam that we're interpreting. So, about 4% more year, and the number of radiologists that is increasing, it's 0.4%. Now, those numbers are both really small, so they don't sound like they have a big impact, but there are 10x difference. So, if every year there's a 10x difference in the amount of imaging, where's the amount of people you have to manage that imaging? Like, just imagine how big that gap is. And in 2022, that gap crossed the threshold, but we can't manage it anymore. So, every year since 2022, turnaround times have been increasing, which means how long you have to wait to get your images before you're seeing how long you sit in the ER. All of these things are increasing, and this is not just in radiology, this isn't every specialty. And I'm based in Canada, where capacity issues are huge. So, and wait times are really, really long. So, I can only imagine what our stats are. What's interesting as we're speaking about capacity, though, is that I'm reminded of back in 2016, I remember Noble Prize-Lorriot and Godfather of AI, as he's often called Jeffrey Hinton, said, "People should stop trading radiologists now, and it is just completely obvious that within five years, deep learning will do better than radiologists." That was a direct quote. So, I'm curious, that doesn't seem to be the case today, but where has it fallen short, and why didn't that pan out? - Yeah, super interesting. Actually, that prognosis from him in 2015, actually decreased the number of radiologists that have gone into radiology, so that's worse than the problem. And we are at the opposite end. I've been in radiology for 20 years. I have never seen this big of a deficit ever. And so, yes, this is a massive problem. Why did he guess that and where did he go wrong? I think people don't realize the kind of AI that you all might be using in your day to day, maybe you're using Gemini or Chat GPT or Clod or GROC. Any of these tools, these tools are called foundation models, meaning they're very general. They don't just answer one question. They answer any question that you could have, and they could do it with all different kinds of mechanisms. They could look at images. You could put in a copied image, and they could interpret that. You could speak to it, which speaking now generally is writing, so it can do language and images. They call that multimodal. What are we using in radiology? There was actually a really funny skit that if anyone has seen the TV show Silicon Valley, there's a Silicon Valley show. It was an episode from years ago where one of the guys in Silicon Valley created an AI tool, and the episode is hysterical. They talk about this tool. How you can identify what's in the image. Now, the first thing they do is they take a picture of the hot dog, and they run it against the AI, and it goes hot dog. And they're like, oh my god, this is amazing. It's going to change the world. Now remember, this was years and years ago. Change the world. This is fantastic. Then they say, do another one, do another one. So they take a picture of a pizza, and they're all like, okay, what's going to happen? And it says, not hot dog. And that's actually where we are in radiology AI right now. It's very easy to just assume that we're as far ahead as where everyone else is using it today, but there's regulation, there's systems that don't really connect together. It's so much more difficult to deploy that kind of thing in the healthcare system. So right now, we're at what we call narrow AI. It's a hot dog, not hot dog model, but instead of identifying hot dog, it identifies things like blood in the brain, or a hole in the lung, or cancer. And those are all really important, but they're not how I work as a radiologist or how any other human works. When I get a result from an AI that says, there's a hot dog, or there's some blood in the brain. Great, I'm looking at maybe 500 images on the head CT. I want to describe where the blood is. If it's new or not, is it getting worse than it was before? What associated findings are there? And then I look for 500 other things on that study. Not just the one. So it's just very different from the experience that you might be experiencing in using AI in your day-to-day life. Where do you see the biggest near-win for AI? How would it make your day in that part of your workflow faster or more efficient? Yes, really important. Radiologists or any physician spends the least amount of time picking up things in the image. Why? It's the fastest thing we do. It's part of our syphilolote back here in the back of our brain. And that has evolved over millions of years. We are actually really good as humans at picking up things. So when you have someone else that's looking, it helps improve the sensitivity of what we have. We pick up more stuff. But does it help us do it faster? Not really. We spend way more of our time doing other things in the workflow rather than picking up the pathology. So when we talk about workflow, like what does that actually mean? Well, it's doing the things that are low value for us as radiologists. Super high value for us to be looking at the images, because that's where the patient is. Super low value for us to be editing the report and looking at the report especially. If you're going back and forth, because humans are not good at changing our focus. If I can quickly ask you almost the opposite question, is there something that you don't think that AI will be able to help with in the near future? Of course, we can't predict a long term that maybe would surprise people. So yes, I can tell you where we're starting to use AI in a way that it acts like a second set of eyes that it can do. It's very good at that. What it doesn't do is take accountability for the patient. I mean, ultimately, we as physicians take accountability for every single patient that we see. AI, we tend to anthropomorphize it to say it to acting like a human. We can only imagine what a human does. It's copying some of the things a human does. So we think it's being like a human. It doesn't have the same level of accountability. That's one thing. The second piece is a lot of things in health care these days are uncommon. When you look over all at a whole population, how many people have blood in their brain? Well, most of what I see, even for the patients that come into the hospital so they're sick and we expect they might have something. It's about 5% at the time. So a lot of things in health care are uncommon. The less common something gets, the more times the AI will provide a result that's incorrect. And it's just the math behind it. It's related to what we call disease prevalence, which is how common something is. And what we call a metric of the AI, which is positive predictive value. When it tells you there's a positive finding how often is it right? And actually, that gets really low when things are uncommon. So it's very hard for that long tail of findings for the AI to provide a right result. You need a second person and a human to look at that to get rid of all of the times it's wrong, all of the false positives. So AI produces a lot of incorrect results. The physician with the AI together that make the better combination. I'm really interested in the evaluation side of AI. And I know you recently published about a new approach to evaluate AI models and their potential value in radiology. So I'm wondering if you can tell us about it and some takeaways on how to think critically about the positive impact of AI here. Yeah, I think a lot of people just immediately think if you look at the AI accuracy, especially what the vendor might give or what you see in the FDA, because anything that involves an image, medical imaging, has to go through the FDA before it can be marketed or sold. And with that, the FDA will provide statistics that help determine if something is safe. And we've looked at those statistics a while ago, both with the FDA provided and what the vendor provided. And it tells you things about how sensitive it is to finding disease, how specific it is to not over-calling disease, and a few other things. But as a clinician, as the end user, not a single one of those statistics is what I feel as an end user. And the biggest question to me, because these tools are not working on their own, because they're working with a physician, the biggest question is, how is the physician going to feel about using these tools? You could have the best AI tool in the world, the most accurate tool in the world. But if it is put into a workflow where the clinician is not going to be able to use it easily, it doesn't matter how good that tool is. And all we're seeing right now, especially from the vendor, is like, look how good my tool is. What we need to do is look how well it's going to be accepted and utilized. And I have an example, because before we roll anything out, we spend a lot of time educating our physicians, giving them expectations about how AI works, how this AI works, what to think, how to work with it optimally. And we do that in the beginning. We were doing this literally physician-by-physician on-site. And we went to this one hospital. We said, well, hospital, do you want to look at the education that we're giving to our clinicians? So you could see what we're doing to make sure that people are using the AI appropriately and really understand it. And they're like, oh, great, yeah, let me look. So after that session, they came to us and they said, you know, we bought three other AI systems that no one's using. Can you just go teach them how to use that too? And that tells the story. It's really true. You can't just rely on the AI accuracy, especially accuracy metrics that don't feel useful to us as clinicians. You have to think about what does it feel like for the end user? how could you provide them information and even predict an advance, whether or not they're going to be able to use it. So the kinds of things we look at, we don't just look at sensitivity or specificity, we look at things like how much more effective will the physician be if we add this AI system? AI is often framed as a way to democratize access, bringing specialist level reads to rule or underserved hospitals, help things where capacity might be most dire, maybe in under-resourced areas where they have even more capacity areas. But does that play out when you think of the costs for some of these technologies, the set-up and expertise required, especially thinking about more remote areas, where it might be harder to do these trainings, for example? Yeah, in general, as technology improves in most industries, as technology improves, cost comes down, as cost comes down, you could use it for more places, and that allows us to take, in this case, take care of more patients. There's some things that are a little bit different right now because we're so early on, and there is a cost associated with these tools. There's no hospital that I know of and no radiology practice. It's just like bringing in tons and tons of money that they have all of this money that they can spend on new tools. That would be great. But that just doesn't exist today. In fact, I know a lot of radiology practices are going out of business because reimbursement is coming down and the cost of care is going up. So it's just a difficult environment to be in. So right now, if you have to add on an additional payment, like how do you do that? There's some groups that are saying, well, we're just going to invest in that and believe that we could recoup that dollar somehow that it's going to improve patient care enough that there will be less costs in the future because patients won't have to come back. They'll get the right treatment the first time. I wish we could say that everything was just about quality of care we're giving. But as hospitals are losing money or after a year, they also have to think about the amount of dollars that they're putting into the system. So it's a balance. Unfortunately, the government is not paying for these tools. The government pays for a lot of our health care. There's very little payment for AI. So the question is, how do we make this right? Because ultimately, we can't just pay endless amounts to expect to improve outcomes. We have to be able to do improve outcomes without increasing the cost of care. And I think ultimately, that means about making the cost of care just more effective. And that's where we're trying to move to. It's slower because it's really hard to get these technologies into care. And most of the technology that we have today is that hot dog, not hot dog, which really doesn't make a big difference. But as we start moving to the foundation models that everyone's using today, those are much more valuable and capable. And I think they'll enable us to prove that we could pay for these tools and they will bring about a return on investment. And I'm sorry that as a physician, it pains me to talk about return on investment. We're talking about health care because what we care about is patient care. But ultimately, we need to balance both. Last thing I wanted to ask, AI can only be as good as the quality of data it's trained on. Are there any significant gaps in the training data at the moment, especially given how rare and uncommon you've mentioned a lot of diseases are? Yes. So this is difference between these narrow AI systems, hot dog, not hot dog, and the foundation models, like the Cheshire PT that you're using today at home, the narrow AI models are trained to find one thing and they tend to be trained on less data. The benefit of a foundation model is you just give it a lot of data. Give it a lot of data and you're not actually circling where things are telling it the what you're doing is you're training it by having it help self-learn, kind of learn some of this things on its own. So in radiology, we give the images and we give the radiology report. And then the AI learns on its own, but because we're giving way more data, these tools end up being more what we call generalizable. They work on a larger patient population. But the state of the art today, which is near AI models, you're exactly right. They're trained on a smaller data set. So if you have a piece of data that looks a little bit different and it's out of distribution from the data, it was trained on it won't work as well. I mean, know that that is the case. I think that as we start moving over to foundation models, that's going to be less of a problem because they're trained on just so much more data. That's fascinating. Thank you Nina so much for being on our show and teaching us all about what we can expect in the the the hype, the warranted hype for AI and healthcare. I am very excited where we're going into healthcare. It's one of the only times where technology can have a massive impact. That's Dr. Nina Kotler, a radiologist and chief medical AI officer for Mosaic Clinical Technologies. Be honest. How easily are you influenced? Now there may be a thing or two. I've added to my shopping cart this past week after seeing them online. And I always check reviews before booking a hotel or choosing a restaurant. The researchers at Dartmouth wanted to test just how much other people's opinions can sway us. They found that social cues can shift how we experience pain and how much mental effort we think something will take. Here's how they did it. 111 healthy adults completed three tasks. There was a task where heat was applied to their forearm and they'd have to rate how much it hurt. Another task where they looked at videos of other people with painful grimaces. And the third task tested how much mental effort it took to rotate shapes in their head. Before each trial participants saw a graph showing how 10 previous people had rated the upcoming task. But the data on the graph was fake. These were just random ratings, not from other people and not actually related to the upcoming task. In this experiment, the graphs were the so-called social cues. Participants saw these social cues then gave their own ratings. In all three tasks, the graph, which served as the fake social cue, could strongly shift both expectations and reported experiences. This persisted over multiple days of testing and 72 trials. There was this carryover effect where however they perceived one trial would directly bias the expectation of the very next trial. If the last one hurt, people expected the next one would too, regardless of what the social cue said. They also saw evidence for confirmation bias. In this study, when an experience confirmed what the social cue had predicted, people readily adjusted how much pain or effort they anticipated feeling next time. But when reality contradicted the cue, they largely ignored it. And their next anticipation stayed anchored to what others had told them to feel rather than what they just actually felt. Some of the study participants were more influenced than others, but it was pretty consistent across domains, experiencing the pain, witnessing someone else's pain, or having to do a task that requires mental effort. This research was published in the journal PNAS. This confirms just how much social information can warp our perception in the moment and over time. Others' opinions can bias what we expect, our expectations color what we actually feel, and what we feel feeds forward into what we expect next. The author suggests this could help explain the placebo effect, the expected positive outcomes from a treatment. Understanding that could help set more practical treatment expectations in healthcare, but on a personal level, it's a pretty good reminder that just because someone else says something is hard or even painful, it doesn't mean your experience will be the same. Stay open to experiencing things differently. For Warner Bros. Discovery, Curiosity Weekly is produced by the team at Wheelhouse DNA. The senior producer and editorial correspondent is Toreesa Carey. Our producer is Kiaranoni. Our audio engineer is Nick Charisamy. And head of production for Wheelhouse DNA is Cassie Berman. And I'm Dr. Samantha Yuin. Thanks for listening.