AI and the Future of Global Health

0:04 Introducing Ben Brockman and CHAI's Global Health Work So welcome to our brand new episode of On the Map with Global Health, a podcast from the Huber Department of Global Health at the Rolling School of Public Health, Emory University. I'm your host, Monzo Patri, and I'm thrilled to have you join us as we explore the dynamic world of global health. 0:20 Our podcast will cover a diverse range of topics from non communicable diseases to antimicrobial resistance, KA development in global health and much more. Today we are joined by Ben Brockman, who's a Senior director and head of AI at the Clinton Health Access Initiative, also known as Chai, a global health nonprofit that partners with governments around the world to improve the delivery of healthcare. 0:46 Prior to joining Chai, Ben spent time in the private sector at CVS Health and Boston Consulting Group and nearly a decade at ID Insight, a research and analytics nonprofit where he founded the data Science team. 1:01 Ben holds an MPSID from Harvard Kennedy School and lives in the Greater Boston area with his family. Ben, welcome to the podcast. 1:13 Speaker 2 Thanks Manzar, really appreciate you having me. Looking forward to the discussion. 1:18 Speaker 1 So let's start off by discussing a little bit about Chai. Could you tell me about your current organization? Sure. 1:27 Speaker 2 Thanks for the opportunity. So Chai is a little over 20 years old. We were founded in 2002 by former President Clinton after he left the White House at the height of the HIV AIDS epidemic. 1:42 And the initial work that Chai did was working with with governments mainly in sub-Saharan Africa to broker partnerships with pharma companies to increase market access for antiretrovirals at a price point that was affordable to citizens badly in need of those medications at the at the time. 2:05 And over the last 20 plus years, we've really evolved into a a global health nonprofit that partners deeply with 35 plus countries where we have offices that with teams in those countries that support ministries of health in their day-to-day work of better delivering healthcare. 2:28 And we still maintain with this, this global market shaping perspective we call it, where we enable the flow of commodities from vaccines to medical equipment to antiretrovirals, just like the early days of Chai. 2:45 Through these deals we we broker between the private sector and the the public sector. 2:53 Speaker 1 Thank you. 2:53 The 10/20/70 Rule for AI Implementation So your career, Spence founding ID Insight, data science function, working on AI at CVS Health and BCG, and now overseeing AI strategy at Chai. As you've shifted from social sector to private sector AI and back to supporting government health systems, what feels most different about how you build and deploy solutions? 3:15 Speaker 2 That's a great question. I, I might actually start out with what's pretty similar across the these different experiences, which might be counterintuitive for folks who have not spent as much time in, in AI. And so one of the things I, I took away from my time at BCG as we had this framework we always talked about with deploying AI, we framed as 10/20/70, which was only around 10% of the, the time of doing AI work was actually about the algorithms. 3:47 And then around 20% was all the data systems and, and broader technology that those algorithms sat on top of. And then really the remaining 70% of the bulk of the work was all the human side of it. It was all the change management work, everything from figuring out what's the problem we need to to be solving and whether AI is applicable. 4:09 Thinking about the responsible AI dimensions, which I'm sure what we'll get into later, thinking about regulatory constraints, figuring out the human AI interface of how people will actually use these tools. And I think that's been pretty consistent across both the, the private sector and public sector that when we talk about how to roll out AI solutions, it's actually much broader than just the algorithms or even the technology itself, but there's this big human component. 4:41 And so shifting from that lens to like, what are the big differences? I I think compared to the, the private sector, the, the social sector, governments in, in general as well as the, the nonprofit sector, there tends to be a little bit less digital maturity. 5:00 You see data systems that are a bit less developed data might not be in sort of the, the latest cloud databases that are, are standard in the private sector. So there's a little bit of a difference in in baseline tech maturity, but I also think the types of problems and some of the constraints that the public sector and, and, and social sector more broadly operate under are a bit different and actually make the work a bit harder. 5:30 One of the constraints I think we we don't think about too much is for governments. A tech solution has to work for for everyone. Private sector firms have the luxury of saying here's our target market, we're just going to focus on serving these customers in English that have a high end smartphone. 5:50 Government doesn't have that luxury. They have to deliver public services and all the languages that are official languages, all of their websites have to be disability friendly from from day one and governments for everyone. So technology has to be built that way from day one, which just makes a bit harder to actually build and deploy these solutions, but even more worth it when when we do get it to work. 6:19 Five Key Areas for AI Transformation in Health So AI is advancing at an unprecedented pace. The From your vantage point leading AI strategy at Chai, which emerging applications do you think will have the greatest impact on global health over the next few years? 6:34 Speaker 2 Yeah, it's a a big question. So I I'd say I only joined Chai maybe five months ago now. So I have been spending my, my initial time here working with our, our leadership team and our colleagues in our country offices, really brainstorming what are the problems that our government partners are facing day-to-day where AI might be, might be useful. 7:00 We really want to be be problem driven and, and not just run around looking to, to deploy AI where, wherever we, we could. So I think there's really five areas that we think there's a big opportunity on both in the, the short term and, and some of these will be a bit more of a medium term play. 7:22 Revolutionizing Disease Detection with AI Diagnostics The, the 1st is around AI powered diagnostics. So outside of AI, Chai does a lot of work in the diagnostic space, whether it's rolling out programs to to better utilize rapid diagnostic tests for malaria or HIV or syphilis. 7:40 And over the years we have done work that figures out how to get those diagnostics not only into market at a cheap price, but how to build them into clinical guidelines and and such. So we think there's a big opportunity in the AI space to expand the range of diagnostics that that are available. 7:59 And we've already done some work over the the past several years on AI detection tuberculosis based on chest X-rays where there's a limited number of radiologists or as a as sort of a second opinion or copilot for for radiologists. 8:18 And then the second area we've looked at is automated visual examination or Ave. we call it for cervical cancer. But there's a whole broad range of AI powered diagnostics that we think are are potentially very powerful from AI that enables point of care ultrasound to more radiology type work and beyond. 8:43 So I would say diagnostics are, are the first bucket. 8:46 AI as a Co-Pilot for Health Workers I'm I'm excited about the second bucket that you often hear about when people talk about AI for global health is what I would use a shorthand is building an AI doctor. And I think we're, we're not, we're not there yet. 9:02 There's, there's a definitely some intermediate steps between where we are now and, and what it'll take to realize that vision. But I, I think in the, in the short term, this means AI clinical decision support. What does it look like to have an AI copilot that various levels of health workers, whether it's a community health worker, a nurse, a clinical officer, a doctor has either running in the background or sitting as a second opinion as they are are going through and trying to conduct a differential diagnosis for for a patient. 9:39 So that's number two. 9:40 Enhancing Patient Management with AI Systems So these first two are really in the clinic experiences, but broadening out from that, the third thing I'm excited about is what happens when a patient actually leaves the the clinic and what does it look like to have ongoing patient management. 9:56 I know not just in the US but around the world is oftentimes patient, they might be newly diagnosed with diabetes or hypertension and they've only had two or three minutes with the doctor because the doctor has a long line of patients to see and that diagnosis might sting. 10:14 They might be not know exactly what are all the changes that they need to to pursue, how to manage their their medication regimen, etcetera. So what does it look like to build AI systems that provide support not only to patients, but to the family members that are caring for those patients to ask answer questions about recent diagnosis, answer questions about side effects. 10:41 So all these wrap around services, I'd say are the third thing I'm excited about #4 is zooming out a bit further and thinking about across the entire government health system, what does it look like to better manage health systems overall? 10:58 Optimizing Government Health Systems with AI In a lot of the in in the vast majority of the countries where try partners with governments, most Healthcare is provided in government clinics. So the government is responsible for everything from the commodity supply chain to the financial forecasts of, of health commodities and health workers, as well as the the training, promotion and assignment of health workers. 11:23 And all managing this very complex system across hundreds or thousands or 10s of thousands of health facilities in the large country involves a lot of data and a lot of management challenges. And I think AI can be very helpful there. 11:38 Accelerating Health Worker Education with AI Tutors The final thing we're excited about is what is the potential for AI and improving the training of the health workforce overall. So in addition to people being excited about AI doctors as as sort of this catch all benefit of AI you did, the other thing you come into here is an AI tutor. 12:02 And we think that there's, there's potential for this in the medical space, not only in medical schools themselves, but in continuing medical education to really customize and, and help clinicians better, better understand the material and engage them along the way. 12:23 And really over time, it's going to be more important for education in the medical space to include a component of it that is how to work with AI as things like AI diagnostics and clinical decision support tools come online. 12:42 Ensuring Culturally Appropriate and Valid AI Models Thank you. 12:43 Speaker 1 So many AI tools are trained primarily on data from high income countries, so how does Chai ensure that the models you deploy are clinically valid and also culturally appropriate for the diverse populations you serve? 13:00 Speaker 2 Yeah, this, this is a really important question. And maybe I'll just first start with the caveat that Chai's always working in partnership with governments to deliver services. So we're often not in a place where we are delivering the care ourselves. 13:15 We're often not building the technology ourselves. We're working in partnership with technology developers and government partners to, to make all this happen. So with that caveat aside, I'd say this is a really important problem that that we think about in all of our work related to AI. 13:36 And my professional background started my career was in monitoring and evaluation and, and Economic Research. And I, I approach these types of questions from a measurement perspective. And if we can't measure potential bias or potential inaccuracies and algorithms, we can improve it. 13:59 So I think the first step is really trying to identify where things are working well and and where they're, they're coming up short in any solution. And then from there, how you actually fix things once you identify those problems is a little bit dependent on the type of solution. 14:18 So if you forgive me, I'll, I'll go into a little bit of the the technical leads here. But if we think about two different types of AI, so maybe taking the chest X-ray radiology AI I mentioned earlier, that's what we might call a supervised learning model or machine learning model, where basically how the algorithm is trained is you have 1000 or 10s of thousands of examples of X-rays that have been labeled by experts with the outcome. 14:50 This one is tuberculosis, this one is healthy, this one is COPD, etcetera. And over time, the algorithm learns from processing all these images and and you develop the algorithm to maximize the chance it's labeling images correctly in that type of system. 15:10 How we address these biases is really getting data that is representative of the populations we're we're trying to trying to best serve and having evaluations that let us know whether we're doing a good job. It's, it's not always easy to to do this. 15:27 Data collection is hard and expensive and takes time and care, but that's generally that the path for the first bucket. I think what's interesting now is all listeners of your podcast, I'm guessing by now will have come across ChatGPT or Cloud or Gemini. 15:45 We have these big large language models, which are these machines that are algorithms trained on roughly the whole Internet and all all public data that's available to to these algorithms, as well as a bunch of private data that these firms have have collected. 16:07 And these types of these types of algorithms. If you or I go to try and use them for something and we find a problem with it, we can't just load 1000 more images directly into the algorithm and, and improve it and get a, another version of ChatGPT back. 16:24 So there are, there are ways to do that and, and fine tuning and whatnot, but, but generally you have to use a different toolkit for these types of challenges where ChatGPT or COD or Gemini or even open source LMS are, are not meeting the need. 16:45 So that can include things like changing how we prompt, what are the instructions we give to the algorithm for clinical decision tools. We can do things like we can provide clinical guidelines as a reference document that we're loading into the prompt. 17:03 And there's there's other techniques that allow us to do that. And then we can also think about what is the sort of collection of models that we can use in concert together to actually get to good solutions. And a lot of these more complicated AI systems, it's not just a single algorithm and a single prompt. 17:23 It's AI have one model that's really good at translation from Amharic into English. And then I feed that English translation into the large language model. And then I have a separate model that's doing image classification and it's feeding that take away into the model as well. 17:41 And over time, I'd say the models are getting more general purpose and and this type of orchestration might be less required. But as of right now, these are all the the sort of tips and tricks that we are deploying to when we identify potential sources of bias, how we can actually dive in and correct them. 18:01 How AI Addresses Clinician Burnout and Task Shifting That's awesome. 18:02 Speaker 1 So with over 100 million Tele consultations happening annually in some markets, clinician burnout has become a critical concern. So how is AI powered clinical decision support helping to address this challenge at scale? 18:18 Speaker 2 Yeah, it's a, it's a good question. I, I think we hear a lot about AI automating things away. But in this space it's, it's not always clear if when the AI is deployed, whether it's it's creating more work or it's just shuffling the work around. 18:42 Or there's a great Harvard Business Review article a couple weeks ago that said something along the lines of AI isn't automating work, it's an intensifying work. And what I think it it meant was sometimes when we can automate away some of the more trivial parts of our job, we're we're just left with like really the intense, mentally taxing, hard to do part of our job. 19:11 And if we're stacking that on top of one patient after the next after the next, if each doctor is having to really have their brain switched on for these compressed 2 minute consultations and doing 3040 fifty back-to-back, that can really take a, take a toll. 19:29 So I, I think we have to think carefully about one, how we deploy AI and E systems so that it, it leads to, to more sustainable work practices. And it, it's going to take a little bit of time for, for us as a sector to figure out exactly what this looks like. 19:51 There's a lot of talk in in the US about ambient scribes, these tools that are sitting in the background, excuse me, taking notes as the the clinician talks to the doctor. But there's sort of mixed evidence as to whether they're actually reducing clinician burnout or not. 20:11 I think in the global health setting where we just face a huge human capital shortfall. I think The Who is estimated to, to really address current patient needs, we would need to hire something like 10 million additional health workers in the next 5 or 10 years. 20:31 And just frankly there there's not the room in, in budgets for that to happen. So we're just going to be operating at this human capital shortfall for the, the foreseeable future. And I think the, the question is how can AI help address that? 20:46 How can it help these burned out doctors that are that are doing 10,000 Tele consults a year? And I think what I'm really excited about is like, where can AI be used for task shifting? 21:02 So there's this concept in global health called test shifting, where we think about how a less highly trained cadre of workers can do something that is typically reserved for a more trained cadre of worker. So how can a community health worker, I know Lonzo, you spend time at BRAC where where we first met. 21:24 BRAC has this amazing network of community health workers that have been trained to do a lot of the the work that in in certain narrow spaces that a nurse might do in, in other geographies prescribing a basic anti malarial or, or, or doing a child health check. 21:45 And the question really is, can we use AI in the the same way? Are there safe, responsible ways where for some of the easy cases, we can just, we, we can, we can push the solution to AI? 22:01 And one of I can't emphasize enough that like this all has to be done step by step very carefully with a lot of testing and, and rigorous clinical research. But given given that the challenges we face, I think it's something we we really need to to explore. 22:22 From Academic Promise to Real-World AI Deployment So when it comes to clinical AI, what are the biggest gaps you see between what looks promising in academic research and what actually scales in real World Health system? 22:33 Speaker 2 Yeah. So on this one, I think there's actually 2 gaps I'd want to want to think about. The first is the sort of gap between what we see in the research or lab setting and what we see actually deployed in in the real world. 22:49 And then the second gap, which is something that that we spend a lot of time thinking about and working on a Chai is what is the gap between technologies being deployed in high income markets and those same technologies or drugs or medications being deployed in low and middle income countries. 23:14 And what we've seen over the last 20 years is sometimes there's a 10/15/20, even 30 year gap or lag in that second category where there there could be a highly beneficial medication that's deployed in the US or Europe. 23:31 And due to patent restrictions or sort of lack of a viable commercial market in other countries, those life saving or drastically life enhancing treatments or devices that might not make their their way to market. 23:48 So we spend a lot of time thinking about how can we build the market, working with governments and philanthropies and the private sector to actually make this happen. And so thinking about the the five categories I, I laid out earlier, I think both in the diagnostic space and the clinical AI space, we see a lot of really promising proof of concepts in the diagnostic space. 24:15 A few of them have gotten to scale in market. TB chest X-ray I think is probably the most broadly deployed AI solution now in in the world with with companies like Cure AI and in India and many others really scaling that up on the but I think there's there's a long way to go. 24:38 If you look at the number of AI powered diagnostics that have a academic publication from a leading university or research lab, it's there's probably 30 or 40 or 50 really interesting and promising ones, but only two or three of those are actually at scale in in market. 25:01 So I think there's a really big gap there on the diagnostics front and it's something we're going to try to to help close a Chai on the clinical AI front, I think because of large language models have have really changed the game here in terms of what's potentially possible. 25:18 This is this is more recent and a lot of the research is actually coming from AI labs themselves or from clinic networks themselves as opposed to academia. And I'm really excited to see the research that even in just the last week or two has has come out on this. 25:38 There's a new paper from Penda Health, I believe it was in Nature about their work in Kenya deploying clinical decision support. And Google released their first paper about a experiment with their Amie platform. 25:55 It's Amie looking at their a clinical decision support prototype that can have a direct conversation between the AI and the patient and is at par from a diagnostic capability perspective with a primary care physician on a number of dimensions. 26:17 So we're, we're, I'd say we've moved past the initial like proof of concept, interesting blog post phase on the clinical AI front to getting to good initial experimental evidence. But we're still a little ways away from seeing at scale deployment, particularly in in public sector systems. 26:40 But given the amount of interest, I, I think we're, we're not too far away. There's a lot of really smart people working on this. 26:51 Speaker 1 So how does Chai balance introducing cutting edge AI tools with building local capacity so that health system can eventually maintain, adapt and govern these technologies independently? 27:06 Speaker 2 Yeah, it's a good question. 27:07 Empowering Health Systems for AI Maintenance and Adaptation I would say the most important thing here is we want to be thinking sort of about the end state at the beginning. So before we start any pilot project, we want to be thinking about what is the route to scale of this pilot project? 27:25 What is going to be the required tech infrastructure? If we're doing the pilot in a place that has much better Internet connectivity than the rest of the country, what is the plan actually look like to scale this to the the rest of the country? What is the business model that actually needs to to get this to to scale? 27:47 And even if we're working with nonprofits, we still need to think about a a business model who is going to pay for the solution at scale? What does that blend of financing look like potentially between philanthropic and and taxpayer revenue sources and what price point would we actually need to be able to deliver the solution at so that it is cost effective? 28:13 And those are all things that are at Chai. We, we want to be working with our government partners to think through from, from day one, and then we try to work, walk, walk on this journey together. Chai is is fortunate to often second staff into ministries of health. 28:34 So we are working in a very embedded fashion and, and it's not a case where we, we want to pilot technologies and then sort of throw it over the wall and, and hope that it gets scaled up. 28:49 Ultimately, the solutions we're we're trying to to help our government partners bring into the world, we want to be building them together from day one to meet the the challenges that governments face and are trying to solve for citizens. 29:11 Delivering Affordable AI in Resource-Constrained Settings So at a time when ministries of health in many low and middle income countries are grappling with significant budget cuts, how important is to focus on cost effective targeted AI deployments and what does that look in practice? 29:28 Speaker 2 Yeah, it was on the previous question was just thinking about the cost effectiveness point. And I, I think you, you've really nailed the current challenge here in just to level set it's, it's very tough times. And the, the global health space, the impact of aid cuts has been devastating in many of the partner countries that governments that, that try partners closely with. 29:59 And we are actively working to support our Ministry of Health colleagues in those countries to really make impossible trade-offs that they're facing. If you have a limited budget and you have to cut services somewhere, do you cut, you cut HIV prevention program or HIV treatment program? 30:22 There's there's no good answer to that question. There's it's and it's, it's really I, I don't really have the words to describe how, how sad and frustrating it, it is that that we're facing this as a, as a sector and as a world. 30:42 And I think from the AI perspective, it just really makes it all the more important that we don't focus on shiny toys. There's not a luxury right now to waste any money on some cool AI gadget that doesn't solve a real problem. 31:05 So I think the first, the first thing about being cost effective is we have to be working on the, the right and, and big problems where AI can, can be useful. But I, I, I am optimistic in it on a few points here. 1 is technology scales well and it gets cheaper as it scales. 31:31 So I think when we're looking for budget efficiencies, things that we we can demonstrate that they're effective in, in pilots if we can get them to to scale the marginal cost of delivering that tech solution, assuming we we have the the right infrastructure, which is a big assumption. 31:55 If we can actually get solutions to scale the actual marginal cost of software at scale is, is quite low. And when we think about the cost structure for AI, we see all these headlines about AI getting better every month, every two months there's a new big model release and it can then the big hard language models can, can have some new capability. 32:22 But I think what's really under appreciated over the last two to three years is a is a second trend, which is if you look at a given capability of model, so say GPT 4 which came out I think two years ago now a equivalently capable model has come down and cost roughly 90% year on year. 32:46 So over 2 years since that came out compared to the price at which it launched, it's now 1% the cost to access that level of what it was when it came out. And we see this quite consistently over the last couple years. 33:03 And who's to say as to whether this, this trend will continue. I think there there are limits to, to physics and how many chips we can might be able to, to squeeze on a in a fab and, and how much power we have to, to run data centers and what not. 33:28 But overall, the trend is for a given level of model capability, things are getting cheaper over time. And I think what's really will be exciting is if we forecast out this trend line another couple years and we think about OK models that are as powerful as we have today that you need to run a big data, you need a huge data center from Amazon Web Services or Google Cloud to run. 33:57 Is there a future where we get similarly capable models that can run on a low end smartphone and then there's zero marginal cost to be running that clinical decision support tool on a on a new phone? And I don't have a have a timeline on when we're when or if we'll we'll get there. 34:15 But that's the trend line that I, I think we have a potential to to be on and I think that will really unlock a lot of cost effective AI deployments in the global health space. 34:30 Building Foundations for AI in Low-Income Countries That brings us to the last question of the episode. So, Ben, looking ahead to the next decade, what infrastructure needs to be built now to ensure that Lmits can fully benefit from future advances in AI for health? 34:48 Speaker 2 OK. Thanks for the question, Monzer. So maybe I'll, I'll, I'll end where I I finished. Thank you. Excuse me. I, I think the obvious answer to this one is there's, there's hard infrastructure that that's going to need to be built to fully realize this. 35:04 So we're going to need reliable electricity and power generation, We're going to need Internet that's sort of basic utilities to be able to deliver this to a broader swath of, of the world. 35:24 But I think that the less obvious place that we need to invest in is on the, the softer side to fully realize that the benefits. And the first I would say is on health context and data. And if we can get to a place where we are making health guidelines more easy, we make health guidelines easier to digitally access. 35:51 If we make epidemiological data on how disease manifests in different places and in different communities more available, If we can build some of these public goods. 36:07 And this is one of the things we're exploring on in our AI initiative at Chai, is if we can build some of these public goods that enable entrepreneurs not just in Silicon Valley, but in Nairobi and, and Lagos and Bangalore to build on top of, I think we're going to see a lot more promising AI results and impact in in the global health space. 36:33 And the final piece I think we need to invest in is, is people. And not just engineers, though we will definitely need engineers, but people who think about product, people who think about user experience, philosophers and bioethicists who think about what is the right role of this technology in the world. 36:56 And really clinicians who deeply care about technology and how it can be used to help patients, really investing in in understanding how all of these pieces can fit together is fundamentally a human challenge. 37:13 And to really realize that, I think that the truly transformational potential benefits of this, we have to keep putting people at the the center of all these conversations. 37:27 Thank You for Listening to On the Map Ben, thank you so much for being here with us today. It has been an amazing discussion and I'm really excited about where AI takes global health, you know, next. 37:39 Speaker 2 Thanks so much Manzer, really a pleasure. 37:43 Speaker 1 To our listeners, thank you for tuning in to this episode of On the Map with Global Health. If you enjoyed the conversation, please subscribe to our podcast on Spotify to stay updated on future episodes. Please also feel free to reach out with your comments and feedback via the Huber Department of Global Health Instagram page. 38:02 Until next time, take care and stay engaged in the world of global health.