Simulating Clinical Trials with Orr Inbarr from Quant Health

(upbeat music) Welcome to Impact AI, brought to you by PixelScientia Labs. I'm your host, Heather Couture. On this podcast, I interview innovators and entrepreneurs about building a mission-driven, machine-learning-powered company. If you like what you hear, please subscribe to my newsletter to be notified about new episodes. Plus, follow the latest research and computer vision for people in planetary health. You can sign up at pixelscientia.com/newsletter. (upbeat music) - Today, I'm joined by guest, or in bar, co-founder and CEO of Quant Health to talk about simulating clinical trials, or welcome to the show. - Hi, Heather, thank you, it's nice to be here. - Or could you share a bit about your background and how that led you to create Quant Health? - Sure, so Rewind maybe 15, 20 years ago, I was a pre-med student on the Boston suburbs, and my mother being a physician, kind of, you know, and also just me being Jewish, you know, kind of, you know, being a good, you know, either doctor or lawyer kind of prompted me down that typical route. But somewhere along the road, I realized the long and arduous path that it takes to becoming a physician. And I think more, even more compelling for me was the realization that becoming a doctor ultimately meant that I would only be able to treat one patient at a time, which may seem obvious, but I was always looking for ways to impact the world at scale. So my father, being a computer scientist, also, I guess, helped spark the other side of me, the engineer in me. And so after undergrad, I basically started exploring life as a consultant and a citizen of the world, and kind of realized the value that engineering and that computational methods can have on pretty much all fields, but medicine especially. And so I decided to go down that, the road of pursuing a master's in computer science, specifically focusing on data science and machine learning, and very naturally combining that with my love of medicine and life sciences. And so very quickly to me, that became sort of the path that I was destined to be on. And so basically after school, I was essentially really at the forefront. I always, well, I would say I always tried to be and place myself at the cutting edge of medical research, intersecting with computer science and machine learning and AI. And so I founded my first company in that space in Boston in 2017, focusing on real-world data and precision diagnostics for oncology. And during that time, I got much more familiar with pharma, how pharma uses data to solve some of their biggest pain points and challenges. And one of the things that became apparent to me as a company that was providing both data and analytic services to pharma was that there is a constant desire in drug development and pharmaceutical research to always bring more data and to sort of generate, acquire, and get your hands on more data. And this makes sense since it's a very evidence, very data-driven industry. But at the same time, there was a mismatch there because there's actually quite a lot of data already out there. And I think data, when you look at the different industries, especially now with Gen AI, kind of really just regurgitating a lot of the same data over and over again, healthcare is probably sites that we need to generate to answer those questions. So that prompted me to essentially form my next company, which is QuantHealth, whose mission really was to solve the complex questions of drug human biology, bridging the gaps across these very diverse and complex datasets to answer kind of a new generation of questions that are, for the first time, sort of in human history, now able to be addressed given the advances in compute infrastructure and AI models. It was a very exciting time to really start the company. And I think it still is. When we started this was well before Gen AI was even a thing. And so that whole revolution has given us a lot of tailwinds and is definitely fueling what we're doing. So it's an exciting time for sure and a lot to look out for. - Tell me more about what you're doing at QuantHealth, what problems are you trying to solve today? - So QuantHealth is a clinical trial simulation company, essentially, if you kind of take a step back and look at how drugs are being developed today and with an emphasis on clinical trials, we're essentially doing the same things that we were doing 50 years ago. We're basically, we have a drug that we think has some potential and we just go ahead and find patients in the real world, give them the drug and see what happens. And there's very little sophistication to that. Kind of look at what we're doing produced at scale. Go back to again drug development and none of that happens. We discover the drug or we figure out how to manufacture in small quantities and straight to humans it goes, essentially, without any virtual testing, without any simulation. And so it's no surprise that over 90% of drugs that make it to human trials ultimately fail. Just because we don't do sufficient testing that is cheap and scalable, we're looking at this in a sort of a very cold manner. But that doesn't even address the human element, right? I mean, these are real people who are giving actual drugs who, some of these drugs could be unsafe, some of these drugs could be safe, but could be ineffective. And when you're on a trial, it's typically between that and another drug that you know is effective, but you're taking a chance. So a lot of it is besides just being expensive, inefficient, a lot of it's also just unethical. And yet it's just the best we have. And so we do it anyway because without that, there would be no drugs at all. So all that's to say, again, that the need for doing better data-driven assessments of drugs is paramount. And trial simulations is one of the most promising ways to go about that. Because you can do it in such a holistic and all-encompassing way, because again, trials are really, really complicated. And there's a lot of variables that go into that. So if you can truly simulate a trial and sort of all its components and really give a good signal on what the results will be, you can really help the pharma company make better decisions about, A, what drugs to even take the trials or not, what programs to discontinue. For those that do go to trial, how do you make a better decisions on what patients to target, how to administer your drug most effectively, maybe synergizing it with other drugs, so that ultimately we get to the promised land, which is an approved drug, which just happens so rarely today. So that's essentially what we do. We help answer these questions around trial futility and trial optimization. - How does machine learning help in answering these questions? - So it's core. I mean, it's central to everything we do. And while there are a lot of companies out there that use AI or use machine learning, and today that's probably pretty much any other SaaS startup, for us, the machine learning component is the essence. We actually build the models themselves. We don't take, for instance, gen AI foundation models and fine tune them. We actually build the foundation models, so to speak. They're not gen AI based. They're built on different architectures, but nonetheless, we're an AI native company. So we actually build the models that fundamentally model out the drug human biology to actually run these predictions on a patient level for these trial simulations. So we're, you know, more than half the company is data scientists and engineers that really focus on that problem. - Are the models trained to predict something like, is this patient going to respond to treatment or not? Is that that type of binary decision? Or are there other tests that we tackle with machine learning? - Yeah, it's more granular than that. So in most clinical contexts, you're interested in a temporal prediction. So that is to say, for instance, will this patient remiss, will they experience a remission in their disease in the next six months, for instance? Sometimes the question is even more granular. And it could be, for instance, you know, if we're talking about a weight loss drug, how much weight will the patient lose in the next, over the next three months, for instance, things like that. So the question can be typically nailed down to a single point in time where kind of the measurement is taken, the outcome measurement, but it's very precise because that's how the data is ultimately standardized and analyzed for the FDA. - What type of data do you work with in applying these machine learning models? - So it's a combination of data sets, basically in order to predict, you know, how these patients will respond to novel therapies, you essentially, you need two core ingredients. So the first is patient data, right? And so we're modeling out these kind of digital patients, if you will, and so we need patient data to represent those patients, right? Treatment histories, diagnostic information, outcome data, treatment histories, lab results, vitals, et cetera, that sort of thing. And so we work with a variety of data aggregators that essentially extract information from the healthcare system, you know, EMR systems and insurance claims. The second piece is essentially drug data. And in order to model out novel drugs, we need to understand how those drugs actually work. And so for that, we build out knowledge graphs from a variety of different kind of genetic databases, pharmacology databases, a lot of different publications. And we stitch that data together to essentially build a map of drug human biology, and understanding how different targets participate in different cell processes, and then how those targets are affected by different therapeutic entities. And so that kind of lets us build out these sort of digital drugs. And so we then have these two different data domains, right? This patient domain and then this drug domain, and then by combining those two things along with some other data sets, but those are really the main ingredients. We're able to train large deep learning models to understand how different drugs with different mechanistic properties affect different patients with different clinical characteristics, and thereby run inference on new patients and new drugs to predict how they will respond to a novel therapy in the context of a trial. - What kinds of challenges do you encounter in working with and training models based on these two different types of data? - Oof, challenges are never ending. Well, I mean, the first challenge is always with the data itself. Clinical data, for instance, can be extremely messy and large. And that combination of large and messiness can be particularly challenging. And this is in a world where a lot of that data is semi-structured and it's not even text. So even in a world of kind of gen AI, you can't just kind of snap your fingers in and get the solution. So it requires a lot of work to structure and harmonize the data. I'll give you one example. One of the models that that we're building tracks HBA1C, right, in the blood. Different, you know, hemoglobin measures and whatnot. And for most patients, we only have HBA1C readings, maybe once a year, if we're lucky. But in a clinical trial, you're interested in measuring the effect on a monthly or even weekly or even daily basis. And so we have to build these imputation models to essentially help fill out sort of the missing data, so to speak, right? So there's this whole kind of layer in a lot of these data sets that is kind of the latent information that you know is true about a patient, right? You know, a patient is obviously a living being that experiences different events and is under a constant changing environment, whether that data is captured or not in the electronic health record. So you have to always find a way to infer some of that information, even if it's not directly available in the data. So that's a big one, you know, then you have data bias, you know, a lot of the data, you know, different patients are treated in different ways across different geographies, you know, and across different, even socioeconomic contexts. So accounting for all that and sort of finding the single source of truth on a lot of that is oftentimes difficult. And so those are things, those are typical problems that we deal with. - How do you validate your machine learning models? You know, in particular, you mentioned bias. How do you make sure that your models don't end up incorporating that bias? - Yeah, so that's the tricky one that we constantly kind of work against. The general idea is to incorporate exogenous data sets that you can use in some kind of external fashion to benchmark and ideally debias your models. And so we use publicly available data from clinicaltrials.gov, which, you know, is one of the largest publicly available clinical trial registries where companies are essentially required to post the results of their clinical trials. And so we have this, we're fortunate enough, right, to have this very large database of, you could consider it a gold standard in a way, although clinical trials also have their own bias, but we can't control for everything. So we have this clinical trial kind of gold standard data set that we can then use to measure ourselves against and see where we're biased or where we're, you know, potentially kind of systematically over or underestimating and make those adjustments and essentially debias the models as much as possible. - In getting these models working and validated, I imagine there's a fair bit of knowledge about healthcare, about how drugs work and their characteristics that would be very important to incorporate. How do your machine learning developers get up to speed on this knowledge or learn it so they can incorporate the important characteristics into the models? - Yeah, it's tough, right? 'Cause to your point, you know, there's the kind of state of knowledge on various drugs is constantly advancing, right? As different PhD students and researchers and pharma companies are advancing the state of our understanding of these drugs. And so we have to track that we have to incorporate that. And basically we go through, so every trial that we simulate, we first go through a data enrichment process where we look for the latest information in terms of research publications, in terms of, you know, recently completed trials that are relevant to our drug of interest. And incorporate that data into our data sets so that, again, we kind of have the latest and greatest of any given drug in development. It's a complex process that requires a lot of Q and A and automation, but it's really important. - How does the regulatory process affect the way you develop machine learning models? Are the things you do differently than if you weren't in a regulated domain? - It's a good question. You know, what we do isn't fundamentally regulated because we're essentially helping pharma companies make better decisions about their internal processes, right? That being said, what they do, obviously, is ultimately regulated. And so they are used to being scrutinized very carefully. And so naturally they scrutinize us as well, right? And for good reason, and then I think that's a good thing. So that's sort of approach to having everything be controlled and validated and regulated that definitely spills over into what we do. You know, that just puts a very high bar, right? On what we do. And for that reason, we go through an extensive model validation process to understand, you know, very granularly what the data is that goes into the model, how the models are behaving, how they are learning, how the models are actually performing on the prediction. And then, you know, well, those predictions and simulations validate against actual clinical trials. So all of that is part of our validation process. And it's definitely a direct result of sort of the scientific rigor of the industry. - Is there any advice you could offer to other leaders of AI-powered startups? - Well, I would say especially in today's world where it's so easy to say, you know, we're an AI startup or we AI this and AI that. I think one of the things that, one of the truths that holds constant that I've seen for the last decade or so in AI startups and still holds true today is focus on the product and on the value and not on the model, not on the prediction. That's important too, right? But that's, you can oftentimes miss the mark by focusing just on the core technology. And so how do you actually build a solution that actually solves a real problem, a real need that fits into a real user workflow gets them from point A to point B as quickly and as efficiently as possible with as little doubt as possible. And that might sound obvious, but I think a lot of entrepreneurs kind of miss that somewhere along the way. - And finally, where do you see the impact of Quant Health in three to five years? - I would say 50% of all trials, phase two and phase three trials, will utilize either Quant Health or a technology like Quant Health to design and execute their clinical trial. - Imagine overall that'll make these clinical trials more efficient, fewer drugs failing before they get to patients and a lot of good overall. - Oh, a hundred percent. I mean, it should have a dramatic difference in the amount of drugs that reach patients and as well as save a ton of money on drug development, which of course frees up resources to develop more better drugs. So it'll be huge, no question. - This has been great or I appreciate your insights today. I think this will be valuable to many listeners. Where can people find out more about you online? - Well, Google Quant Health, QuantHealth.ai. Yeah, and you'll see a lot of information there. A lot of interesting things. Thanks for joining me today. - Thank you, it's a pleasure. - All right, everyone. Thanks for listening. I'm Heather Couture, and I hope you join me again next time for Impact AI. (upbeat music) - Thank you for listening to Impact AI. If you enjoyed this episode, please subscribe and share with a friend. And if you'd like to learn more about computer vision applications for people in planetary health, you can sign up for my newsletter at pixelsscientia.com/newsletter. (upbeat music)