Ep 83: Guest Megan Othus and Time of Day Administration

Mae'r amser yna. Mae'r amser yna. Mae'r amser yna. Mae'r amser yna. o'r cyflwy. the first infusion that might be relevant because after that you're going to effectively have loads of drug on board for the next several months. Do you think that's very both nodding? So I can say at least how I came at this was purely reading it, reading these papers as a statistician and not knowing the science, all the science behind the checkpoints. And so I read the papers and found a number of recurring statistical issues, which is why I wrote the paper. But then when I came back to my clinical colleagues with it, they provided all of this information that I didn't know about those receptors being saturated and they're really just not, you know, several of the investigators said like I just didn't make any sense these results, but I couldn't explain why, you know, the data were the data were that they were being published. And so it felt to me like there's like this sense of relief when I could kind of break down some of the statistical reasons that you might end up with the end up with the answer they did, but still be consistent potentially with the intuition. There is about how these agents work. Yeah. So I guess the the basic hypothesis would probably be something like if you give the first, let's just say checkpoints in impative for the sake of this discussion now and put to one side, all the other different types of immunotherapy at a certain time of day versus another certain time of day categorical, but there's probably loads of detail in there as well, that there may be more efficacy. And it seems like it's the story is about giving it earlier in the day and the morning is better than giving it later in the day. That's that that's what I've taken away from it, but maybe it'd be good to hear from you some of the, let's say pitfalls or catches or what you've come across in the data, because obviously, you know, you're an expert on this. Um, so try and try to figure out where this starts. So thinking about that. So when you pick even like as you were saying kind of generally earlier versus later, but how do you draw that line of early versus late? You know, that the day is is many hours and in general, the analysis of kind of done this two group like an early late and picked a time that they consider to kind of do that. So that's always problematic when you have something that could be if you're thinking that this is somehow circadian rhythm mediated. You would think that somebody who had it one minute before that deadline versus one minute after their outcomes probably wouldn't be that different, but by the nature of kind of slicing it at that time, you're going to put them in two different bins. So, but that said, what most papers have done when looking at this is essentially look at all possible cut points that you could chop the day into for early versus late and they did their statistical tests, usually a law of rank test and then compare early versus late and then picked the biggest one. And said that's how I'm going to analyze the data and then went through their data showing that. And so there's a number of issues with that, but for people familiar a little bit with statistics like the multiple comparisons issue is something that's pretty well understood. And so when you look at lots and lots of things, you have to adjust your p-value to account for that. And you know that you're also when you look at lots of things in one data set, you may be picking a time that is very particular to that data set as an artifact of that data set and you not actually be able to translate and be reproduced elsewhere. And so that was one of the very easy things to see said, well why? And what struck me when I was reading the literature was how that time, that early versus late time varied from publication to publication because rather than take the previously found time. And I mean what I would think as a statistician I would want to do is if somebody already published, I'm going to take their time and see if I can validate it in my cohort and report that validation. And that isn't what was being done. I don't know if they checked it, but by doing that analysis and not finding the same number does mean that also there wasn't a lot of consistency. And those early versus late time points went from 11 in the morning to 430 in the afternoon. So it was a pretty big span of the day that was being chopped. And these algorithms are going to give you the time that gives you the biggest split regardless of whether it makes sense. That makes sense. Like it's just an algorithm you put data in and it's going to give you a number out. Even if kind of the actual trend is not kind of this bifurcated outcomes. So that was one of those, that was one of the issues and there are ways to get I mean one you can adjust your p-values for multiple comparisons, but also you can visualize how that trend, how that association changes over the day. And if you would expect there to be a circadian rhythm pattern, you would expect to see some sort of pattern, some sort of shape or ball or or some sort of shape or kind of plateau that would be consistent with that. When we looked at our data, it looked what I called a sawtooth. It was just like up and down, up and down, up and down, not indicating any specific pattern for time of day. And then the algorithm picked the tallest sawtooth, but it wasn't that there was any sort of, what did look like, a meaningful pattern with other than that just happened to be the spot where they're right. In James, if I can add some historical context here, there was this paper out of Emory called the memoir study where they looked at the checkpoint inhibitors retrospectively that they had given and they wanted to see if there was this time of day association. And I'm, we can certainly put it in the fact check if we need, but they did pick their date based on they looked at all their infusions and maybe they picked their median time. And so then it was before and after. Or as Megan said, they put the data in and they just picked the one that has like the biggest difference, the time of day. So once we saw that paper I said to Megan, well, we've now done this 1300 patient study. Can you do it like that? So she did exactly what she said as a statistician would do. She replicated their cut point and said, okay, by using their cut point, do we see a difference before and after? And we didn't see a difference. So then I went back and asked her, okay, well, don't use their cut point. Just any cut point. Is there any cut point? And Megan's like, of course, I did do that. I mean, that's the obvious. The next thing she was going to do is look for a different cut point, you know, that doesn't match their data. But there just wasn't one as she's saying, you can statistically ask the computer to tell you what's the time that has like the largest difference between the two groups. And then it didn't fit any sort of trend. So like an hour before, 15 minutes before, 15 minutes after, like the pattern of that graph goes in a different direction. And so that doesn't make a lot of sense that there's actually an actual cut point on the time of day. So there was this whole issue of, you know, a lot of these papers were saying, I think you're an author on one of these papers that did it. I'm sorry to point that out. I think Checkmate 238 did it. In that NEJM last kind of final publication on it. And it said, patients who got the majority of their first six months of infusions in the morning, maybe we're trending to a better survival than patients who got the majority of their six months of infusions in the afternoon. But then Megan will tell us there's an inherent bias in looking at data that way. Right? In our paper, the hazard ratio goes almost to one 15 minutes later, like Zach and I was saying. So again, like, so these papers on top of showing their one time should also report a sense of media analyses at least a couple other one. Show all the data, but to show that it's robust to a 15 or 30 or even 60 minute, because if it really is circadian rhythms, that should not be that much time should not matter. Yeah. Yeah. I mean, in my defense, I'll probably say that in that publication, I mean, I think it could have been analyzed differently. That's just say that, surely. And so then the other question in my mind is going to be about confounding factors, about why people have treatments at different times of day. And the obvious ones might be that, you know, let's just say that people are younger and fitter and, you know, they don't rely on transport to get to the hospital. They can have their treatments earlier in the day. And then maybe the people are coming later, you know, just for the sake of the argument, they might be more elderly, they're getting transport, they've got more medical conditions. Maybe they've got more symptoms from their melanoma in this case, which we know is absolutely a prognostic factor. And in the analyses done so far, there'd be any attempts to control that or is it not even worthwhile trying to control that because we need to do a prospect to randomize trial to really nail this question properly. Well, I think it's important to at least try and control for it. You can't do it perfectly. I mean, my perspective will never be as convincing and definitive as a randomize trial, but you should at least look and report and see if that hazard ratio is attenuated when you take into account those patient factors, which hadn't been done. So that was one of the other things we did. And one of the big ones, so it's on my list of things to ask an intern to do is I have this kind of free text on this trial. So I should also see most of my trials don't collect time and date. This section may one trial and 17 years that happened to because it was an FDA registration trial. So in general, we don't collect that data. So it isn't, it isn't easily available. But so we have this, we have a medical history that was right in text that was provided to us. I am hoping to maybe have a student with the help of AI, give us like a more comprehensive description of the health status of different patients and be able to control for that more explicitly. I can control for performance status. And one of the things we could control for was the distance it took with the distance between where they lived in their infusion center. And that is also associated with in the US at least the further you live from your from where you're getting your cancer treatment on average, the worst your outcomes are. There's lots of reasons for that kind of the sorts of jobs and education and the socio economic aspect of it, but also where how how close you are when you have a bad side effect and and how long it takes to get. So for some of our patients, it just is not physically possible to have an early infusion without say coming the night before and spending the night, which was not, which was not the plan for this study. So when so in our data, we could show or it was very clear in the data that further you live from your infusion center, the later on average your infusion was, which I think everybody can kind of understand though getting even better data to get a better sense, we also to collect quality of life. Measures on patients, so that's another way, which we didn't in this analysis, but to think about how to incorporate that if kind of the patients, they're general, they're just feeling worse and it's just harder to do everything if they scheduled their appointments later another feature with that kind of timing since for check waiting, there's patients are getting them over time, bring up that analysis, staff, mentioned was that the time isn't static for most patients over the course of their therapy changes and at least in our data set, a longer you were on therapy that earlier your infusion was. So I didn't actually wasn't expecting that when I went to go look for it, I was just, I just knew I needed to account for the fact that my change over time, but then when I saw this signal and I was thinking about it, I think it was a week that I was making appointments for my kids and it was, they needed to see the orthodontist and I was just going to take a bracket for orthodonture was off or something. So I'm just going to take the first available appointment, I don't care what time it is, I'm just going to deal with it versus when I schedule their appointments that I can schedule several months out, then I'm going to pick one that's like easier for my life and my schedule. And so I don't know, but one of my hypotheses is for this for for data new, they see patients and treat them I understand how that could happen over the course of their therapy is patients have more and more control over being able to schedule their appointments, they're choosing or maybe they're feeling better as the therapy is working. But the big thing is that when you, so one need to account for it, but two in that at least an R data set, the longer you lived, the more likely you were to have more than half your infusions early. And so there was going to this built in there's a kind of built in bias there when you don't account for that change just by the nature of the scheduling dynamics of a large cohort of patients. In in analyses where we say, you know, the majority of your six months of infusions were in the morning or in the afternoon. What's the type of error we're making or what's the type of bias we're incorporating because we're dropping off patients who maybe took one infusion in the morning and died or recurred. And so they never are in the data set of took six months worth of infusions. Well, okay, so there's two, there's two things. So the correct way to do it is do exactly that to just start with your subset of patients who made it to six infusions. And then I want to say it necessarily is biased in the sense of the outcome, but it now only applies to the subgroup of patients who make it to that point, which is not all patients. So only applies to the patient within a sitting in your office after six infusions. It's not the patient who before they get their first infusion, you're advising them because you don't know if necessarily if they're going to be in that group or not. So then there's a question of who can you make that, who does that apply to actually a lot of the publications analyzed everyone that way, starting at the beginning. And so that that is where the bias really came in was meeting everyone in there. And that was why they didn't do they did the majority. So then you also if you only had four, they could still calculate and kind of put you in regardless. And that those so the both of those. So the second analysis is just wrong. It needs to account for the dynamic nature of like the time of infusion. The first and else is you just have to be really careful in interpreting it because it doesn't apply to all patients. The place that is subgroup that like with with long enough doing well enough on that therapy to be sitting in your office talking about that. James, a question for you, because the half life is generally 20 to 25 days for checkpoint inhibitors. And so if we think it takes five half lives to clear a drug. So now we're talking 100, 120 days. So three to four months plus. Do you think there's any relevance about what happens with infusions number two, three, four, five, six when they're being given every three to four to six weeks. I don't see where it would unless there's a sort of a subtle effect between you know, drug on board and then even more drug on board as it were, which I think the difference is going to be small. And then I mean the other thing about immunotherapy in general, I think in jet points inhibitors is not necessarily just about the drug being on board. Right, these these drugs can have as we know permanent effects on the immune system. Maybe if you take the analogy flipping a switch, once you flip that switch, it's flipped right. I mean, you know, if there's all sorts of examples of that, I think. I was going to sort of my next question or would be to say, do you think there's anything more to learn from retrospective data sets at the moment that we don't understand. So are we now ready to say look, you know, this is an interesting question. Actually, it's not clear that there's really an effect in there at the moment. So we need to just do prospective randomized studies. So before the nature medicine paper of the randomized trial in China came out, I would have said, I think everybody should just reanl and respect it data. And adjust the p values, we should think it and and have like a more accurate view of the can't of these retrospective out. And now season put them in context, say after we presented ours, a number of people have contacted me to say, we looked at our data and we saw no signal, but then we just didn't publish it because it was negative. So I think there's also just kind of that simple publication bias also. So what I wanted to write the paper to show how to do the right analyses to then hopefully ask people say, hey, can you do it kind of accounting for these nuances and adjusting for your p values. Hopefully also create a space where then there could be some more balanced publication to show this isn't the only trial that didn't see a signal and since we and census was accepted a couple months ago, there's been a couple. And then we saw all in abstract presentations also not finding a signal. So I think there's some fun with randomize data and I realized that the crime I haven't looked in the last couple weeks ago, it's still kind of had the banner saying you are investigating that trial. really big and I realized it's lung cancer and I realized it's chemo, you know therapy. But given the momentum of all these retrospective analyses that were wrong plus that result, I'm not sure that I think a randomized trial will probably be needed to be convincing for some people. I think it's too bad because I don't, it shouldn't matter from the data that I've looked at at least and maybe at an individual hospital, it does matter. But I think one of the nice things about the trial we looked at is it was conducted at hundreds of hospitals across the US and so we're incorporating that variation that actually exists in the real world of where patients are getting treated and showing up that kind of for the most part their local hospital and what that looks like. I'm convinced, but while our kind of the general, what would be convincing more generally random, my say that I think is just more convincing. I don't, I think that there is equal place for that randomized trial. I don't see any that, you know, I don't really see any, but we won't be harming patients. I think the biggest thing though is, whether the patients already have cancer and to have a burden of making them worried about the time that they're getting their therapy and that if they have family or other responsibilities or just feel too terrible and can't get an appointment until the app and don't feel like they can make an appointment until later in the day, feeling like they might live. I mean, it's not just like a subtle effect right that is being reported. It's a really meaningful difference in how long people live. And so a randomized trial kind of perpetuates that burden if we have to do it, but then you may also that trial may not have a representative group of patients because the patients who have to travel further who just can't be there for a morning infusion. So we will just have to say no. So then I don't know. So an ideally a randomized family maybe in a, maybe in a country that's smaller. And so where those travel times. Yeah, I mean, yeah, but I, I have heard from when I presented that even in smaller European countries, this kind of distance effect is observed there too, not just in the US. Yeah, I mean, there are some excellent points. I really agree. The point you just made about people kind of feeling bad if they can't get there. And then just a couple of other points, which is that I mean, we've already. And I'm sure everybody has had lots of interest from patients because there's got a lot of air time, this story and actually even colleagues are saying, oh, you know, should be change everything. And I just said, well, no, I don't think this is sufficiently compelling at the moment. I think I don't know what you think something, but I think the other thing is it probably people will be interested to hear what you think about the nature of medicine study. Because it's a bit of an elephant in the room really isn't it in terms of prospective data. I don't know, I think it'd be wrong to finish without saying at least a bit about that as much as is known, let's say at the time of writing something you want to say something I think go on. No, I mean, I, I didn't try it. I mean, she did sort of say that the difference they saw in that study was almost extreme, like kind of hard to believe. But it's chemo immunotherapy. So then what's the effective chemotherapy on time circadian rhythm and there has been some stuff published on chemotherapy and time at day. But I mean, as far as I can tell the reason it's also under review is that the protocol versions that they kind of listed on clinical trials. There's just like some inconsistencies between the protocol versions and then their their record on clinical trials. Not the least of which I think Megan said this is what triggered that time of day and what even trigger them to do this analysis because this wasn't the primary endpoint all along. So they changed the time. In the final or the final version of the protocol that was included with the paper doesn't have an event driven analysis for progression free survival. And so in general when you analyze progression free survival, we do it based on events that tells us how why our confidence in our was and so that's where our precision comes from or a fixed time point say three years after the last patient was randomized those were kind of those are essentially the only two. Maybe you could come up with a combination of them but you have to formalize exactly when you're going to do your analysis and pre specified and do the analysis and for those of us in clinical trials, we've sat around waiting to have that time happen. So we know that you don't just get to pick it you have to sit and wait and follow the protocol. The protocol didn't have analysis point included. Which might be related to the change in the design and the time. Those times changed over the early late change, at least in the reporting of clinical trials, go over time, but in the conduct of the trial, it they perfectly met there a sign time of the published version. So just kind of understanding I think there's questions about that versioning of the trial that at some points, at least wasn't listed as randomized trial was listed as an observational cohort understanding what kind of really happened and what reflected the reality. So I went to the I went exactly to the design just like how did they pick this you know all those things though that's what I went to but other people have seen that the version some of the protocols. There's multiple versions of the protocol, but there's protocols from say 2022 that have papers from 2024 in the references. So there's also some. Versioning like documentation that is unclear also. Maybe I'd like to conclude with two points if that's okay guys and this s 1404 paper is it looks like we looked at a little over 600 patients here Megan looked at it let me not. I didn't know if I did anything here. So these 600 patients it does seem to be one of the largest data sets on this time of day question so that I would just like to make that point. And then secondly I just want to read off the conclusion here that our co authors you know they did have input on how we were sort of like putting this in context for a clinician and ultimately you know one of our co authors and I mean I should probably say the name but really felt strongly we needed to put a real punctuation on this what should people be doing with the time of day and organizing their infusions controls. So here's the essentially last sentence of the paper these findings do not support reorganizing infusion center schedules schedules to prioritize morning administration of adjuvant anti pd 1 therapy for patients with melanoma which would have substantial operational implications. So I can't remember what it said in two three eight and we can always fact check that because that was the other adjuvant anti pd 1 story that looked at time of day but it seems like right now and recognizing that this is an adjuvant setting and metastatic setting or could be other factors at play again other confounding factors other things that have to be accounted for in the adjuvant setting it may not be reasonable to start reorganizing. And then the other thing that sort of top of mine now that we've switched from infusions to subcutaneous injections in a lot of places what does that mean you know it takes about four days for that subcutaneous injection to get into the blood stream. So now does it even matter what time of day that shot is because ultimately it's about it getting systemic so anyway that's probably a nice place to stop because there's future future work to be done here I think. Yeah well now that was great thank you so much I learned a lot and congratulations as well because I honestly I think you know the field and many ways in the discussion because there's been lots and lots of discussion about this was sort of crying out actually for something like this so fantastic and I hope people read the paper and cite it a lot as well. Or they just go back and read and I mean that hard thing with that randomize trial but I hope people go back and look at their data and it gives them you know especially groups that looked at it internally or were part of that and then reorganize their system go back and look at I mean the physical still there. Okay but if you actually do the appropriate analysis and that signal isn't there and you're making patients and physicians lives harder to kind of accommodate this. Why? Yeah, why? All right great well thanks Megan this was wonderful thank you for joining us. Thank you for inviting me. Thank you. Bye. [Music] (upbeat music)