FF 81 Metformin Termination as explored by Target Trial Emulation

Who wrote the word "pucilinebidi" because I really have to look up a word when I read an FJC but Swap, are you raising your head? Yeah, I threw in some rhetorical flourishes. But it is totally right, the FDA was chicken, that's what I'm trying to say and a bunch of other people. But FDA in particular. Welcome to another episode of "Freely Filt," the regularly irregular podcast that summarizes and discusses recent FJC journal films. FJC is an online destination for both publication peer review that provides summaries, visual abstracts, interactive chats, newsletters and podcasts in the research developments that are driving the Prology Book. This podcast is for educational and entertainment purposes only and is not intended to give medical advice. If you have questions about your health, this isn't a place to find answers. But if you are looking for a Royal King Journal Club masquerading as a podcast, we've got you covered. This podcast will discuss off label and unapproved medications. Hello, my name is Joel Toff at kidneyboy.bsky.social on Blue Sky. Tonight we have Swap. Hey, I'm Swapnil Henema. I'm an aphrologist, epidemiologist and the University of Ottawa. I am at hswapnil.medesky.social and I do have a disclosure in the form that I don't stop metformin until people start dialysis. And Jordi. I'm Jordi Cohen. I'm an aphrologist and epidemiologist at the University of Pennsylvania. I'm on Blue Sky at Jordi BC. I don't know what other dots afterward. And I also too can continue to metformin on all of my patients until they go on dialysis for many years. Oh my god, you use metformin on dialysis? Until they go on dialysis. Continue to be until they go on dialysis. I was like, there's cowboys and then there's Jordi. No, never I've never continued on anybody on dialysis that I am willing to check in for because the chicken, the chicken appropriate, pre-accussion, that's what we call that. Okay, and we have one special guest at Warnow Foo at Warnow. Introduce yourself. Hi, I'm Edward. I'm an assistant professor and epidemiologist at Lyon University Medical Center in the Netherlands and also in still and training to become MD. So I cannot prescribe or discontinue metformin yet in patients. Wait a minute, you are an assistant professor and medical student simultaneously? That's right. So how did that work? Like you started med school and you did a PhD or something like that? Like, I'm sure our listeners would be interested. Yeah, please. Yeah, it's good. I started with a bachelor's degree in medicine. So it's six years in total and the Netherlands could into a bachelor or three years and the master or three years. And after the bachelor, I already started very early in my bachelor with research and then after I enrolled in the MD PhD program. So I first finished my PhD and then got the amazing opportunity to also to postdoc in the US in Boston. So I pursued that and then I went back to medical school to still become MD. And when will you finish medical school? I think somewhere in 2027. And do you scoff at some of your teachers that are like just associate professors? You're like, oh, get away from me. No, no, no, of course. I'm only just an assistant professor and a medical student. Julie's an assistant professor. I'm not even an assistant. I'm the worst. He's terrible. Okay, that's an amazing story. We love that. Okay, so as as Dirty and Swap alluded to, we're going to be talking about this recent study stopping versus continuing metformin and patients with advanced CKD, a nationwide Scottish target trial emulation study. And this is we have been we have been kind of sneaking up on this topic in a number of times that we are seeing every time people look at metformin, it seems safer and safer. And this comes from I think the caution regarding metformin, which was approved in the United States shock late, like not approved in the US till 1995 for a drug that was in discovered in the 50s. And I think a drug called fenformin that was approved in the US in the early 70s poisoned the well. Benformin has about 20 fold risk of lactic acidosis. Like 60 per 100,000 versus three per 100,000 for metformin risk of lactic acidosis. And in fact, it was the very first and I don't know of any other drugs and someone could correct me if I'm wrong. That was pulled by the FDA with like, they're like, you need to stop the struggle within 90 days. This stuff is absolutely coming off the market. We are going to ban it from the United States. And it's interesting. It still spots up like there was a recently been formin was being added to Chinese herbs as for diabetes. And it was causing lactic acidosis. People taking these herbs like fenformin was clearly the bad actor and it kind of poisoned the well for metformin and made everybody terrified of this. And you still see it like metformin not available in among inpatient medications because of the risk of AKI is so high among in patients. And there are specific contraindications. You need to stop it before a contrast study because they can develop AKI. It's like six derivatives later they could get lactic acidosis. And every time we looked at it, reevaluated, they're like actually not so dangerous. Maybe we can initially know GFR less than 60, then know GFR less than 30. And now we're plumbing new depths with your study. Swap, do you want to add anything to the background there? Yeah, I mean, you covered most of the stuff. Another thing is because of that contrast stuff, which I'm sure there were some patients who got metformin, got AKI, then someone did not stop that metformin and maybe they had also gotten stemming and that's why they went into shock and they got lactic acidosis. So it's on the drug label, unfortunately. So because it's on the drug label, the perception is that metformin is nephrotoxic. It's definitely not nephrotoxic. That part is for any of the listeners, it's not nephrotoxic at all. It's only in the setting of AKI and I would argue maybe not just AKI, but often shock, right? When you're not able to metabolize lactic acid, that's when you get that mala, the metformin associated lactic acidosis. The other problem though is that if anyone of you have seen mala, it's a horrific condition. You know, I've never seen backups that low, right? A bike up of two or three, a pH of 6.8. It makes the patient very, very sick. So that's part of the reason why, because of this, you know, everyone is afraid, is because mala is a very, very difficult and dangerous condition. High fatality rate, right? Patients need to be in the ICU, etc., etc. But it's not, I'm not sure anyone has ever shown that the risk of mala goes up as kidney function goes down, right? It's not like a linear relationship. It's more of, you know, like the patients have seen with mala, many of them, most of them had normal kidney function and then they got AKI. So this whole GFR-based cutoff was sort of like a very, let's be cautious. We say kind of a situation rather than having any evidence at all related to that. But you're right that most societies, most regulatory authorities say you should not start informant, whether GFR-based is to not start below 45. But if you have started, you can continue. But at GFR of 30, you should stop, right? Just a hard stop of stop informant, whether GFR is 30, just because of this perceived higher risk. And you already, what made you be brave to continue informant? Looking at the data, I just remember as a fellow, we had learned this obviously in residency, not to ever prescribe it and more advanced chronic kidney disease patients as you just described and not to continue it and even further advanced patients. And everyone just does things, right? Because you're told not to do it and you have, you develop your sort of heuristic, your laurels that you rest on and because I was told to do this, I continue to do this. And I learned early in fellowship, it's great to question those things because as soon as you start researching the background behind the ones that don't come with a citation, there often is no citation for it. And this was an example where the same incidence of lactic acidosis comes up with just about every other hypoglycemic. And it's very rare. It's like, what I think is like three cases per 100,000 or something like that. And that's the same thing you'll get when you Google just about any other hypoglycemic and the incidence of lactic acidosis. My form just got the bad name and I didn't know the background of Fenn form and so thank you for that, Joel. And that I think of that historical background. It clearly created this greater level of concern. It's like your pre-test probability to people that this drug is going to be more likely to cause harm even though it obviously is completely different. It also casts me up because every drug that starts with Fenn apparently is horrific. Yeah, I like that swap brought up mala. It's metformin associated lactic acidosis which I've seen a number of times but I do not know if it's metformin induced, right? I get a lactic acidosis, I got a patient on metformin. I treat it as if it's induced, right? I provide them dialysis to remove that metformin but usually there's other situations going on you. You point out, am I septic shock? There's something else that could possibly explain the lactic acid. It's not in every case but in enough cases that I'm like, God, I don't know if this is really induced by metformin but it does get clicked in my brain as, oh, I saw another case of mala, right? Because it's just the association that makes that diagnosis. But is it just that diabetic patients are more susceptible to sepsis? So where we're sort of? No, absolutely. I always think about that but I'm like, you know, in this situation I'm going to attribute that causation. here because I'm going to provide dialysis as a rescue therapy, whether it gets better or not. And I think we need to add that on. One of the things that's so frightening about mala is people, this quoted 35% mortality. And again, is that from the incident that caused the AKI that caused them to have the high level of metformin or the cardiogenic shock that caused the lactic astrosis, it's never dissected out again, just an association there. Exactly. The other last thing is that the FDA, though it approved metformin in 1995, they still had the GFR cutoff as like a creatinine cutoff of 1.4 for women and 1.5 a man. So that's like a GFR of 60 or so, right? Roughly. So it's only in 2016 that they said, oh, below a GFR of 60, you can use it. And again, in hindsight, this is so much, so frustrating to see because remember 2015, 2016, that's when the flows in came online. So until then, all we had if you were not giving metformin is self-unileuria and maybe a little bit of those glit as zones or what have you are insulin. So none of these have any end organ benefit. The only drug in the 90s and 2000s which had end organ benefit was metformin and you could not use that with the GFR below 60 until 2016, right? So paradoxically, you know, the data for the safety of metformin is being generated. And at the same time, it is becoming a bit a little bit redundant because we have all these other fantastic drugs that are coming online, which is kind of, you know, again, it's frustratingly paradoxical. It's still good, I think, but formula is still a place I'm happy this research is being done. But now we do have alternatives, but in the last few decades, when we did not have good alternatives, we were forced to use, you know, self-unileuria as an insulin instead because of this whole fear of mala. And before we go into the methods in the in the NFJC discussion in one of the last paragraphs, there's a sentence. This is as clear an example as can be seen of regulatory overreach as pharmacological pusillinimity in the misdirected pursuit of safety. And I just need to know who wrote the word pusillinimity because I really have to look up a word when I read NFJC, but that was a common I had to read that had to look that up. It's what are you raising your head? Yeah, so the summary was written by Christina, Mealy and Brian, but I threw in some rhetorical flourishes. But it is totally right, the FDA was chicken, that's what I'm trying to say and a bunch of other people, but FDA in particular. Again, there are reasons that we need not go into the philosophy of, you know, thalidomide and all those things that led to FDA being so afraid, but there are there are reasons which were unfortunately not scientific. Outstanding. Outstanding. Okay. How did you end up on this study? Yeah, so my own expertise is that I combine also inference methods in my research with routinely collected healthcare data, mainly to answer questions on the effectiveness and safety of treatments. Of course, in the nephrology space, but lately I've also been doing some studies in heart fur and type 2 diabetes. And last few years, I've been using a lot of the target triangulation framework, which we will discuss a lot during this podcast and the authors or the lead authors and the senior author on this paper knew that was one of my expertise in the field. It's really a good framework to improve the quality of observational studies and get an answer that is hopefully as reliable as you can get from observational data. So I think the authors saw the work that I had done. So you were brought in for your expertise in this method, not for your content expertise on this topic. Exactly. Yeah. Got it. Okay. Yeah. So the question, this is using the target trial methodology, which will come to shortly, but the question is, the FDA, as we just discussed, most health regulatory authorities say, stop, mid-for, mid-for, mid-gf, or 30. What happens if you don't stop? And specifically, they're looking at cardiovascular outcomes because, you know, the from the previous trials, one of the things that my firm has been shown based on the UKPD's trial was cardiovascular outcomes. So they said, hey, let's look at patients with advanced CKD to see if stopping versus continuing metformin has an effect on cardiovascular outcomes. Now, how do you study that? So they had, as Ed said, you know, combining routinely collected data. So in Scotland, they have a bunch of databases. So they combined the Scottish Care Information, Diabetes Collaboration, the Scottish Renial Registry, the Scottish Mortability Records, which I guess would have, you know, information about that and the National Records of Scotland. So you can, you have a system where a lot of, you know, big, big brother is watching and big brother is collecting all this data. So there was a lot of, you know, data, I guess, on drugs, on hospitalizations, on GFR and on outcomes. So there must have been a bunch of work to combine these databases. So hold on. So I just want to get, I just want to get a sense of, you know, how did you get here? Like, it's a Scottish group and is that why we have Scottish data? Or did you find the best data sources to answer this question? Did the question come first? Or did it like, we need data on Scottish people and we're going to grab this database. Like, what's the, what's the, the source there? What's the chicken, what's the egg? It's probably that the authors had access to Scottish data, they are Scottish and that's the data source they used. Of course, in an ideal world for every question, you would look for the most perfect data source to answer that. Okay. So you've seen a lot of these data sources. How would you rank the quality of this data that you were looking at for this study, for this question? I'm not too familiar actually with the Scottish data source. So for me, it would be very hard to assess exactly how the data are generated. But what I do know is that indeed there is a very large difference between different data sources in how reliable they are and how good they are to answer calls or questions. I can comment on the GPRD data and the thin data at least. I've never accessed Scottish data, but I've used the rest of the UK data, which I believe include some Scottish data. And that data is very high quality. Similar to what you've used in your other studies, I believe. I'm in that it's this electronic health record data that follows people from our life course or that at least records if they leave practices and go to a different practice. The cool thing in the United Kingdom that their setup is that the GPs handle everything. And so if somebody sees a specialist and a specialist says, "Oh, I'm going to recommend this," the GP then puts in the order for it. And so you have a central source of truth and that's what's recorded through this electronic health record data set, usually. And so I can't test the diabetes cohort because this is 99% of diabetic patients in all of Scotland, which is different than what's covered by GP and the health improvement network in the UK, which is like 10% sample of the population. But if it's anything like their nature or what they're able to get, then you get some pretty useful longitudinal data that's quite realistic to what the person actually is doing and receiving, particularly the prescription component, which is why I've loved it for pharmacopodemiology studies in the class. Because you know when the person has the prescription written and you can often determine when the person feels the prescription as well. And in the UK, they're on 90 day cycles where the GP has to recycle every 90 days. And so that's something else that's like a nice sense of reliability. Like there's not much flexibility within that in terms of the way that their health system works unless you have to make a medication change. So it works very, very well, especially for marginal tropical models, which are in these like perfectly sized buckets. And for such a consensory waiting, which Edward will talk about where you have like these very discrete time periods that you're stuck with. The other part of the database is that this study could not have been done in the US because at a GFR of 30 you had to stop, right? And places where there was a hard cut off and people were stopping the metformin, then you obviously, you need to have patients who are getting metformin, even if you're cloning, you know, right? So that could not have been done in the US. Right. And we don't want the continuing of the metformin to be a marker for bad doctor. Exactly. Have you a marker for good doctor? Yeah, but it probably will be a mix, right, of really good and really bad doctors in the end. Well, and I'm looking at this study. I'm thinking of it as the doctors who continue it are the ones who pay attention and sort of ask some questions or question the reality of like what you work within and are actually continuing a drug, even though you're told you shouldn't. And so I actually do think it's representative of different clinicians more than it is differences in patients. Yeah. And again, this, this dovetails really nicely into the next, you know, I'll quickly cover eligibility, but on the whole methodology. So the eligibility was patients with a they have to have type 2 diabetes. They had to already be on metformin. So they are, this is not looking at starting metformin. This is about continuing metformin. And they had to have a stage four or five security incident stage four or five security between the study period that's, you know, 2010 to 2019. So they previously had a higher GFR and between 2010 to 2019, they developed GFR less than 30. They excluded patients on dialysis or who had kidney transplant and people with less than 18 years of age. But the way this was done in the emulation is they had to have a security stage four or five and they had to be adherent to metformin. Right. How do you describe somebody who is not started is that in the previous year, about 80% of the year was covered with a metformin prescription, you know, so between January 15, 2010 and October 30, 2018. And the treatment strategy was again, if this was an RCT, which the study is trying to emulate you are looking at, you know, intending to continue what's intending to stop. So this is where things become very hairy. And I'll describe it at a very, very high level and I'll let and correct me or tell more details. The problem with doing an observational study without TTE is exactly what Jordy was talking about before, right? So it's a marker if for good things are bad things. So for example, if I'm continuing metformin, because you know, Jordy and I are great doctors and like Joel and we continue metformin. And then if the patient does better, it's because they are getting care from a great doctor. Or it could be a marker of somebody not being a good doctor and not looking at what the criteria of a drug should be stopped. And that's why they are not stopping. And then bad things happen because they just are not a good doctor. So that's one. But the other thing is that it could also be that a patient develops some side effects of metformin. And that's why the metformin is stopped. Maybe they got an acetic acidosis or maybe something happened. And that's why they maybe they had an MI and they got contrast and that's the metformin was stopped. And the database doesn't know necessarily to pick up all those kind of biases that are there. So if you do a routine, a standard observational study, people who in whom metformin is stopped are going to be different than people in whom metformin is continued, right? For those other reasons, which are not captured in a database. So even if you do propensities, go matching or whatever, you may take out some of those confounding elements, but you may not be able to take out, you know, care by Jordi versus care by Joel or, you know, people like Jordi versus people like Joel and other aspects like that. So I know Jordi's spraining. So I let her interrupt before we go on to it. Oh, yeah. No, I think I don't probably get to this. But unfortunately, even target trial emulation doesn't overcome unmeasured confounding. There's no tool we have that does. The key is to design your data set and to try to ask your question in a way where that's not going to influence your outcome. There are sensitivity analyses that can be done of one that's like sort of controversial as for example, the E value where you can calculate how much unmeasured confounding can impact your result and whether your results would still be persistent, even if you had a substantial unmeasured confounder like the really good doctor. But unfortunately, that stuff can't be overcome by any of these. That's the only can only be overcome by randomization or by like maybe some modern technologies that can try to improve upon how we collect some data and the electronic health records. For example, like you can use a really fancy study as a natural language process thing to try to dig into notes to figure out like, you know, is this doctor that much better than that doctor? Yeah, let me chime in here as well. So I think it's good for the audience to go one step back and just explain to them what target trial emulation is because it's just a framework for designing and analyzing observational studies. So it's basically in full of two steps. First, you explicitly specify what the hypothetical trial would look like that you would conduct to answer your calls to question. And then in the second step, you're trying to emulate it with observational data. One of the most important things to realize is that it has a number of advantages, which we can go into a bit later in the podcast. But one thing that is remaining and that's inherent to observational studies is there could be a meshed confounding and target trial emulation does not solve this. So the issue you mentioned, swap like the good doctors continue with it and the bad doctors don't continue with it and maybe the good doctors treat their patients better and then the patient outcomes are better. If you don't collect that in the data, then your analysis will be biased. If you collect it in your data, for example, the good doctors better treat the HBA1C or they start additional never protective medications and you adjust for that, then of course you correct for that bias. Yeah. So it is possible, but it depends on the data you have. And one thing I like to say, like I love that explanation and thank you for being so clear with how you describe something that we know is tough, is going to be a lot to unpack. But one thing I like to frame this when I'm teaching students on the topic is it's a pragmatic trial. We're not pretending we have placebo. We're not pretending we have blinding. We're pretending this is the trial where you pragmatically have people stop or just continue drugs. So this is much more similar to, for instance, the DCP trial, the one that was ortho-done versus hydro-orthyazide, where it was really pragmatic where it was like an electronic health record, driven study where people were clinicians were told either you're giving ortho-done or hydro-orthyazide. Patient knew what they were getting. I've been in most of these studies and they're often completely unblinded. As I mentioned, that's the study that's being designed here. So it's not going to be a placebo controlled, perfectly double-blinded trial in addition to the fact that it's not going to be able to address unmeasured compounding. It doesn't weaken it and we'll talk and I know Adam's talked more about this. I think it's just really holds observational studies to a higher bar to try to get them to be closer to being like a pragmatic trial rather than letting them sort of flow with all of these horrible sources of bias that are often ignored. Exactly. And it doesn't mean that if you do a target-traimulation study that there's always a measured confounding, no, the result could be causal. But again, it depends on the data you have and the methods that you use. And the person that Ed did his post-op with, I imagine you were with her, not her, and Miguel Kraft in Boston. I think we're liberated with him yet. Yeah. Okay. So he wrote some great papers that during COVID, for example, he used this approach and it ended up replicating exactly what the trial showed six months later. And the big push for doing this was because we needed timely results. At that point in time, it was we're never going to have someone doing this as a trial in time to know what we should be doing. So let's try to get some data first. And it was really powerful because then you had these trials come out not long after that were almost identical. Ed's done great work with Stop-Ace trial where you did this before the Stop-Ace trial came out and you showed very, very similar results to what was shown in the actual trial at all because when you can do your best to design it and try to overcome as much unmeasured confounding as possible by adjusting for what you do know, then you can really get results that that comes close to truth as possible. Exactly. So I remember the tosyllism app in COVID and I didn't believe right when Miguel had not study came out. I'm like this makes no sense. And then of course a few months later, the trial comes out which has very, very similar results. So that was an amazing way. Now going back to this form an example, you know, the your methodology is in this study is the clone, censoring and waiting aspects. I know you can't explain all the details to us, sterile in a podcast, but I hope you can you know dive in a little bit more. So one of the things that was to me is is trying to avoid informative censoring right because as we said if somebody's stopping metform and they're stopping it for a reason in an observational study here, you are like if they are assigned to continuing to stopping metform in but if the metform in is still being continued, you censor them at like 90 days or something like that is how this was done. You can you explain a little bit more about without going into the software or the coding or anything like that. How should we think conceptually? Sure. So conceptually, I think it's first important to realize that many observational studies are biased, that not because of the data or because of immediate confounding, but just by the way that the authors or researchers analyze the data in a way that's not correct. And one of the most important principles is that you need to align when people are eligible, when you assign the treatment and when you start follow up. And that's sort of a golden study design principle, whenever you do a study, every observational study and you are interested in the cause of effects of drugs or treatments, you need to adhere to that principle. And it's the same as happens in the randomized trial, right? You screen when they are eligible at that moment, you randomize them to arms and then you start your follow up and collecting outcomes, right? So those three components need to align. And if you look in the literature, this goes wrong many times. So there are some reviews that show that around 70% of observational studies violate this principle. And if you do that, you introduce either immortal time bias or another type of bias, called the completion of susceptible bias. And then, you know, it's just an incorrect way to analyze your data and then no matter what you do, your results will be biased. And what target trial emulation seeks to do is sort of adhere to the same design principles randomized trials and then align these components. But for that, you need sometimes different designs. So one design that you could use is clone sensor weight. And that's also important for people to realize that clone sensor weight is not target tribulation or if you do target tribulation, it's the same as doing clone sensor weight. No, it's just sort of a method that helps us to align these three components when you're doing your study. The point is that in this study, our treatment strategies are stop metformin within six months versus continuum at four men. And if you are at the moment when they're eligible, which is when their GFR first drops you low 30 for the first time, then you don't know in which group they are going to end up in because you could look in the data into the future. But whenever you look into the future to determine group assignments, then you will always introduce immortal time. So in data analysis, what you need to do is you sort of grow old with the patient. You are now at the moment of an EGFR below 30 and the patient could be in both groups because in the future, he could stop metformin or continuum metform, but right now we don't know. We are at the 30% EGFR. So then one thing you could do is randomly put the patient in one of the arms in your observation study. That's what the randomized trial also does, right? You randomized them. So you put it randomly in one of the arms. But because the patient's data are sort of consistent with both strategies, we can assign him to both strategies as well and then follow him for as long as his data are consistent with those strategies that you're interested in. And that is what cloning is all about. With cloning, you make two copies of the same patient and you put one patient in each of the arms and then you follow them until they are not adherent to their strategy anymore and that's when you send them. So you don't exclude them. You censor them at that time, right? It's not that you put them in the other arm. You just start collecting outcomes. Yes. And you stop their follow-up and you stop collecting outcomes. Yeah. So I'm looking at the Sankey diagram and there's a 5% to patients that die within six months. Doesn't this group have it a mortal time by? bias? No, why not? Yeah, that's a very good question, Joel. So a mortal time bias arises whenever you put patients into treatment arms based on future information. So for example, that would happen if we are at the EGFR 30, right? That's our baseline, that's our eligibility. And then what we do is we look into the future because we have all the data of the patients and we see, okay, in six months, the patient stops metformin. So that's why I put him in the stop within six months group. And if you use that future information, it means that per definition, he cannot die for six months, right? Because you have looked into the future, you know he lives for six months. So that gives an unfair survival advantage and then you introduce a mortal time. What clothing does is you immediately put him in all the groups that he is compatible with with all the treatment arms. And then if somebody dies before the six months, then that death will count in both arms. And in that way, you won't have any immortal time bias. And that's how you handled those early deaths in this study. Exactly, exactly. And this is quite complex, this whole procedure. And there are many other methods to ensure that you have alignment of eligibility treatment design then start off all up. So we just put a preprint online where we explain in depth this process. It's very useful for researchers who either review papers or who want to do this stuff themselves to know, you know, when do I need to use which methods and to ensure that you always adhere to this golden study design prints. So it's just another question. Your full cohort is 4200 patients. Is it larger than that because you have clones that are in both arms is actually the number of people analyzed larger or might not think about this correctly? Exactly. So let's say after all the eligibility criteria, age older than 18 years, no dialysis, etc. You ended with 4200 patients in your analysis set. What you then do is you clone everybody into two strategies. So you make two copies of the same patients. So you sort of have 4200 times two clones and each clone will appear in one of the arms. Gotcha. And I want to jump back very quickly to immortal time bias because this is something that comes up very often in the literature. And so I had some great work shouting from rooftops about this in COVID. We had a big problem with a lot of people publishing papers who were not typically doing this type of research or who thought that they were doing it but getting louder and getting higher platforms because they anything COVID for a few months was getting into New England Journal of Medicine. And there were several large high profile papers that came out that had a ton of moral time bias or measurable time bias, which is very similar, which is where people get credit for receiving an exposure, for like a hold duration of treatment, even though you have to survive long enough, to be able to even get the exposure. And so something that was really notorious for this was the ACE inhibitor A or B situation that came up during COVID, where a whole bunch of studies came out. One was in one of the, I think, circulation that we wrote a letter of a letter to the editor because we were so frustrated by like the magnitude to which it was obvious that immortal and a measurable time bias made these drugs look like they were helping these very septic patients where they were counting somebody as having gotten the drug whenever they got it during the hospitalization. And so you had to survive, basically your ICU state, then get out of the ICU and then get your ACE inhibitor A or B restarted. And then because you would survive that long and you had it restarted during your hospitalization, you got credit for getting it during your hospitalization and it looks like it was magic. And so this is a really, really common issue that comes up and people sort of can wave it out of existence like the shed eye mind trick because depending on how you describe your paper and describe what's happening and what your study question is, it can look like nothing was done incorrectly. It's like not obvious without having sort of a good background and really knowing how these should be designed. So it's great that ads being really loud and getting this out there. So people are aware of it. Your Jason review paper was fabulous at explaining this as well as a really common issue that happened in nephrology a lot in the past. But it's out there. It's super common and until you really get used to how to spot it, it's something that you can easily miss. Once you learn, you see that it's everywhere and it's very frustrating because it really can alter results a lot. Yeah. And as a reviewer, there's actually a very nice heck to spot immortal time bias. The sort of easy way is if the results are too good to be true, then it must be run. So whenever, no, but it's a very simple screening test. So whenever I see a pharmacopharmacal epidemiology study that looks into the cause of effects of medications and I see hazardous 0.20 or 30. I immediately know there's something wrong. And the magnitude is so large, it cannot be emerged confounding. It probably is a study design error like immortal time bias. So that's one trick I use. It's very easy for everybody to use. And the second trick is you can often look into the survival curve or the Kaplan Meyer curve. And for immortal time bias, you will often see that for one of the groups, it stays at 100% for a certain amount of time because of the immortal time. So these are two very simple tricks that you can use. Whenever I review a paper, it's very easy for me to use those two tricks. And this is one of the issues that comes up a lot with propensity score matching in particular because people are getting matched to the time point that doesn't match with when their actual exposure and index date should be and or eligibility date has been saying as a better term. And so it's it's a big issue. And that's part of why not all propensity score studies are bad and they can be really useful, but they can also be very misused. Exactly. And we often talk about confounding, but I think these things are a bigger issue, which are often not picked up. Right? As the examples that you've talked about at published papers where there were huge issues with, you know, lead time bias or immortal time bias and they are out there in the literature. So anything else we need to talk about yet or we can go on to the yeah one small comment. So observation studies have quite a bad name. And I think partly it is because many of these studies had biased results, but then they were compared to the trials in the trial showed completely different results. Right? And then observation studies get a bad name like they're always biased and people often assign it to immediate confounding. So one small other example I would like to make was on the ideal trial on timing of dialysis. So many studies in the 1990s showed that earlier dialysis initiation was better. Guidelines were then adjusted and you saw for example in the US in clinical practice that Egypt far became higher and higher when people start dialysis. And then the ideal trial came right in 2010, no difference. And people were again shocked right. All those observation studies are biased. And later on in 2021 we started to analyze also observational data and we wanted to to find out why were these observation studies so far off from the ideal trial. And we actually found that all these observation studies that have been published in those 20, 30 years they all suffered from these methodological biases because they did not align eligibility treatment and start a follow-up. So they introduced immortal time bias, lead time bias, and the completion of susceptible spires. So often observation studies get a bad name. So can you just be more specific that they started their mortality or they started measuring their effect at how did it happen in that trials to you? Yeah, yeah. So if you always need to think about if I do the randomized trial, what would it look like? So for example, we would include people when they're each for first drops below 20 so that would be their eligibility. Then we randomly assign them to groups and then we start a follow-up. So that's how you would do it in a randomized trial and when you do an observational study you would need to do it at the same time. So you include them when they're each for drops below 20. You don't know in which strategy they will end up in just like the the metformin paper we're talking about. You cannot look into the future and say okay that patient starts dialysis when it's e.g. for us 10. So I put them in the e.g. for 10 group because then you're again introducing immortal time so you can never look into the future. So whenever patients are included and are eligible, they can be in both strategies like the early and the late dialysis start. We don't know yet. So then again you need to use the cloning and then the clone sensor sort of weight method. And what these observational studies did incorrectly is some observational studies started the follow-up when dialysis started which is the incorrect moment. So then you have lead time bias but also the completion of susceptible bias. And if you start your follow-up at the correct moment when they first drop below 20, then many researchers looked into the future to put people into treatment groups and they introduce immortal time bias. And the same thing is to happen with survival benefit of transplantation. So they used to do these studies where they would like look at the time of transplant versus like Simon and I else. And you can't do that because you need to just look at the time of when you would have been eligible to be waitlisted for your transplant and then follow people forward or when you would have been eligible to get the transplant at least. And so there's this whole notorious literature of that area as well that the benefit of transplantation isn't quite as big as it was thought to be. And part of it is because of that part of it is also that there's other issues. You have immediate post operative mortality risk that then lessens over time. But the bigger issue was that there was a lot of immortal time bias being introduced because of your time zero not actually matching a true eligibility and you're looking forward into the future. People who ended up actually getting transplanted when many people never even got that far. And so in that situation, Jordy, you'd want to start the time at the when they get listed. What you'd do is get less data when they're active on the wait list. Like when they're first activated on the wait list even that's fine too. But you don't want to do it when people are just never even eligible to get a transplant offer. Gotcha. Do you think can you explain a little bit more about the depletion of the screen? as acceptable bias and that bias had that role in this one. But yeah, so the completion of susceptible bias is actually a form of selection bias. And for the non-appy people selection bias is not the same as confounding confounding is when two groups sort of are different, right? One group is healthier than the other and then it sort of obscures the true cause effect of a treatment and selection bias, for example, occurs when people are lost to follow up. I think it's best to explain this if we think about randomized trials again. So consider we do a huge randomized trial of s-sh-t2 inhibitors. One million people in each arm, right, are randomized to start s-sh-t2 inhibitors and one million people are assigned to the placebo group. And in each way, this trial's perfect. So there is no loss to follow up. Adherence is perfect. It's perfect trial. So what you do in this trial is that, of course, you would count all the outcomes from the moment they're randomized and then compare the groups, right? That's how everybody would analyze this data. What would be an incorrect way to analyze the data is if you throw away the first two years of follow up and then look who has survived for two years and then you set your new start of follow up at two years, right? That would be a very weird and foolish way to analyze these data. Nobody would do that. But that's actually how people analyze their observational data. They don't start their follow up at the moment when people get the treatment, but they start when they have already used it for two years. And when you do that, you get this depletion of susceptibles. And to understand how that works, OK, we know s-t2 inhibitors are protective, right? So if treatment is truly protective, then fewer high-risk patients will die in the s-t2 arm, because the high-risk patients with a lot of risk factors they are protected by the s-t2 inhibitor. On the other hand, you have the placebo arm. And there you will see that a lot of the high-risk patients will die in these two years, because they have a lot of risk factors. They are at high risk of mortality and they die. And they don't get s-t2 inhibitors to protect them. So at two years of follow up, you will see that there will be more high-risk patients left in the s-t2 inhibitor arm, because they have all died in the placebo arm. So at randomization, you had that these arms were similar. You had the same proportion of high-risk patients, but actually after two years, you will see that s-t2 inhibitor arm is sicker. And this is this so-called depletion of susceptible patients are depleted from your data set. And it's actually a form of selection bias. And it occurs whenever you start your follow up at the incorrect moment after treatment has been assigned. So sorry, if we then analyze as those groups are, then maybe the benefit of a jelly-to-nibedus may look at the new age, because the benefit has already happened in the first two years. But then if you also do a propensity score matching at that time, which you should not. But I'm talking about doing really bad studies. At two years, then I may throw out this susceptible people, because there's no one to match with them in the placebo group. They have all died. So then again, you are left with non-s susceptible people in a jelly-to-nibed placebo. And again, you will show no effect, because you had to throw out all the people who were alive because of a flow zone in the first two years. Is that right? Or is that a different way of thinking about this? No, what you're saying is correct. So all the susceptible patients have died in the placebo arm. So then the effect or the association, because you're not looking at calls of effects anymore, it's biased. So the association of s-yltitude inhibitors with mortality will be attenuated. So for example, in truth, it's 0.80, it has ratio. But now it may be even 0.95 or 1.00. Or if it gets really worse, s-yltitude inhibitors can look harmful. It could go to a hazard rate of 1.20. And propensity score matching is not going to solve this. This is a study design error. You could do propensity score matching, but if you would want to correct for this depletion of susceptible bias, you would need to adjust for all risk factors for death. And literally all risk factors for death. So that means smoking, how many yoga somebody does, how many vegetables they eat, their genes, et cetera. You don't have that in your data. So there is no way to adjust for it. So the best way to do it is to do a good study, target triangulation, and do a good study design. I just want to get more specific, though. So this requires-- and then Joel, I'll let you go in a second. I'm sorry-- but to get more specific, so you don't need to do it at did, where they required it within a year beforehand, and they knew everyone's prior follow-up period. Or you do something called a new user now. It's worth the first time that people are getting exposed to the drug after you've been able to follow them for at least a while, at least a year or two years, if not their lifetime, to know that they've never had this drug before. And then you know that you haven't depleted your sensestibles and that you're starting with people who are all fresh. It's also a reason that we often do active comparators. So the clone sensor waiting is really important for this type of study design because you're comparing, stopping something versus staying on something. And you don't have an active comparison group. So you really need this design. It's much more powerful here. Whereas in a lot of these other studies, you're usually lining people up where they're all starting a new drug at the same time to help with that time zero assignment too. Yeah. And then in your grand-round, Zadada, what you did, you used this same explanation to talk about the obesity paradox in diabetes, which I thought was excellent, that you cannot get the exposure to obesity when you start looking at them at the onset of dialysis. That these patients have had obesity for many years, probably decades before they started dialysis, probably have depletion of susceptible patients prior to starting dialysis, which I thought was a-- for me at least was a novel explanation for something that we've been talking about. I've been talking about my entire nephrology career. I love that explanation. And I thought it was quite compelling. But I think we should be getting to results because it's been an hour of methods. So just to wrap up, the authors also did some sensitivity analysis using marginal structural models and they did some covariate balance assessment, which is again looking at the standard ways of looking at confounding with a threshold of 10% indicating imbalance. But really, the main meat of the methods was in everything that we have discussed so far, the target trial image. So I think we should-- So one last question, terms of how sophisticated target trial emulation methods, where would you rate this? Again, there are many papers out there which call themselves TTE. And they use simpler or more complicated methods. I know your BMG IDL analysis was-- the initiation of dialysis with different GFR thresholds was extremely complicated at least for us to understand. Would this be a simpler analysis than that? So actually, in this metformin paper, we use the same statistical methodology, so also the clone-center weight method. But once again, I think it's good for the audience to realize that the target emulation, it's just a framework to design and analyze your study. And many methods fall in it. If you, for example, compare two drugs with each other, like s-chial-detune, hepatitis, drs, dioperm receptor agonists, then you need a whole different suite of methods. You do a head-to-head comparison, so you use an active compare-turn use design as Jordi already alluded to. That could also be a target trial emulation study. And probably there, the methods are and the design is much simpler. Point is that it's important to explicitly specify the target trial you're interested in, the hypothetical trial. And then you need to use the right methods and the right design to answer exactly that question. Sounds good. I think we can talk about some results quickly. But I think the meat of this podcast was really the methods. I totally agree. And so I think the results are fit with mine and swapnoles priors, so that's pretty cool. And so overall, there were 371,742 Scottish individuals of type 2 diabetes before April 30, 2019. On them, about 4.5% ended up reaching chronic kidney disease stage 4, of whom we had about 4,200. So about a quarter of them were using a metformin in the year prior to their CQD stage 4, meeting the inclusion criteria for the study. The cohort ended up being followed for a median of 2.5 years. And people on average were age 77. The people who were using metformin in that first six-month period that infantess about 40% of the cohort who stopped it. So there were about 40% of individuals who stopped metformin in that first six-month period. And of those-- of the individuals who continued taking metformin, there were 54% of them who continued taking it during the long follow-up period. And 5% who discontinued their metformin use later on in the follow-up period and 44% stayed on it long term. And so looking at overall risk based off of those who continued versus did not continue the metformin, the author's found that there was a better three-year survival than the people who continued metformin in that six-month follow-up window using that clone sense or waiting approach. Compared to those who stopped it within that six-month window of reaching chronic kidney disease stage four, the actual hazard ratio that you can look at is 1.23 for those who like hazard ratio. So 23% relatively higher risk of ending up with all caused death in individuals who stopped their metformin versus those who continued it. The authors didn't find a substantial difference in major adverse cardiac events between the groups. So this is primarily death-related. Because there's a 1.8. There's a 23% difference in total mortality and no difference in cardiac cardiovascular disease. Correct. I think I think I should nuance this because when I look at the mental figure S5 where we show the cumulative incidence curves of major ethyl cardiovascular events, you actually see that there is a higher risk for most of the discontinuation, which is very consistent if you compare it with figure 2 with ocos mortality. And it's too bad that in the results section we don't show the absolute risks and the risk ratio. But I think it's pretty consistent with the ocos mortality effect. And just the strength, because of the smaller number of deaths, you just didn't get significant there. Is that what your thought is? Actually, sometimes what can also happen with hazard ratios, right, if the curves cross, and of course they do in the first six months because the strategies are a little bit the same in the first six months, then you see that there is also non-proportional hazards and that can attenuate the hazard ratio. Another disadvantage of the hazard ratio is that it has a built-in selection bias and that's why if you often look at targeture amylation studies, you see that there are more often reports of absolute risks and risk ratios. So I think that may be playing here as well. Yes, so what made you guys report the hazard ratio? Good question. Maybe the journal asked for it. Yeah, we were too asked for it. No, I'm guessing. So I mean, having lived this, we were, we struggled like back 10, 15 years ago trying to get these papers published. Journals were really, really, really scared of them. And so there was a lot of pushback in terms of like the presentation and how you express the data. So it's really an interesting point that I think that some reviewers are still getting comfortable with this. And usually it's a risk difference or something that lets you look at more absolute differences that gets reported with these target trial amylations if you look another journal. I don't know if it was your review or request. That's why I was curious or if it was just perhaps a choice for a limited word count. But I thought it was interesting. So thanks for sharing that and highlighting that ed. And that's a really, really, really helpful point. And so on an absolute scale, the cardiac, cardiac effect where we're still likely quite similar to what we see or consistent with what we see with mortality, but don't show up as a statistically significant from a relative scale. And then in terms of other outcomes, there was also higher risk of respiratory death. So just, I guess my question is, did you guys look at all these different causes of death and respiratory ended up being a signal? Or did you guys go in thinking, hey, maybe respiratory deaths are going to be something that we should be, I noticed it's not part of the, when you define your outcomes, respiratory death was not one of the pre-specified outcomes. I think it was also one of the editorial or review requests at HAKD, because it's interesting. Oh, no, one. Secondary outcomes were major adverse pattern events, three-year risk of cancer and respiratory diseases cause of their death. Cari, I'm apologize for that. Yeah. Go on. Yeah, but we also wanted to investigate if there is an increase, if there is a signal for mortality, but not for cardiovascular events, then it must be driven by something else. Right? If you're looking into the data, could it be due to cancer, could it be due to respiratory deaths to see what's driving sort of the difference? And then, of course, the question becomes why is there a difference in respiratory death? Right? And there are actually two explanations for that. One, there is a true biological mechanism, why metformin prevents, let's say, pneumonia and stuff like that. Or it could be that it's confounding. A machine confounding that there is no true cause-affect of metformin on respiratory deaths. So what we're picking up is people who are sicker, discontinued treatment that are sicker, and that's why we see an effect for respiratory deaths. And then the marginal structural model results, which were these results that looked at the time-updated exposure to metformin over a longer window, accounting for time-updated factors that could contribute to changes or discontinuation of metformin. And that one showed remarkably similar results. The hazard ratio was 1.34 for all cause mortality. And the Hong Kong data, this is the Yang study in E-clinical medicine, they also found a respiratory signal, is that right? Yes, I think we're referencing the end of the discussion. Yeah. Interesting. So, you have some precedents at least, but they also found a cardiovascular signal. But so did the study, the study, it just wasn't on the relative scale, like we were saying. Well, we're going to judge a study by their own methods, right? They determined how they were going to present the results. I mean, come on now. That's a very good point, Joe. And I think we should always report the absolute risks and risk ratios and risk difference. And we should have done this for May's, this secondary end point as well. Okay, excellent. Now, I was saying that the nice thing was all these other plots that they had. I mean, we've done them out from the supplement, but you know, the Sankey plot, the love plot, it was really nice. I think that visually they do make some intuitive sense. I'd not seen a love plot before. I don't know why they are not reported more commonly. I get this shows standard differences in tables, but to see that in a plot where, you know, this is the, I didn't go the figure as as for where you can see the unweighted and the weighted change dramatically. So, you can still see for HB even see an insulin. You know, there are some, it's not like zero, but it's, it's pretty close to zero, but it's visually, you know, it's a really nice way of showing the effect of what you're doing. Yeah, I had not seen that before. Yeah. Yeah, it's very cool. Yeah, they're, they're really commonly buried in the supplement, Joel. That's why you haven't seen them before. I take the slings in the aerosol. This is my purpose. This is why I'm here. This is something we tend to expect in Tunger trial emulation and anything that's using any type of these inverse probability treatment weights or other types of propensity weights. You usually have some sort of balanced plot. This one I agree is really elegant. Okay, adverse fence, any lactic acidosis in this study? Did you guys, was this something to pre, do you guys look for this in your study? No. So, we did not have data on lactic acidosis. We didn't have the lab data to look at it, but it's a very important outcome. Yeah. And it's, again, if it's 3,000, right? You, you're unlikely to get it. You pick anything up here. Exactly. Right. For sure. Yeah. Four thousand patients times three years, just 12,000, yeah, you're not even close. Okay. One case you wouldn't even know a charm. It might be. Exactly. Okay. And I think this was actually, this was, this was studied in the, in the Hong Kong trial. The target trial emulation. It, the Hong Kong target trial emulation. Thank you. This is something else that happens is some people do present these as trial. And I don't know how much of it could be due to potentially language barriers or due to intentionally trying to have things be presented as trials, but this is something that I've encountered in reviewing papers that it's not clear that it was a target trial emulation and it's being presented almost like a trial. So it's part of the importance of target trial emulation is how it's presented. So at, at, at, at work, foods groups does a fabulous job of presenting them and explaining what they're doing and writing their methods out and making it very, very clear what this is. But it's not always. So I think there are some bad actors out there. So that's something else to keep an eye out for. Yeah. Sometimes it's, it's the framework and that, that's with all things that are new and flashy. I truly think that target emulation is an improvement a, a bit some people sort of, yeah, misuse it and, and make their study look like a randomized trial, but it's just an observational study, hopefully with good methods. Yeah. So in the Hong Kong trial, they found a 3.5% risk of lactic acidosis in the people that discontinued metformin 2.6%, a 50% lower risk in people that continued metformin. And, and that was not a significant difference. Yeah. But, but to go back to, you know, Ed and Jody's point earlier about this, and I think I asked this question to you before as well. Ed, is that this, this seems to be the, you know, the latest toy for people to use. You know, I remember, I still remember, I remember, I remember, you know, logistic regression was like, you know, you just do that and you're done, right? There's no bias anymore. Everything is cool. And then we had propensity score matching and that was the best way to do these studies. And now we have our TTE. Are there good ways of doing TTE and are there bad ways of doing TTE? Like, if I just have, you know, some software or package, which is, you know, what happens with, you know, anyone given SPSS and a data set, is that a risk that our, our STE is so complicated that it's hard to, to do it wrong. And as a reviewer and a reader, how do I, how do I, you know, make sense of this? Yeah. Because it's such a black box, right? I know, I look at the author list and, you know, I see Jody, I read when I'm like, okay, I'll make it happen. And I say, okay, this is fine. But that's not a good, good way to assess this, right? Is there a better way? Yeah. So target triangulation, sometimes people call it the target triangulation, but calling it target triangulation doesn't make it one. I've seen a lot of papers, which have also reviewed that say that they use target triangulation that when I look in their methods to use a wrong design, they introduce immortal time bias, right? That's, that's pretty odd. Target triangulation is also more than just putting a, a, a target trial table, right? Where you specify your hypothetical target trial in your paper and then saying you have done target triangulation. I would really enforce these steps, specifying the target trial, but then also making sure that your design is correct, that the methods are correct, such that you really emulate that target trial in your observational data. It can be very simple. For example, if you do a head-to-head comparison between two drugs, the design is pretty straightforward, active comparison in your design. You can use perpensical score weighting to make the group similar at baseline. You look at an attention to treat effect, but it can also be. become quite complex. I would say clone Sanserweight is a pretty complex design. As a reader, I think one of the most important principles, and I think this is the sixth time I mentioned this, just check. Do the authors correctly align eligibility, treatment assignment, and start a follow-up. That's the only thing, or that's one of the most important things that you need to check as a reviewer. I agree 100%. One thing I want to add to that, though, is that because this is becoming so popular, and there are a lot of examples of how to do it right, people are using the wording, and saying, like using that quote, it looks like a Miguel Hernandez sort of thing out of a JAMA editorial he had written, and I've like, it's so important to align your index time with your, when patients are actually eligible to be in it, and when you're actually getting your treatment exposure, and using the identifying correct index time, this type of wording comes up a lot in people writing target trial emulation papers, and then you can then proceed to read on and see where they violate it. And so they're sending their quoting it, and describing that they're adhering to target trial emulation very, very, very explicitly, but then don't. And so I do think it's something to be cautious of, like be mindful of that actor, if you're not somebody who is having like a clear time reading the method section, and it's like not explained in a way that makes sense to that also is a red flag, because people should be able to articulate it in the way that Ed just did, where they can explain exactly what they did in the way that may not, like, seem like you can go and create your own right now, but at least you feel like you have a sense of what was done. And if it's written in a way where you don't feel like you can add up what was actually done, then that's a problem. And so I really do think that it's being misused and being misclaimed in terms of its abilities to overcome issues when it's not handled correctly. And then there are lots of people using it for great things. And I love all the questions your group has been answering at. I'm such a fan girl of all the work you guys have been doing because you've been really-- Thank you so much. It's exactly what target trial emulation was designed for what you guys have been doing. It's the situations, this is men for scenarios where we either it's either unethical to have a trial. So I gave that example of transplantation. You're never going to randomize people getting a transplant or a cystic on dialysis. But other examples of that happen a lot in the neurology, these later stage, neurology patients who are close to dialysis, we're never going to be randomized into a trial for these things that people still debate about or aren't sure about. That this is something where we desperately need these studies. The stop-aces is another great example, where even the randomized trial to exist, but because it's so hard to recruit these patients, they're often underpowered to answer some of the bigger questions. And so you need the larger data sets that can help us answer all the questions we have. And so I think that it's incredibly powerful for those scenarios. I think it's also really powerful for scenarios where we want to convince people to fund the trial. So I use these a lot to try to convince people to pay for me to then do the actual trial, saying, look, there's a signal, but they're still going to be on measure compounding because there's no way we can overcome that for the study question. So can we please randomize patients? And so I think that this is taking, as you said, Edward, taking the observational studies to a higher bar, increasing our expectations of people to design them in a way where we can hopefully get closer to trusting the results, but knowing that obviously there can still be limitations that we can't address, but these get much, much closer to reality. Yeah, I think that's a pretty good summary of what target domination is all about. Thanks, Tony. So an area of nephrology that I am super interested in is contrast associated in nephropathy. The data, at least for venous contrast is all retrospective and observational. A lot of density matching, but I don't remember seeing any target trial emulation. And I think about how these studies have done. It doesn't seem like you would gain much, right? Because the exposure and the measurement happens simultaneously. We know exactly when they get the contrast or get the CT scan without contrast. And that's when we start measuring. Do you think that this method would be helpful for studying this concept? I have two thoughts. The first thought is if you want to study it with observational data, then first imagine what the randomized trial would look like, right? Yeah. So how would you design the trial, Joel, which patient would you recruit and to weld arms, which you randomized them? I would. And this is, sorry, this is where the value of thinking about it. This is where the selection bias comes in, right? Let's see, you're looking at GFR of less than 30. Yeah. You know, Jardy and I don't believe contrast nephrology exists. So we'll be happy and randomized. I'm happy to have this on the same story. Come on, man. I know, but 10 years ago, Joel, no, my God. I'm going to get hung on this for 10 years ago. For the rest of my career. No, but that's a difference. By the way, it is, it is, there are some people who will include these patients and some people who will not, just like them at farming questions. So, so I think that aspect is there. But if we would do the randomized trial, right, we would include people with the GFR less than 30, 30, yeah, with an indication to get a CT scan. Yeah, right? Yeah. We could think about which indications or to all indications, but that's up to the trial list. And then some would be randomized to get the contrast. Some would be randomized to not get contrast. Yeah. And what outcome would you look at? You could look at, I don't know, GFR after 30 days or AKI during an X amount years of major adverse kidney events kind of as defined by a for example, yeah. And the beauty of using one of these EHRs from Europe would be that you can or from Canada would be you could follow people longer. And see, it's not so much the AKI that we care as much about. It's the what is your kidney function, 30 days out. What is your kidney function? Two months out, three months out when it actually matters. And so that I think could be a real power of using retrospective data for these types of questions and it can be done. Technically, you could emulate such try and observational data, right? You could run a randomized trial, but you could also do it's an observational data. I think for this question, the great risk is the measured confounding as Swap says. So there is also another design which is very interesting. Maybe you've ever heard of it. It's called regression discontinuity. I was going to mention that there is a study where they took patients with GFR of like 28 to 30 versus 30 to 32 because up on 30 is a Qatar. Exactly. When you have hard cutels in medicine, you can use it to your advantage as a sort of natural experiment. So randomization by by actually the doctors, right? Because they are so afraid to get this contrast induced AKI. And then you see that below 30, nobody or very small percentage gets the contrast. And above 30, you're 30 to 32, then a very large proportion. And there is such small difference in GFR and we'll know, right? EGFR is very imprecise. So probably there's no difference within these two groups. There is no confounding and it's like like it is already randomized. So the cool thing you could do, Tuzul, is go back to the people who did the study and ask them to rerun the study, looking at something called negative control outcomes, where you have them also look at a whole bunch of things that would not be at all related to them getting that IV contrast. And this is a cool tool that's used in a lot of target trial simulations when you're worried about residual confounders or biases, where you can then look at other outcomes. And if the other outcomes are as you would expect negative, but this finding is and this finding is too, then it helps you sort of increase your confidence. Or if you're finding as positive and those are negative, as would be expected, it's helpful. So it's something that can help increase confidence in whether or not we actually believe the finding. Okay, thank you. Appreciate that. Okay, is there anybody have any other thoughts on this study that we want to talk about? Yes, I have a question for you. Are you going to stop using? Are you start? Are you going to not stop metformin now? Are you convinced? Again, I mean, the medical legal aspect is separate, but let's say there are no medical legal concerns. Are you convinced with continuing metformin? Yeah, no, I've been convinced for a while, mainly because of the patients that I've seen with mala have all had other cause like they had a more prox, none of them have not of them have developed mala because they've had progressive CKD and a slowly slowly progression and presumably an increased exposure to metformin. It's always been they developed acute kidney injury or they had some other catastrophe that they underwent. And so it has for a while, I've had I've not been dogmatic about stopping the drug though it does break some other questions is that one of the other recommendations is we're exploring the dose and giving these patients lower and lower doses. Are you guys doing that or you you know, Jordi, are you turning the are you still giving them a thousand BID? No, no, I've lowered the dose from the start because I mean doing something that's off label you always are going to be a big gunshot. So no, that that was always the case. We always lowered the dose to kept them on it as long as the primary care condition was okay with deferring to you on that. And I've had a couple that said no, I don't feel comfortable with this and we stopped it. And then it feels to the conversation. The conversation that I hate to have that I hate to imagine is the patient rolling into the family practice office and they're like what what is your nephrologist doing? This is crazy that you're on the drug. See it's right here in the PDR that you're not supposed to use it in this situation. This this doctor is clearly an idiot. Like I know it's something that I worry about because there's this authority, deferral to authority that is tough to you know tough to overcome. Yeah, exactly, that's true. But the things where you really hate stopping metform and at GFR cutoff is then you know, their sugar control is kind of borderline there iffy and you know by stopping metform and you're going to introduce you know insulin or a glyptin or what have you? or something useless. So of course that is less of a concern now because you have. - Well, at this GFR, the flow since really don't have a lot of A1C effect, right? - Yeah, exactly right. Exactly right, yeah. They still may have cardio protective or kidney protective, but. - Let's refresh that. - That makes sense. - They do. - They do. - They do need no protective and cardio protective. - Do you? - But the glycemic effect is very minimal. - Yeah, exactly. - Yeah, so, but hopefully, you know, and how easy is it to. Now the medical legal aspect, how easy is it for you to go off-label? Clearly, Jordi's doing it and it's not hard. - Yeah, it's not hard. - It's not hard. I always would document. The key is document a lot because you pretty much write your notes often here thinking what would happen if I was in a court of law and in a lawsuit and can I defend my actions, which is unfortunate. But yeah, so I often just rationalize it. Usually it was a weight-related issue where the patient had struggled a great deal with weight and we finally got them to a goal or they've seen improvements on the metformin and then adding insulin, just going to be. Just like horrific effect on everything. And these were people who typically didn't have very, very high A1Cs. So really, do you want to be adding insulin? Or like he said, a glyctin, which is not going to be very effective. And so these were the patients. I do think now it'll be interesting, since GLP1 agonists are increasingly available, that that could be the alternative. And that's a much less scary alternative in some ways than insulin and a glyctin. Since you'll see that A1C benefit and the weight lowering benefit. But often if they're not already on a GLP1 agonist and they're this patient and that scenario, it may not be a good fit or they may not have tolerated it also. So I think that that's not going to be a magic bullet. And in that sense, this study is useful. Now we have even more scientific data supporting that, not only that it is not harmful, but there is a benefit in continuing our forehead. This PMID is going to be going in every single one of my notes now. I'm nice. Excellent. And thank you for this. We're going to go on to tubular secretions. This is part of the podcast we talk about. Other things that we're doing outside of Neffro world. Let's start with Swap. What do you got? So I've talked about a murder board before. The murder board is a book series, novelize. And a novel written by Martha Wells. But finally, the TV series has dropped. It's an Apple TV. It is interesting. It is different. It has Alex Skarskard as the murder board. But it's a lot of fun. I think it's going to be very different than the books in many respects. I think there are some aspects that are better than others. But it's fun. If you ever read the books, I would strongly recommend you read the books. But after you read the books, the TV series is fun. I saw the first two episodes. We liked it. Jordy, what do you got? So I'm not going to use the newer Star Wars stuff that's been coming out all over the place. But Andorr, I just think is so cool, because it's all about the character building and the world building in a way that other Star Wars never got to do. And that I think we dreamed of when we were kids first diving into the Star Wars world and saw him in season two of Andorr. And I have not finished it yet. I think the final episode just came out. And I am just in love with the depth of the characters and everything that they're doing in terms of making you really think carefully about what's happening with the politics of the network as well. Looks amazing, real world parallels. It's mind-blowing. It's one of the best TV series of all time. Yeah, you had in photo. It's very good. Very, very good. It is better than very good. Come on. It's awesome. And what do you got? I actually don't have a lot going on. I work in hard on the grand. So there's much free time. [LAUGHTER] Two are ones going in this week and another are one that I'm a co-investigator on. I feel your pain. Brittle. Brittle. Brittle. We did buy a house last week. So that's a good-- No, where do you live now? Where do you live? In light in the Netherlands. Yeah. In the Netherlands. Oh, very good. So I got two. So one, I saw Thunderbolts. And I tell you, I've been really burned out on the Marvel movies ever since Endgame and hadn't really enjoyed much that I'd seen since then, including the television shows. I thought it was all pretty weak. Thunderbolts works. They have finally gotten it done right. It's a-- the writing is excellent. The story is good. The visuals are perfect. It was a-- it was a joy. It brings back kind of old classic Marvel. So if you've been away from the franchise for a while, I'd recommend coming back for Thunderbolts with an asterisk at the end. Quite good. Yes. And then I watched the Three Body Problem on Netflix, which I liked. But I finally read the book. And it was light-thulpe. It's so good. It's so good. And so absolutely loved it. I bought the second in the series. I haven't started that yet, but I've really, really enjoyed a Three Body Problem. Three Body Problem by Lou. She's in Lou. She's in-- she's in Lou. She's in Lou? Yeah. Very good. Well, do you know the history behind that? How the first chapter was published in a different order in China? Tell me more. Yeah, yeah. Because it's gotten the public-- people's the public of China stuff. Yeah, they go ahead. Yeah, no. So they knew it wouldn't get published in China if that was right up in front and center. And so it was buried really, really deep in the book. So it's a different order that you read the book. And if it's in original Chinese-- The book opens up with the-- And the book opens up during the Cultural Revolution. And it's a pretty hard read about innocent academics getting murdered, essentially. It's pretty bad. And so I can see that. And for a guy like me, I just didn't-- I didn't understand. I'd heard the word Cultural Revolution. I didn't quite understand how revolutionary it was. Actually, just-- I saw-- there was a tweet today. And I'll put a link in the show notes looking at Life Span by David. And you look at the Life Span in China during the Cultural Revolution. And it just-- it falls way down like they killed lots and lots of people at young ages. And it affected total life span, which is bananas. So I highly recommend a three-body problem. But it's about-- it's not just a cultural revolution, right, for readers. It's about-- Oh, that's just the first chapter. Yeah, it's about-- it's really hard-- it's the old-fashioned hard science fiction. There are really, really hard science cool, really cool concepts within this story. Cool concepts. Yeah. Super cool concepts. Really interesting thought about alien invasion, right? I mean, how many alien invasion stories have we read? And this one provides-- This is probably the most realistic, yeah, in some respects. It's pretty good. It's a pretty good analysis of it. And a lot of new-- as Swaps says, a lot of new cool ideas are played within this. Hey, guys. Thank you for joining us. This is a great podcast. I'm super glad that we had Ed here to be able to have him. [MUSIC PLAYING] , "When things they go wrong, your kidneys, they stay strong, ready to take the lead." No matter the problem, you don't have to worry. Your kidneys know just what you need. [MUSIC PLAYING] "Millions of nephrons working together beautifully, constantly." [MUSIC PLAYING] "Infatuation with glimmerular filtration, I just might shed a tear." "Flozen, prescribing, and not dializing." "Isn't that why we're all here?" [MUSIC PLAYING]