The Causal Gap: Truly Responsible AI Needs to Understand the Consequences | Zhijing Jin S2E7

So really, causality has a hope to make these artificial intelligence systems to be more responsible for the decision they are making. Personally, I'm also very concerned about two topics. One is education, like education of our future generations. The other is how information is spread, especially if it's relevant to next round of elections or public policy making. I think their language models might have sort of unseen, unpredictable, large consequences that might affect our society in a really negative way. We have recently finished a work called a causal AI scientist that basically grabs us a set of formal causal reasoning techniques from do calculus to other statistical methods like difference in differences, instrumental variables and so on. There, we are taking this more symbolic-leaning approach to let language models only hand those simple routing things, such as check whether there is a valid instrumental variable and then let it generate code or call an existing piece of code or instrumental variables to get the causal effect. Another line of my multi-agent work looks into AI safety problems, so we put multiple agents in simulation. One of our 2024 Newerb's paper, Gobson, governing the comments simulation, basically looks into if we have a digital fishing village and initiate every large language model as a fishermen character. So we simulated a whole calendar year where each month there's an iteration of fishing and most models didn't survive until all the 12 iterations. Hey, causal bandits. Welcome to the second season of the causal bandits podcast. The best podcast on causality and AI on the internet. Math didn't seem practical enough to her, so she decided to pursue a career in computer science. She's interested in language, ethics and causality and they're interplay in multi-agent systems. She grew up in Shanghai, and now she prepares to open her new research lab in Toronto. Ladies and gentlemen, please welcome Dr. Xi Jinping. Jin, let me pass it to your host, Alex Mola. Hi, Xi Jinping. Welcome to the podcast. Hi, Alex. Thanks for inviting me. I'm really, I'm really glad you found time to talk to us. I think there are so many topics in your research that will be very, very interesting to our audience. I wanted to start with a topic that is very hot today, something related to large language models and agentic systems. We had a few guests before in the show who work at this intersection of causality and large language models and agentic systems and sometimes reinforcement learning. Like we had Andy Rulampinan from Google DeepMind, Amit Sharma from Microsoft Research, and this intersection is also central to your research interests. I was wondering, where are your thoughts on the ethical implications of using large language model-based systems, what we call agentic systems, today in real world scenarios? And how do you see causality playing in this context? Yeah, thanks for the great question. I think nowadays there are a lot of discussions on how to develop AI responsibly or the rising safety trend as well. To me, specifically, I feel like many of us develop these models with the good will to make life easier, more efficient, more productive. And then there are side consequences whenever the agents have non-perfect accuracy than for these error cases who bear the responsibility. Or on the other hand, like even if they have perfect accuracy, we want agents to do the decision or not. Yeah, and then bring this forward, I think the role of causality can be helped to reduce inaccurate decisions in the case of, for example, if the HR, like a human resource screening, is replaced more and more by models, then do they use correlational features, which can be gender, race, and so on, or causal features that are the skill sets that's relevant to do the job well. And then, I guess, on the other hand of like weather at all, humans should use AI to replace their decision and grant them agency. I guess then there's a mixture of ethical discussions, legal discussions, and maybe a type of meta-cosality in terms of if the system is deployed, who will be affected, and sort of a bit like a consequence reasoning over the society. When you think about this ethical implications, from the perspective of a person who researched the last language models in terms of causal reasoning and formal reasoning, what would be two or three things that you think practitioners that are implementing these models in their respective fields might not be thinking about, but these things might have serious consequences for them or important consequences for them. Very good point. So I think potentially, the HR example is commonly mentioned thing where the models can make a round decision to generalize it to large language model use cases, there could also be models can write malicious code, and given that more and more programmers are relying on GenGBT and other LRMs to handle their programs, there could be systematic loopholes that's done in important systems. And on the other hand, it could be about when we are trying to, because I've been in touch with people working on different things, personally, I'm also very concerned about two topics. How could we deal with that? That's a big question. I think for technology, there are always two ways. One is to regulate first before the technology is introduced, and the other is let it make a splash in the society, and I have a feedback loop around it. For my understanding, I think on the education side, since many of us, like the school side and the education system side, are trying their best to put regulations. When the end, it might be too hard to ban students from using LRMs to assist their homework, 'cause there are projects and many others. So on that side, I'm more a believer of the feedback loop, of let this happen, and then wrap the education best around this new change. Whereas on the misinformation and social stability side, I'm a bit inclined with, for example, how Europe enforce regulations on social media a while ago, and there should be regulations on how it should be used, and how it is consistent with universal values that at least the local system are endorsing. In one of your papers, one of the papers you've offered called C-Ladder or Clather, you focus on benchmarking clutch language models in terms of causal, in terms of causal reasoning. Just this week, we had in our newsletter "Cosal Python Weekly," we had a piece from David Roder from Criteo Research, where he talks about imagined intelligence versus direct optimization. And he says, "There are basically two paths of thinking when it comes to "intelligent systems," right? One is that we accumulate some kind of complexity, and then we hope that something useful will image at the end. Maybe we don't understand the mechanics, we rather look at the mechanics from the complexity point of view or K-O-Sphere point of view. And then the second path is looking at a system understanding the mechanics and trying to build something, something useful, like a useful estimator of causal effects, for instance, and so on. And David concludes in his piece that he doesn't, in his current view, he doesn't see these things as compatible, he doesn't know how to combine this imagined and this direct or systematic ways of thinking together. In the clatter paper, you are in a sense trying to build a common ground for these two perspectives. That's at least my understanding. I don't know if you would agree with that. What was your experience in this regard? And what are your thoughts about these two paths? Is there a common ground? Are we moving towards building a common ground for these two approaches? Or rather, they will remain in a sense separate in the foreseeable future? - Yeah, I think this is a very important question to ponder for all the researchers pursuing, call the reasoning or other reasoning capabilities of language models. So translating the framework in the context of clatter formal causal reasoning, I guess, what is meant by direct design is more like give language model more narrow prompts, tell them exactly that we're estimating average treatment effect 80 and look for these confounders and then please consider the following adjustment methods and so on. I think that's definitely one way to make some under control and reliable. It might be a good way for run two causality in that do calculus with three rules is complete and sound solution, which is also one of our following work covering. Whereas on the other side, people do have this, like in the field of machine learning, people do hope to update the parameters of the neural network so that the network itself can handle causal reasoning in and to end manner. And with that, I guess if we look at two types of stages for large language models, the first stage is pure pre-training and just sequence completion as a signal. Then with that, we have at least in the clatter paper and maybe also follow up studies observe a certain cap of accuracy for models on the benchmark. However, recently people have been actively thinking about how to fill up the rest of the gap between the current performance and perfect performance on color reasoning and other reasoning tasks. So potentially adding reinforcement learning and other techniques would be helpful. Then we might see a potential middle point happening because to make reinforcement learning really work, we need dense enough reward signals. However, how do that come from? It could come from that there is an expert system implementing certain steps of causal reasoning really correctly. And then we use that to both to generate numerous data for the model and to have step-by-step reward signals. So the reinforcement learning system can be more directed towards the desired reasoning. - That's very interesting. It reminds me of a debate we had in the machine learning or AI community a few years back, maybe more than a few years back, maybe a decade or two back. So we had this system, I don't know if you heard about it, it's called psych, C-I-C-Y-C. That was a huge symbolic reasoning system knowledge base. So there were many people encoding their knowledge in the graph-like structure and the system was able to reason correctly over this huge body of knowledge. And one of the criticisms of the system was that it requires a huge amount of human work in order to solve problems and it's not self-growing, right? So it's not self-incremental. It cannot, it does not learn on its own. Then a few years back we started having this huge public revelation that large language models might actually learn things that seem very, very impressive. On the surface level, we have a deep learning system there, right? Something that is learning to predict the next world. Then we have this reinforcement learning system that you mentioned where we are trying to align this model or fine tune it in order to make it more useful. But I think what we don't see in this current generation of models is that there's still a huge amount of human work in the back end, right? There are literally thousands of people as I heard or read somewhere labeling the data, doing the reinforcement learning, like providing the responses for the reinforcement learning loop and so on. And there are all the guardrails and so on and so on. So it seems to me sometimes that we made a circle in a sense and trying to make these models that were supposed to learn by themselves, trying to make them reason more systematically. We again put a lot of human work in the back end just in a slightly different way, right? It's maybe a less structured way, at least at the face value. What are your thoughts on this? - So I do feel like a technology. I like the statement from one of my previous professors. They mentioned that technology improve, probably like a spiral. And so therefore we might revisit a concept that was proposed decades ago, but then given the new hardware and the computational structure, like we're implementing that in a much more powerful way, there could be lots of concepts that are revisited if we look at machine learning publications. And at least for the deep learning jump, it feels to me like the engineering and all the compute resources made everything to another step. And then after the big step, the further improvements can again be applying methods that has been previously shown successful. And we still don't know whether there will be a further step forward. And then on that maybe even machine learning is a way to bring some incremental improvement again. And we'll see. - You currently also work on multi-agent systems. How do you think about these systems? What are your hopes when working on these systems? And how you see the role of causality playing out in this context of multi-agent systems? - So there are different interpretations of multi-agent systems or maybe instantiations of them. I can give two type of examples. One type is that I think the ground concept is more that we initiate different large language models. And then the type of multi-agent system that we're building for improving causal reasoning has both two calling ability and also a critique and debate type of skills. So let's say on the two calling, we basically enable large language models to also implement code and therefore ground formal causal reasoning into actual steps that are programmable and executable. We can also analyze user-upload the CSV data. They'll really imitate how an actual statistics department, causal inference researcher doing. And on the other side, adding the debate ability into large language models has the hope of sort of pointing out each other's errors or diversifying the space of exploration. Let's say that maybe for run three, counterfactual causality, there is still a lot of problems that haven't been explored, the proof-ident viability and so on. It could be possible that a model come up with various solutions and bring storm it with another model and together collaboratively refine it to see whether we can automate a mathematicians job to improve the realm of all the deductions in causality. - Where do you think we are currently with the ability or capability of models, building systematic reasoning chains, so to say? - Yeah, so I think at least for causality, we have recently finished our work called Causal AI Scientist that basically gather a set of formal causal reasoning techniques from ducalculus to other statistical methods like difference in differences and instrumental variables and so on. And there we are taking this more symbolic leaning approach to let language models only handle simple routing things such as check whether there is a valid instrumental variable and then let it generate code or call an existing piece of code and instrumental variables to get the causal effect. So I think with that, we might just need a bit more engineering effort to make models understand a various set of scenarios before reaching a package that can be widely usable across academic disciplines. - That sounds really great. Is that something that you already published or something that is still in review or the process? - We recently put it up. So it's available as a paper and it's in the process of being included in the PiY library. So we will put it as a part of the open source efforts. - Oh, that's really great, that's really great. It reminds me of my conversation, of course, with Amid Sharma and some of the ideas that you mentioned here were also the ideas we were discussing with him. - Amazing, yeah. We have a lot of research synergies and I think the spiral development is also our ongoing discussion because I really like his axiomatic training which encourages models and to end the reasoning ability even more. So it's really exciting to see the diverse explorations. - Yeah, that's really exciting. I also think so. What are the next steps for you in your research? - So we have a line of work that's tried to make causality more and more reasonable and we also are trying various reinforcement learning lines of work to see whether they can also evolve and to end the reasoning abilities. And for that entire line, including causal eye scientists, our goal is to make a very handy, maybe a chat to GPT but to the causal version, causal GPT that's really helping both education like future generations when they need to learn about formal causal inference, no need to dive into a heavy textbook with a lot of prerequisites knowledge but I hope it can be very friendly to her to lead the learners through all the steps. And then I always think that when we manage a knowledge, there is education which means teaching people with less knowledge background to reach almost standard and then there is also AFR science, meaning that we want to lead people who, like more or less have a bit of this knowledge and a very deep domain knowledge to be able to utilize this tool and further advance the frontier of science. So we're looking forward to this whole line of causality as a tool helping AFR education and AFR science. And then on the other side, I guess as you mentioned, another line of my multi-agent work looks into AI safety problems. So we put multiple agents in simulation. One of our 2024 Europe's paper on GovSim, governing the comments simulation basically looks into if we have a digital fishing village and initiate every large language model as a Fisherman character. Then do they know about how overfishing lead to environmental disaster? And there it's more of trying to check whether models know the consequence of its own act. And as a toy study before releasing these agentic models into real world, maybe they will also do something that's harmful for the climate, the workspace if they don't know what their actions will lead to. This reminds me of the tragedy of the commons problem. - Yeah, totally. - Yeah, what were the results in the study? - The point when we were simulating this, most models actually couldn't, so we simulated a whole calendar year where each month there's an iteration of fishing. And most models didn't survive until all the 12 iterations. And probably at that time, the state of the art models have around 50% survival rate, making it to the end. Yeah, and a lot of open source models that we tested at that time still have 0% survival rate. And they usually overfish immediately in the beginning. And we're also working on seeing how much if we map it to real world situations, how much this behavior were transferred to if we let models control factory resources and so on. How will they exploit the pool of resources? - That's very interesting that so many instances of these models just didn't survive in the environment. - Right, yeah. There are, there's a combination of different things. Some of them is reasoning problem. Some of them is short-sightedness, like a big portion of them actually only look at this month's outcome, instead of looking into the long-term future. So I think there might be a chance for causality to come in and a responsible AI should know about the causality of its own action in four different settings. One is, one pair of done is shorten effect and long-term effect because there are various ways that makes this monthly or this year's revenue to be very high, but cause problem long-term. And then the other pair of consideration is selfish, like self-interest versus group interest. Yeah, there are ways maybe make one factory earn a lot, but to make others suffer and how to correctly understand both pairs of consequence and make decisions. I feel like it is an important piece for the future responsible AI. - Yeah, I fully agree. When I imagine, I work a lot with people in industry, and when I have conversations about this agentic systems sometimes, I feel there is a lot of perception that the systems are very mature and they can do a lot of things. They can replace humans in many different areas. At the same time, I feel there is not a lot of awareness of the potential consequences, especially meet to long-term consequences of using the systems and leaving them somehow unsupervised to make decisions in the longer term. And I think the example that you gave and your research points to this risk very, very in a very, very pronounced way, so to say. There was another research very interesting about diplomacy. Have you heard about this? - Yeah. - The consequences were also not that positive. Well, what do you think is the missing point that leads to this self-destructive, self-destructive paths that the models take in complex scenarios like this? - So I think there is really a mixture of causality and the morality. That's why also I'm steering my lab's research at the intersection of those more and more. So the fundamental question here is, does being, what's the relation between being smart and being moral? And if we draw a grid, potentially if we observe results until today, a lot of the failures because the models are still a bit silly, like they don't understand that even for their own self-interest, they should do certain actions. Yeah, a failure of self, maybe multi-turn interest and long-term interest, that is some of the driving reasons. And in the future, we might need to handle a grid of like the models are already smart enough. Then it becomes a question of morality or choice. Like they can both do something that maybe make them the monopoly of the world of an industry or the space for other free competition and so on. Then with that capability, do they choose to be, choose action A or choose action B? And then that is a bit more on the motivation of the model. It can come from different sources, different sources such as maybe in our HF process, certain mentality is encouraged, or maybe in the pre-training process, humans wrote about what type of people get successful and these traits got learned by the model and imitated. These could both be possibilities. And I remember in Jeff Hinton's recent interview, he also mentioned that maybe it's important to inject emotions or maternal instinct to models. So they're more kind and empathetic and so on. That's one more direction if the model have the choice, which will it lean to. That's very interesting. When we think about our own motivations, human motivations, there seems to be a complex interplay of evolutionary and evolutionary in a sense of biological evolution and social factors. Do you think that this kind of a complex intertwined interaction between different motivations, different motivational factors could be something that we could create and inject into models in a way that would be effective in shaping their behaviors, whatever effective means? - Yeah, there is hope, like at least for humans, as you mentioned, like there is maybe self greedy interest at the moment, but then our community also pushes to actions that's more beneficial for the group and more sustaining. And then these pushes can be distilled into law or unspoken rules in the community. Similarly, in the context of large language models, there are this constitution AI line-up approach that enforces certain rules and principles into model decision making. And on the other side, I also give hope on if models really reason and know about the consequence of their act and also even reason about what is a more reasonable goal, like maybe destructing the errors or being the monopoly of the market is not that much fun if we also think about aspects on the other sides. Yeah, and if it's hard to use reasoning to reach that, then we will try to enforce maybe how other people's welfare or the players in the field's welfare should matter to model this decision. But this is still really an open field and we probably need the help of ethics researchers and philosophers to chime in as well. - Do you think that moral reasoning would be possible without causal reasoning? - Definitely causality is a key part in different senses. Let's say that within moral reasoning, there are two very big branches amongst others, like consequentialism and deontology. Consequentialism in its essence has a notion of causality in it. In that, for example, if it allows, if it says like telling a lie is bad but telling a white lie might be good, then it's basically talking about maybe if there is a very sick person on the hospital bed and the level of optimism actually changes the medical outcome of this patient, then telling a white lie have certain possibility to make this person recover. And therefore, this action might be justified because of the consequence, then we are looking at the causal effect of it. On the other side, for deontology, let's say, Immanuel Kant, when he tries to look into what are some moral principles that all of us should obey, he still has certain type of universalization, type of reasoning. Let's say we should not rob people because if everyone rob each other, gets money just because of physical strength, then it is an undesired outcome for society. Therefore, like on no ground, we should allow that. So there are some proof by contradiction, but the backbone engine here is causality. So there could be various reasoning process behind that taps into causality. - When we think about Kantian ethics in particular, you mentioned Immanuel Kant and the moral imperative, one of the moral incarnations that you mentioned. If everyone was behaving this way, then the thing itself would become impossible. That's one of the formulations. But I think even when we think about in Kantian terms, we can always met some special circumstances in which just applying the rule might be very difficult. I don't have a good example right now, but we can imagine a situation where there are two moral rules that are contradicting each other in a certain case. So that would also require us to reason in a systematic way in order to get to some outcome, to make some decision. What are your thoughts about situations like this? And what would that involve in artificial systems? In order to deal with situations like this, where we get to this possible contradiction. - Yeah, totally. I think causality is a very fundamental part of it. Or maybe there are two types of reasoning, or two type of elements that's needed when we look into a conflicting scenario. One is sort of keep running the simulation of what will happen if rule 1 is dominating. We have a rule 2 is dominating. And what will other people see, maybe not the direct part is involved in the system, but if this is established as an example, how will the other, like let's say that we have seen in a lot of ransom situations internationally, like the number of lives on each side is not equal, but when all the nationals watch that news, there is some effect to the whole society that a certain action is leading to. So that's also part of the consequence. On the other side, apart from causal reasoning, understanding what a future will be like, give an action a and action d. On the other side, I feel like there is also some, like intuition, human intuition, human common sense. Like maybe in certain cases, it doesn't make sense. Numerically weighing the two choices, but like we feel something. And yeah, that's a common shared intuition among people, there may be another regulating factor. And these two together help us treat situations that we have never met with with more principle solution. - I have two questions in my mind. I don't know which one will be better now, but I will pick one that my intuition tells me to. So some time ago I went to François Cholet's page with Arc AGI, and I've seen the benchmark for last language models on Arc AGI too. The benchmark dataset, the new benchmark dataset. Now it's not that new, but it has been a while ago. And I thought, oh, actually, I'm thinking so much about how these models can work on this and whether this requires them to reason systematically and so on and so on. But I actually haven't done these tasks myself. So I booked an hour in my calendar and I did, I think maybe like 30 tasks or something. And my thoughts were that I agree with François, that if you understand, start understanding like how these tasks are constructed, they become essentially very easy for you, right? For a human being. And so there were maybe one or two tasks that I thought that there were two different rules that I could infer from the task. And so one solution would be deemed correct on the one incorrect, but nevertheless, I was able to find both rules, both possible rules, right? And then say, okay, one of them probably is more likely. When we look at even the latest language models and their performance on these data sets, they are still not too great. I don't remember exactly what the numbers are, but they are still much worse than even like a teenager probably would do on these tasks. Why is that the case? - I think we might need to dive more detailfully into the questions because a naive solution, I imagine would be that we write a very detailed instructions then maybe a step-by-step how to unfold human reasoning in this case with a couple of illustrations. In that case, there is a chance that the models can pick this up if they're following this reasoning template. Yeah, the other things could be that actually, more thinking that the answer to this question might be hinged down the exact type of reasoning. There is an ongoing line of research at certain large language model companies called illicitation. So in certain cases, models might already have this type of reasoning. They have seen it in a pre-training or the post-training process and maybe the context of how the question was set up in this specific query didn't make the model think about this realm of reasoning and expectations. So it might be worthy to look into this. And if after confirming that this is really a problem, there is ways to do ROHF and other processes to distill this. Then the question becomes do we want it to be a principal layer of reasoning before it enters everything else or how do we prioritize? Yeah, different rows of reasoning, maybe a pure interest and pure problem. Solving centered reasoning versus ethical reasoning before the model conduct any action optimization. So if I understand you correctly, what you're saying is to give a neuroscientific metaphor here, is that there is a part of, there is a neural structure in the model to say metaphorically that would be capable of tackling this kind of a problem. The problem is that this neural structure is not excited or turned on so to say when the model is approaching this kind of a problem. So it's not recognizing in a sense that there's already a existing tool inside of it to tackle the task. Yeah, it could be totally like that and we'll only to see the exact problem to double check. In our conversations with Emily Kijeman from Microsoft Research, she's also involved in the PiWi project you mentioned. We were talking about Sora and this early video models. And so we had a conversation about this hypothesis that these models might have local approximations of how physics work, but they are not physics simulators, right? So we were able to see from the early videos that there were a lot of examples showing that there is a violation of physics, even if the prompt is just following a, let's say regular physics, right? It's not suggesting irregular physics, right? The models are generating something that would be physically impossible on planet Earth and no Dominic 2023 or 405. But some of these things look very plausible. So one of the ideas we were discussing was that oh, they might look possible because the model was able to infer from the videos, from different videos showing different angles, different scenarios, different situations, some approximations of how physics works work locally. And so there is a local representation within this model that is a very good approximations of physics, but it's not a general model that could simulate physics. Do you think that when it comes to causal reasoning in large language models something like this would be also possible that there might be some approximations to causal reasoning and different ranks, like rank two, rank three, and so on, but the model in its entirety is not a good simulator for causal thinking. What would be your thoughts? - Yeah, so I think there are two types of causality here. One is knowledge-based, the one is formal reasoning-based. Let's say that if we're talking about smoking causes cancer, it comes in in the training data so much that models probably already have a shortcut here. Or when it rains, the ground is wet. Yeah, so at least in one of my papers, we call it the knowledge-based causality or common sense causality. Models do have a large set of them. And on the other side, if we're talking about reasoning tasks, such as if we give hypothesis here, then is the model quick enough to realize that there's a confounder or there is a collider causing the correlation to be different than the isolated one. Then that could be a case of like the models are still not that internalized with true, formal causal reasoning. And they could learn from, I don't know, confusing correlation with causation from the training data itself as well. Yeah, and we could say that's like, because a lot of humans feel at these tasks. And if the training data largely comes from how human compose their understanding of the world and write out things, then the intuition becomes the wrong bias coming from humans as well. And what do you think would be the path forward towards making these models more accurate when it comes to systematic reasoning? So I think so far, the aspect that we can make sure, maybe if there's a spectrum, also sort of go back to the neuroscience analogy, then in the spectrum, it's very clear that we're handling a causality problem. Then maybe both models and humans will be alerted and think about, now let's pay attention to listing all the possible confounders and figuring out the calligraph before moving on. That's great. And I think the real problem is maybe when we act model a very daily situation or we casually put out a prompt, then the models might intuitively think about directly looking to pre-training data and output whatever that's a smooth completion here, but not to actively think about causality. So I think the lab setting is more likely to be addressed very soon. And then it's more about how to permeate that into a scenario where we are not that alerted or the model is not that aware that it should actually mix color reasoning into it. I think these type of things maybe universally or very easily activated color reasoning might be a potential direction to aim at. In your career, you started working with causality at some point, but before that you were interested in natural language processing. That was very similar for myself as well, by the way. What inspired you to study causality and to make it to put it at the center of your research? Yeah, I think causality and the language are two different things that really excites me. I think they both have properties of universality in them. Like the language wise, we talk every day, we express ourselves through writing, speaking, listening, and complicated ideas getting to books to pass on to different generations. That's great. And then, authoritatively, if we think about the content of these knowledge, I would see that causal knowledge is something that really shines out and make us feel that this is the right grasp of how things work and it can benefit our future actions a lot. I do personally have lots of causal thinking but in a very informal, colloquial manner. And encountering my PhD supervisor Ben Huncherkov is definitely a great leap. I think before I started my PhD, Ben has causality for machine learning, a lot of his talks are really mind-blowing in terms of how we can build a model that is mechanistic and also domain variant. So it has both elegance in its own and also effect on making models treat. Yeah, I really like the conclusion that if we only live in the ID world, they're learning what have a machine learning correlation-based predictor is great. But if we want to tag into the, oh, the world when situations change, then being with causality is the way for us to handle and see in situation much more reliably. So I think that is an academically elegant branch of research and also nice philosophy in life to have it through tech along career life and so on. What's the best advice you ever got from Ben Huncherkov? I think he has a lot of them and I'm also personally very inspired that Ben Hunch has expertise in mass and statistics, but also he has solid studies into physics and also ongoing interest. And above that, I think one of his past degrees is related to philosophy as well. So I think always have these lines of thinking into our mind and the actual mixture of experts is great. And also although all the domains we mentioned are very classic, Ben Huncherkov is also in general curious about everything when new students come and in interviews. I've heard one of the past student interviews, they were even talking about poems. And yeah, I think having, in his case, three solid domains where he can keep drawing inspirations from. A lot of causality in size actually comes from classical physics that has mechanisms that's invariant. In addition, sort of combine it with daily new information coming from students, collaborators, encounters of diverse background. I think that just helps make this whole. I personally feel like it's academic and philosophical, framework, more and more complete over time. For you, it's also a special moment because you are on the verge of a professional transition. Yeah, I think so. And I feel really lucky that actually I got on the job market relatively early rather than compared to my peer PhDs or people in other stages of their academic career. It feels very lucky that it's a combination of the PhD research is smooth and there were a lot of deliverables along the process and also because more and more schools start to pay attention to the rise of large language models. So I had the fortune to be interviewed a lot of places and encountering different opportunities. Any other decision? Was? My decision is the University of Toronto. It was great. It's great, thank you. And I'm really, really looking forward to one of the first plays of the modern deep learning. How does it feel? It feels very exciting. It's definitely a different set of researchers than my normal, let's say, in Germany, my Max Planck-Institut Circle and, academically, I have this ACL Circle and the causality community. I think the University of Toronto gives me a feeling of very large and diverse whole university structure where I could knock at the door philosophers or psychologists and so on. On the other side, the machine learning line of research feels very historical whenever I talk to colleagues because they can tell the birth stories about how different activation functions work, how Adam Optimizer was emerged, and how early lectures of Jeff was like within University of Toronto. And even before ImageNet, how convincing Jeff's arguments are and inspired a lot of generations of U of T students. When are you when you're starting at the University of Toronto? Hopefully soon. I think my students are already starting and we're forming the Genesis Lab, also in memory of one of my early mentors, Patrick Winston, when I was doing an exchange at MIT. His lab was called Genesis and he really wants to implement societies of mind and many different AI theories. I was very, very inspired by all his passion into AI and his principled way as a scholar. Unfortunately, he passed away 2019 and so I think making the lab name both a combination of the pronunciation from Patrick's lab and a little bit of my own name is sort of integrates my hope to also proceed in the career of AI with more determination and trying to have something unique into the field. That's a really beautiful story and it really feels you have a lot of gratitude towards him. Yeah, yeah, definitely. And I think maybe everyone have the shared experience of like when you're just starting the career, you explore a lot of different research projects and people have different work styles and there are so many different AI-related topics or within CS, different professors have their own style. And I'm really, really grateful to have very kind mentors. I think Patrick was great showing me how a successful scholar who is very, very senior can be like. And I also have direct mentors, PhD students, also leading me into natural language processing, especially diging and japan school. So some of them I met during undergraduate, some of them during an internship at Amazon. And they really helped me to embark my NLP academic career before I grow strong enough to navigate the tide of large language models. That's really great. Before we finish, I wanted to ask you about your more personal note about your experience with different cultures. So you were growing up in Shanghai. Yeah. And now you spend a significant part of your life in Germany and in particular in Tübingen, which is not a huge city. It's a smaller place with its own culture and its own vibe, so to say. Now you're moving to Canada. How did these experiences shape you? And what would be something that you think was the most important in them in making you who you are today? Oh, I really, really appreciate this question giving me a moment of reflection. I think in Shanghai, I am really, really grateful to how my parents protected me out of the super stressful education system. And then there, I learned to be very focused and very into education, books, and various like painting down knowledge, great determination. And in Tübingen and later Toronto, but Tübingen is a place where I spend a lot of time with the peak of my intelligence and the peak of my energy because of the timing of the age reasons in. And then, as mentioned, I think Benhard's expertise area like mass, statistics, physics, philosophy, although I'm very amateur in all of them. But coming as a natural language processing researcher, but being aware of all these domains and being surrounded by scholars, often featuring experts with expertise in one of these domains, opened my academic mind and make me appreciate the continental Europe style of reasoning and also for me, the life in Tübingen is very tranquil. And I can focus on a problem continuously without any interruptions. So it can be in a walk 10 minutes away from the office, full of greens, and a new ideas emerge. I think that is really, really a gift in life. And I think the story of Immanuel Kant walking every day and many other European scholars just keep serving as prototypes when I am a scholar in Tübingen. Well, I will still hold the co-affiliation with Max Funke Institute. And I'm moving to Canada. It sort of gives a very different vibe in that now I see people from humanities disciplines more out of my scholarly circle in Tübingen. And I see a university whose maybe number of students is equivalent to a small town. And that gives a very different vibe. And people can come through the city. They can arrange their life in very different ways. Plus all the faculties whose name is already super famous in deep learning and play both historical and important current roles. That gives me a feeling of what's the next step of AI? Where do I expect myself to be in the history of AI if I look back 50 years later? Yeah. So it's a really, really exciting place. And with a flux of talents now that certain political situations happen and more and more people might look for places while coming open science and open ideology. Yeah. So it will be a fascinating start that I'm looking forward to. I'm wishing you the greatest possible experience in this journey. It sounds really incredible. And I'm so happy to hear that you find so much opportunity in this. The last question I wanted to ask you is about reading. You mentioned reading just before and the importance of reading for yourself in your own development. What are two books that change your life? Great question. It's hard to list at the top two. But I think just very intuitively right receiving the question. I really like the history of Western philosophy by Bertrand Russell. In general, like Russell's writing and his way of thinking, I was especially inspired by like overview of various philosophers and the starting from the ancient Greek time where lots of debate and pure curiosity. Like before there were even division of disciplines. Like just pure curiosity leads to explorations and formalization of a lot of topics. Another peak I'm really appreciating is the emergence of political philosophers in the Enlightenment movement figuring out what really works for society. And then the other book that contributes to my PhD time is a biography of Leonardo da Vinci. And I sort of feel that a lot of us researchers, we might be able to resonate with how Da Vinci observes life, take very, very nice notes, and try to draw connections of things. Yeah, there are a lot of ways of doing things that I feel like I both personally resonate with it and I look forward to mentoring my students into these ways of thinking. And keep a very curious heart for everything disregard the fixed disciplines so that we follow our heart. If you're interested, certainly interesting something in psychology, go ahead and pursue it. And connect it back to natural language processing if we can or we navigate something that's cool. And also just, yeah, be brave on whatever topic. Once we apply our thinking into it, there is some space for innovation. So I really like that type of courage and diversity. What's your message to the causal community? To the causal community, I think maybe I'm way to junior to give a message there. But if there is a space for my very, very personal view, then I would think that I look forward to causality's connection with more and more domains. I look forward to its actual impact on things in industry or academic wise, how it affects different departments even more. There are a lot of valuable ideas, or maybe even how to transform future news, news consumption, and so on. There are a lot of great essence in causality that we should spread it to more and more domains. Eugene, thank you so much. That was a wonderful conversation. I really appreciate your time. Thank you, Alex. This is great. Thank you.