Kan AI écht tot het einde van de wereld leiden?

A podcast about the world of science and digital technology with support from IMEG. The Chip Labo of the World. Artificial interaction. Artificial intelligence leads to the end of the world. That's in a new and still ongoing book by Eliezer Jodkowski. But we don't have Jodkowski in the studio here, but Professor Lode Lawaard has also written a book and we're going to talk about the apocalypse. This is Bidsalatoma. I'm Lode Lawaard. I'm Pieter van Dooren. And I'm Dominik Dekman. Lode, what are you? I'm a teacher of philosophy, technical philosophy, at the KU Leuven. And I'm the head of Chair Ethics, where I mainly research artificial intelligence, artificial intelligence and superintelligence. And you organized a congress in Dezalento. And I came here a while ago and there were old doomedenkers filming the AI together. I don't know, Dominik, if it's all doomedenkers. Not all of them. I think all of them are alarmists. And I think that's true. That they track alarm bells and signal a number of problems around the AI. But I think the point of this conference, which is also thought about, what are the solutions, how can we deal with these kinds of risks? So they can say doomedenkers, no, but alarmists. Was that a selection from a large scale? Or is that about the average of what most people know about it? There is no representative data set. Not at all. On the conference, there were already hundreds of people, people who are interested in artificial intelligence and existential risks. But of course, that small population, it's not the whole thing about people who have seen the academic world with AI. The community is very, very limited. But the mayor of all AI doomedenkers, that is one of them, Eliezer Jutkowski. That's true. He actually put that in action. Yes, as you could say, more than ten years ago in the meantime. But I think it's the same, maybe a little too much. And I don't think it's entirely up to him, that he really was the first one. There is a cluster of people who have been thinking about the same theme at the same time. In some cases, of course, Jutkowski. In others, of course, Bostrom. Nick Bostrom. Nick Bostrom. He is a philosopher. He is also known with his book, Super Intelligence. So we could say that what Jutkowski just published, is that that is a continuation or a description of what was written by Bostrom ten or eleven years ago. Jutkowski has now come with a new book, with the, at least since he has been called, if anyone builds it. Everyone dies. What is the "it" that is? Super Intelligence. Super Intelligence indeed. So we often talk to the EI about Artificial General Intelligence. That is the EI that can do about anything. What is precisely the description of EGI? I don't think you make a lot of mistakes when you say that EGI, an AI system, is that a large number of economically relevant activities of people can be taken over. They go from a euro plan to academic research, to writing, to a book. It is mostly about economically relevant but also cognitive and intellectual. Even if they don't want to contribute. An important thing, I don't think there is a good difference. What is the EI, the EGI or narrow EI? I think that is not clear. How many activities are there? There are 20, 21, 35, 40, 100. It is not clear. But it is about a substantial number of cognitive, economically relevant activities. What is important is always about activities at a general human level. More of the companies that develop technology, such as OpenEI, talk about the EGI, that EI can do about anything. What we cognitively can do. There we are about maybe two years, maybe three years, maybe five years. But that is a bit the big order we are talking about now. Some people are the CEOs of the big drive. Amo Dey from Entropic says that it will be this year or next year. Yes, he is even a bit ashamed. Yes, I think some help me too, because they are quite optimistic. I think that this is the last month. Yes, yes, yes. The last week it is still a little bit. And Mark Zuckerberg in a podcast. Bas van Meten has been working on it for more than five years. So you hear those numbers walk up again a little bit. While we had this feeling six months ago. It is really maybe this year or definitely next year. But what comes after that is something that is even smarter than we. And that is called the Super Intelligence. Artificial Super Intelligence or ASI. And Jutkowski says if someone builds it, then we all go there. The EGI is the human level, my level. By writing, by reading, by doing this. So people can be a kind of forward setting of the EGI. I know everything or a lot of economic relevant activities. But just don't do it on an expert level, and that is only ten or so. Yes, or they can say, while writing academic research and bank management, you always have the top CEO, the top scientist, the top writer. And then it is still comprehensive. So then you are on Super Intelligence indeed. Realistically seen, it is not impossible that we have been there for a few years, maybe over ten years. And what does Eliezer Jutkowski say, if that happens, we all go there. And that is not a risk, that is not a danger, that is actually security. We are going to kill each other. A very strong claim. Yes. I think it is very doubtful. But I understand the dangers around Super Intelligence. I think I understand somewhere what the way of Super Intelligence to extinction is. Yes, she says we build something there. On the one hand, it becomes much smarter than us. And on the other hand, she says that we do not have that in our hands at all. Yes, I think if we actually develop Super Intelligence, what kind of companies it is, an American or Chinese or Russian company, that doesn't matter. I think that the chance is very real, that we indeed lose control there. That there is actually, at some point, the cognitive system is so intelligent that we can immediately think about what we could never think about and what we can therefore lose control of the system. Yes, but then she says skeptically, why would she want to do something else in God's name than just help us friendly? That thing doesn't want anything at all. It is a computer. And why would we now think in God's name that he wants to do something wrong with us? That is then what one of those top researchers probably says. He says, where do you actually make your worries about? Yes, I understand that question. And I think indeed the system doesn't want anything. They don't want, they don't have mental conditions that we have, such as long-term emotions and anxiety and so forth. But it is not because these things are not there, that it can cause great harm. It is not because these things are not there, such as a will and an emotion that you cannot lose control of that. Yes, that means the viruses that we know, for coronavirus, also have no mental conditions. We can also lose control of that, and it can also cause great harm. I was just going to say, there are a lot of things that we can lose control of, or that may be lost. Not so long ago, there was also all kinds of nanotechnology, and there were devils who didn't have to come, because humanity went down. Now that has not happened for a long time, no one is afraid of nanos, since those biological systems are mostly nanos. In this case it is different. We are not afraid of losing control of ourselves, instead of the things that lose control of us. Yes, I think that losing control is of course an issue. A losing control can also be pleasant and can also be neutral. But in this case you have the whole system, which can function digitally worldwide very quickly, and can cause great impact. And above all, you can also make your own decisions. They can be made in between, and they can be made in between, which we don't really see at all. And I think there is a big difference with everything that we have seen in the past years, decades, hundreds of thousands of years. We have a system that is super powerful, super intelligent, super quickly handled, worldwide, to have immediate impact, and really handles itself, or what we call an agency. A crucial thing in the book by Jutkowski is the AI systems. You have to pay attention to that, because we haven't built them, we have grown them. I think we should say that, we create those models in the sense that we start with a sort of algorithm, we feel that algorithm with a huge amount of data, and then we grow such a flexible model with, in the meantime, we are in the billions of parameters, and we actually don't know how that works, what it does. It is said to be a black box, but it is still a very, very big black box. We actually don't know what is going on, we think, we say that we have built those things, but actually it is almost organically grown out of the data on the internet. That's a bit, and you can't control that, is he saying? I think that the scenario of control-free has nothing to do with that. I think that what it makes a comparison with how a child is born, that they know all the things that they have to do, indeed, and they can put a number of actions, or two adults can put an action, because a child is born there, but further you have no control over that. You also don't know what happens. I think that is exactly the comparison. Yes, but you don't know. Yes, and he then says, at a certain point, Munde has such an AI system, a sort of will, and he actually tries, he talks to him, we don't know if he really wants something, and that doesn't have to, because it behaves as if he wants something, and there he goes a bit short on the edge, from then on, he says, yes, it behaves as if he wants something, and so he wants it. And then he goes further on, so he wants something very strange, because we don't know how you are together, he is completely different from a person, he wants something, he wants a goal, that he might have got from us, but then maybe he might understand slightly differently, and so you actually don't know what he's going to do, but you can't stop him, because he's smarter than you. That's a bit of the idea. I think the whole discussion around that we want, as I can say, I understand that, but I don't think we have to be too mysterious about it, I mean, it's about a system where a certain goal is programmed or implemented by the designer, and the system wants to achieve that goal, wants to get in between, it's just about that, if there is one obstacle, the system is going to do everything to turn it around, and if it doesn't work, then it goes another way, and if it doesn't work, then it goes another way, and that kind of historic attempts to turn those obstacles around, you understand that, of course it has a strong will to achieve the goals, I don't think that's so strange, I don't think it's a strange description, so in that sense, indeed, we say that it has the will, but that's of course not about the cognitive condition it gives. And then we often come with those thoughts, experiments that, in this whole year, are all very theoretical, and maybe, and the most famous is that of the paperclip manufacturer. What exactly was made by Bostrom, that thought... I was inspired by Piotr Kalski, he said, "Yeah, but I was going to tell it in a different way." We have popularized it. Yes, it has been popularized by Nick Bostrom, and I saw that. The idea is indeed, so look, think about an AI system, that gives us the courage to make as many paperclips, very trivial things, very moral neutral things, and that system would be able to argumentate or think about, look, everything that is possible, whatever it is, can be enough material, to make as many paperclips as possible, and so, all organic material, people, animals, and other systems that exist, we have to destroy them, to reach my goal. And that sounds a bit, drawn on the bottle, and that's a bit weird, but the idea is, of course, it's about, that if you have a system, and you have a goal, that can mean, what a stupid, trivial goal is, if you don't think about it well, about the guardrails, or the rules that you have, implemented in the system, then you can come up with some weird things, and it's a bit of an extreme example, but it's precisely in the extremity, that you feel something, what is intuition? If you give a trivial goal, to the system, and you don't think about enough rules, then the system can do very human things, and of course we don't want that. Those who don't want to be seen, precisely because the system is so smart, can recognize it, which we actually can't think of at all, because we are limited in our cognitive capabilities. Well, we also don't think about walking around in the street, and such a system is very smart. Yes, and that says, Jodkowski actually, at a certain moment, we always walk in the street, that thing is going to want something very strange, that we weren't exactly what we wanted, that we wanted, we couldn't control that at all, because we don't know exactly how it works, so no matter how we asked one question, solve this problem, we actually don't want lights, we are stuck, then at that moment, we can't stop it anymore. Now, I think that's one of the problems that I have with Jodkowski, so this is about, or is it about, that the AI system, at a certain moment, makes the link with the material physical reality, or embattles in robots or something, and I think that you, I think that superintention is a very real possibility, and I also think that it can cause very big damage, I don't think it's an extinction, like Jodkowski, but I think that it is not necessary for the big damage, that it embattles in robots or in physical entities, it can function purely digitally, and also cause big damage, and realize it. Lo, you wrote a book yourself, and that book is called AI, what no one tells us, and what is it that no one tells us about AI? So I'm in the same line as Jodkowski, but I'm going a bit further, so I think that AI, big catastrophes, can cause problems, now, but also in the future, can cause problems, and Jodkowski said, that's actually true. No, I don't think so at all. I think that the probability is very low, of the big risks, but I think it's high enough to pay attention to that. Yes. So I've already talked about a number of scenarios, that have to do with nuclear weapons, with bio-weapons, with just accidents, I've talked about great power concentration by AI, and I've also talked about loss of control, and so the last five scenarios are very close to what Jodkowski talks about. But I'm not going to say that it's going to happen, and that everything is going to happen, I don't think it's going to be about extinction. At first, we're talking about a number of dangers around weapons, both bio-weapons as nuclear weapons. Yes. There's already a lot to say and write about, many years and no. How do you see that? Well, as I said, so I think, maybe we'll talk about it, but that's the problem of today, I think it's less noticeable, because there are a lot of people, so there's a lot of discrimination. We have to talk about that. There are also a lot of people, like you, the technical philosopher, who say we shouldn't be working at all with those great dangers of AI, because that's all far away. And actually, when you're talking about how AI brings the humanity into danger, you actually do it with the hype of those technology companies, because you bring them together, you confirm their idea that that technology is so powerful. While people, like for example Tim DeGubbro, you name it, also in your book, the researcher at Google, who is our witness, who says, "No, the first technology is not so good. It has a lot of faults." And for example, those faults have made it clear that it is trained from the material of the internet and that material is not so, not so, not so high quality. And they say, we have to look a lot more at the problems of that technology now, as there are, before or the whole time, that these things can still be racist things, but also if we have to make decisions, we have to take forward racist, sexist decisions. We have to look at that, that's what a whole group of researchers say. Well, I follow people like Gibro for example, I think indeed that we have to look in the first place at that kind of transparency, bias and so forth problem. Those are the everyday problems, that's our first attention. But you think that there is actually enough to be talked about? Yes, quite enough, I don't think it will end there. I think that there are also risks of the future, or more catastrophic risks, there is probably much lower. Even though it is not always known, there is probably much lower. I think that we also have to look at that. That is the first thing I want to say. The second, as it is said, by someone like Gibro, those systems, they don't work well, and so forth, and they are not going to be able to do that. Yes, I find that all a bit strange. Of course there is no super intelligence, of course there is an EGI. But 10 years ago, really not 10, 12, 13 years ago, a large part of the computer scientists were convinced that it would be particularly difficult to generate images with an AI system, to write large texts, to write doctors, to research scientists, but that is exactly what the case is today. And that is actually for the most part, what fundamental breakthroughs, breakthroughs have been for 100 years, you can fly, you can become a gunman, and so forth. A lot of people have always thought that it is impossible, that it will be complete, but a lot of things have come through. So I am a bit skeptical of someone like Gibro who says, "There are still full problems now, so we don't have to worry about that too much." Because those problems can be solved quickly. They can be solved, and most problems can be solved. Participate with the hype. I certainly have a sympatic for that argument. As you said, superintelligence will always come, then you do it with the hype of someone like Elon Musk and Sam Elpin. That can be true, but then don't take away that the concerns can be true. So if I think that the chance of superintelligence is big enough, and that it will bring certain big-scale risks with itself, then I think I have to talk about that. You don't have to slide under the mat. Even if it means that I am talking about someone like Sam Elpin, then I am actually copying something. I think if those risks are there, then you have to talk about it. So I don't think we have to slide under the mat. Just because we are more in the direction of the discussion of someone like Sam Elpin. I think that is a bit of a weird argument. You mention a number of scenarios where the use of AI can lead to relatively big dangers for the planet mankind, without being completely expanded. But you mention such a number. So there is no superintelligence or AGI needed to cause big-scale risks. So you have AI systems, an everyday AI system, a narrow AI system, that can only do one thing, and for example the AI structure. AI structure, folding, yes. So that is about the creation of AlfaFold, AlfaFold. AlfaFold, yes. AlfaFold from Google and DeepMinds. And yes, you can create big-scale problems with that kind of AI. Of course, that only has to do with a number of obscure bioterrorists who use the means and who are actually trying to create big terror. So they use what Google actually developed to make devices and then actually use it to make a bio-weapon. So there is a very interesting paper difference, I think, two or three years ago, which is exactly about that. So the company had a AI system developed to develop an antibiotic in the hands of AI. Because of course it is amazing, it is fantastic. It is one of the best inventions ever. Million-dollar revenue. Only that same model, so it was asked in the experiment, to create, to develop, to develop, because it is not physical. To develop, to develop is not toxic molecules. And that went pretty well. Only in the middle of the experiment it was also asked about the system, but look, it does not develop, it is toxic molecules. And apparently, in a short time span of six hours, the system seems to be able to develop 40,000 harmful molecules. And not physically and so forth. I know that. But I think it is enough to take care of it, at least. To take care of the alarm bell that has to go off. So the systems are not particularly expensive. They can be at once, or at some point, can also fall in the hands of a bioterrorist, for example. Or a country with very bad intentions, in such a large scale, in a very short term, to cause damage. And we would, with that system, virus can develop, a very long incubation period, go through, which spreads very quickly, spreads in very specific groups of people, which are difficult or are not directed, and so forth. So that's what I would say. That is a huge EI. Narrow EI. That's no EGI, no super intelligentsia. It can be used by so many people. And if you know that the bioterrorist level is not at zero for many centuries, even if it has increased for decades, then I think there is a reason to be concerned about it. But we are concerned about technology. We are much better concerned about the misuse of which technology, then also about terrorism, about use of ethics. Is it not for the people, instead of a successful technology, which is a little bit afraid of? No, I think the problem is, of course, with people with bad intentions, in this case. Clearly. Only the bioterrorists can use a lot of means to realize their goals. I think that they have now found a new medium with EI. And that new means can be found, that new means makes a big difference. I mean, bioterrorism has existed for a long time, and bioterrorists have also existed for a long time. But it is quite relevant to know that AI, that today can already realize that kind of bioterroristic attacks. Just because you know that the virus can be much more harmful, can spread much faster and so on, then the classical means, the bioterrorists, or the classical means that bioterrorists use. So if I got it right, think about it now, or if I started with a sort of protocol, where those few companies can effectively make a new cell, with those CRISPR-installations, that that organization makes sure that they don't just put a devoid DNA structure for a virus in each other. They have to scan it, we're not going to make a dangerous cell. There was a very recent research, a few days ago, where it came out, that we were trying to change certain molecules, just a little bit, certain substances, and just a little bit different to formulate to make sure that automatic detection of such a CRISPR business would be avoided, so that you, as a smart bioterrorist, by one of those few factories that really has that technology at home, that you can mislead to make something that is innocent at first sight. But it is just the AI that actually does the translation and represents a dangerous molecule. It's already happening with drugs, with bombs, with raw materials, so nobody knows that something dangerous could happen to them. Now, another danger that you're talking about is that I thought it was less likely that what if we now control our nuclear weapons with an AI system, because everybody trusts that fantastic AI system. There's a lot of stability in it, with a red button, and I don't know if it's that much safer than an AI at the moment. For a couple of times, the whole nipton system is out of control. In any case, the title of the book is "A.I. What Nobody Tells Us". So today, it's often about the kind of very positive advantage, and of course, I don't doubt it for a second. It's often about small, gibroid-like risks, but so the bioterrorist is something that often doesn't come to mind. It's a debate, but just like the nuclear threat that is there, it's more likely to be suppressed by artificial intelligence. And then you say, "I don't see it coming so fast, but there are two reasons to doubt it. The first reason is that nuclear defence mechanisms in the Soviet Union in 1980 have already been automated. So we have the dead hand and the perimeter. So we know in the 1980s that the Soviet Union, the system that has to answer when a nuclear attack was started in the Soviet Union, that the system was automated. So if there are automated nuclear systems today, then you can expect that in the future there will be the case and that it will be improved by artificial intelligence. Or that artificial intelligence is also used there. The second thing is, last year in 1924, in Peru, an international conference or a meeting place was held between former President Biden and the Chinese president. They discussed it, looked at that kind of technology, AI, we haven't used it for the management of nuclear systems. So just to say, they have discussed it. They have discussed it. They have discussed it. Of course, this was a handshake deal. Yes. And so it is not binding and so forth. Not less, that suggests at least that there is still thought about it, that there is also thought about it, that there is debate about it, about exactly the implementation of AI before exactly that kind of nuclear systems. So just to say, that's not going to happen, I think there are still a number of reasons to doubt it. And if it is left over to AI, you write, then the chance is very real, that something is going wrong. Very real. Yes. I am, of course, aware as a philosopher, because taking into account that it is likely that it is not particularly difficult, but I don't think it is a good idea. Yes. And a number of weeks ago, when it is new to me, a letter from the General Assembly of the United Nations was presented to me, among other things, Joffrey Hinton, who also covered it, also talked about it. Look, we shouldn't do that. That is a very bad idea. And I think there are reasons to still be skeptical about it. What is it about? The AI systems are trained on big data. And yes, there is a lot of data for AI, a lot of data. Of course, for this kind of system, it can be said that we have very little data and also no qualitative data. Yes, we have very little data of affected or even not affected nuclear attacks on all countries. Of affected nuclear attacks, we have no... Well, we only have two. Two, right? I think two, right? I think two. So there is not enough data to train an AI system, and that is not reliable, for sure. So that is a bit of a problem that we could prevent. There are also, I think, other risks, but that also applies to other systems, or systems that are controlled by people or are controlled. If you talk about battlefields, drones, what is also discussed, there should be no people in that chain, may those drones decide on their own that this is my goal and I will skip it. Yes, and there are quite a few arguments, to draw on people, to automate it, I mean, the fact that they can react much faster, in the case of drones, for example, and so also faster can anticipate problems. It is an argument for, but I think that is not decided. Even if you look at history, in the past, let's say, two, three, four, five decades, the number of close calls was there, where it was clear that the system had to direct that for a nuclear attack, that it failed, and that it is still thanks to them, a human intervention, that the nuclear attack did not occur. I think that in the 1980s, there was a very well-known case where a Russian officer, an AIC, or a technology, directed that there was a nuclear attack, and that it seemed to be justified. Had that man not done that, we were in a nuclear... It was a computer fault, it looked like an accident, and that man said, "No, we're not going to react. It has to be a computer fault." Yes. So there were not so many cases where people had to intervene, but in any case, there were enough cases where it was clear that the system failed. So I am still very skeptical of the completely overlapping of that sort of system. But there are still dangers that you are talking about in your book. Yes. That certain companies, as countries, can become too powerful if they are the first to develop the AGI or ASI system. Yes. So the AI problem that I just mentioned, there are still a narrow AI system, and that's it. That doesn't have to be any AGI or ASI system. Indeed. But there are also some risks associated with AGI, and one of them, or at least two, is power misuse or unbridled, unseen power concentration. And how do we actually have to imagine ourselves? Well, I think for the creation of AGI, there are a lot of means there. So it's not about what companies or countries will be able to develop. So there is already a certain concentration that will be introduced, and it will be able to be strengthened by developing artificial intelligence at a general level. Yes. We see open AI that actually has plans in total a billion euros, and that's a billion. A lot. To invest, well, obviously they don't have it, but they are going to invest the next five, six years on AI infrastructure. In the sense that a lot of money, and then no one else can say, "I'm going to invest even more than a billion." Yes, so there are a lot of means there. And if you develop Open AI, the AGI system, it will probably be one of the only ones. Yes. So a system that can be used in many areas, implemented by a land-based company. And that's actually everything that people can do. Yes. And that is, on a large scale, people can replace. Yes. What is very attractive for companies and companies. Yes. And in that, of course, the AGI system, so it's completely irrelevant. So what you have is a system that can be used for many things, because many countries or companies could be purchased. But what you then have is that certain convincements, certain consciousnesses, or unconsciousness, in that system can be used. Or they can use, in that system, in other words, bias. Yes. We understand, let's say that Open AI is, but it can be Google, the next version of Gemini. Yes. The next version, or the next version, of GPT-5 comes. And then GPT-7 is suddenly AGI. Everything seems to be possible, what a typical office worker can do, and what his boss can do. And it only comes from one company. If you want to fill all your employees by almost free, digital employees, or even replace, you can just be open AI at one company. Open AI. Open AI. And that makes it possible. Yes. And how much can they ask for their technology? As long as it is good to buy, is that a person? Can they ask that company? Yes. And then yes, super much money, super much power for that one company. For that one man, Sam Altman, something like that. Yes. So that's, as you can say, that's one form of problem. That's one kind of problem. They can ask a lot of questions for one kind of technology. But the other problem I have already alluded to, Sam Altman has very clear political prospects, just like Elon Musk has. And they can be implemented in that system, all that is not consciously implemented. The danger is, with other words, that a lot of companies, a lot of countries worldwide, are confronted with very specific prospects, with very specific ideas, political or not political ideas, where it is not really wanted to be confronted with, but the system is not so attractive that it has to be bought by the company, at risk, at a very short time span of a number of months, years, one specific anti-woke, anti-left idea is spread, which is implemented in the system. And you could say, the problem is not the system itself. The problem is indeed, what you said just now, is the person, or the CEO, or the company behind the AGI system. Is that a total nonsense? No, it has already happened. Look at colonialism, where Europe suddenly decided that the European way of thinking, the norm, was for the whole world. Obviously, books are indeed that kind of historical form, but also today. I mean, X, Twitter, from Musk, is something else than a platform to spread it as quickly as possible from certain very specific political prejudices. So if you know that the CEOs have very specific prejudices, have very specific political prejudices, if they know that they now want to develop the best technology and platforms to spread those ideas, then you can also suspect that at a certain point, the AGI is already being used to spread very specific prejudices. And that, I think, we have to make sure of that. It's not about viruses, it's not about nuclear war, and it's not about superintelligence and control, but it's actually about, yes, control is already in, we don't have control more about the system, or about the person who made the system, but who is so powerful, and that we are actually not really going to use that kind of technology. Then you would really be to let the page, let us make sure that there are as many companies, less or more at the same time to reach such a technology. And luckily, in Europe, we still have Mistrel, maybe a little bit more for the prices, maybe not for the golden medal. Or Open Ship. You would still want that, that there is a wide AGI research. Is that a lot of companies that do the same thing? Yes, I think that that seems to be the ideal world. Or that there are, yes, as with nuclear energy, that there are international organs who are conducting research there. So I think that is, of course, an utopia of, I think that is very interesting, maybe not really feasible. But it seems to me, indeed, a bad idea that, that we have a certain amount of AGI, that by one or two countries, or large companies, development is delayed. Yes, and then you actually come to the end of your book. If we are all going to the table together, all countries around the world, and we make clear promises about that, then we come to the end. Come, but yes, it sounds like that. Yes, we have to laugh a little bit about that. You actually also come out in your book. Here international cooperation is still taking place. I understand your brand. But what are the alternatives? So they could say, so if you have AGI, the previous technology, and that actually means that, to come with super-intelligence, you can of course have that. It can take over everything. And so also AI research, and that is actually the whole clue. Yes. The AI research can be automated by itself. And so you can get into a loop, like a positive feedback loop, where AI research is improved, and improved, and so on. Once the AI is slimmer, it can develop faster than a normal person, the next version of AI. And then it's going to be faster. And by the way, if you think that is pure speculation, the day after day, AI research is automated, so that is not a spy. That is already a reason to suspect that it will come. Do we have to, what are the alternatives if you don't do it? Yes. We mean, if you don't want international cooperation, what are you going to do? Are you going to leave it to the market? No good idea. No good idea. Or leave it to countries that have a competitive environment, and go as fast as possible. Superintelligence. Well, the point of me, but also someone like Bostrom and Jutkovsky, is that superintelligence can mean that you lose control. So the risk is that there are one, two, three, four countries with superintelligence, which they lose control over themselves. So then you think that we are in a very unusual, a very strange, but also dangerous situation. You will of course be able to develop superintelligence to counter the other superintelligence. I understand that. But not less. I think that five, four, three, two countries that all have those resources, that that is risk-free. And it's not going to be about the bad intentions of the company. It's going to be about a company that may lose control for the system that has made it. But those international consensus, and even a dream of a kind of international organization that sees itself on the most advanced AI systems, sometimes you actually think of that too. Indeed. A new international study, as it is for nuclear energy and nuclear weapons. Yes. It's not really in the map at the moment. No, it's not. It's true. I try to describe what I think is desirable and I think that it must be bearable. Of course there are current geopolitical tensions that lead to that kind of solution. Yes, so in 2023 there was a moment to be there. There was that Bledschli declaration. In Great Britain, in Bledschli Park. So the place where time is the second world war Alan Turing, the enigma code of the Germans. At that place they held that congress about the danger of AI. And then you had the heads of states. I believe there were thirty countries that declared together. We had to do something about that. We had to stop those dangers. That Bledschli declaration. That was the moment. Then there was someone else in the party, President Wellieswa. The Bledschli conference was the first in a series of events. Every six months in South Korea. Then there was this year in France. Then there was one in Paris. And there was a tone completely different. That was a lot of people. Then it was more about you have to put up with the dangers. But above all, we have to go, go, go. To miss all those fantastic opportunities of AI. That was actually two years ago. Yes, but I think you have to carry out the possible introspection. It has to be said, as a dissolution with artificial intelligence research. It's not a number of things to dissolution. That it won't work on the long term. AI is super-intelligence that can now dissolution and EGI can now dissolution. But maybe in ten years. Even for that delay. That needs time. Yes. In and in the spirit of political leaders. So we are only two years further. I hope that happens quickly. Yes. I think it is also desirable that that happens. But that that now in the lens of this year another turn went on. Yes, I would not dare say that it is definitely now. That the moment has been lost. I think we still have a lot of time. Of course. But I don't think there is super-intelligence for tomorrow. Not at least I think that we are still there. That we have to play the ball quickly. Another moment around the same time. When the Bletchy congressman also said that open letter that a lot of people have signed. He said let's just have a little break and stop the development of those bad things. That never happens. Yes. But they asked for a six-month break. Let's have a six-month break now. And then we can maybe look at the dangers a little bit calmly and make a solution. Those six-month breaks did not come. They did not come. I have sympathy. A lot of sympathy for the idea behind the letter. Namely, you have to stop or break or come to a moratorium. Certain developments. But of course I have no sympathy for those six months. Because those six months you will not have any EGI. The solution to the problem of EGI and super-intelligence will not solve it. You can't find out anything. It is there. So you should take a photo with it. You should not do anything. We are not there yet. The ball is already rolling in that direction. And then so much money is given out that you then think that if it is possible to develop EGI and super-intelligence then we will achieve it. We do everything we can. But good. It is a kind of red flag that has been raised in a way. And the international has received a lot of advertising. And a lot of people have signed it. So I think that has contributed to the kind of internationalization of certain movements which are more aware, also politically aware of what is around us. That kind of not small-scale problems that can be solved with EGI and super-intelligence. So I have a lot of sympathy for the motivation behind the letter, also for the effects of consciousness. But the concrete idea of six months is now less. It is mainly because we have to be alert and stay alert. Yes, alert, but I think we have to act preventively. So other than with other technologies, I think of nuclear weapons, I also think of social media, I think we have to think more preventively. What are the possible risks? There are certain things that we don't want in the case of super-intelligence. While we want to take the advantages of super-intelligence naturally. That is clear. But on those possible risks I think we have to anticipate. We travel so fast to something and it goes so fast. And yes, that international consensus that asks time, that is not so good. Now, I am hopeful. If there is sufficient awareness, strong awareness and widespread awareness of serious risks, I think we can continue to do that. I believe so. Maybe it shows that a lot of things have been said for over 100 years, because we actually don't want that. There is a political awareness that at a certain moment it grows and that it leads to certain actions. Of course, that is not the ideal world, but that is already for super-intelligence or EIA and EGI also take the risk. Is there the current autocratic trend between bikes that doesn't help at all? No, that's true. What do you think about this? It all depends on how unpredictable it is now. We don't know if the EGI will suddenly fall out of the air and we don't know how it could all go wrong. Now, take 20 years after the death of Facebook, almost 20 years after the death of the iPhone. We have now become very afraid of the influence of smartphones and social networks on children. We start to avoid that suddenly from school. It took 20 years for us to realize that there were disadvantages. But how is that now with EIA? We are out of chat now, it has not been 3 years yet. How do we look back 20 years? In any case, it is clear that what I described in my book, but also Jodkowski and Bosterman are saying that we don't have super-intelligence, we don't have EGI, we are already thinking about it. So maybe we are in the past in 2060. But at least in 2014-2025, we have already started to think about what the possible risks are. So there seems to be a difference. And there is a world-wide already clear sound system by that kind of consciousness. So there is actually a difference between what we are now thinking about possible problems, a comparison with the iPhone, a comparison with social media, where little or no one has ever thought about it. In any case, there is still a difference whether we are on time or not, but there is a difference. We are already working on it now. And that was not the case for social media, for example. We were not working on it. What was read to concentration to all kinds of problems that we know now around social media, we were not working on it then. We were working on it 20 years later. While there are actually already books written about themselves, like what no one tells us, also JetGPD, not from Lodelaouaart. Lodelaouaart, thank you for talking to us. With great pleasure. Yes, we have got it.