AI-Designed Viruses: A Virologist’s Warning (In Defense of Virology - Episode 5)

[Music] Welcome, everyone, to Science from the French. My name is Bryce Nichols. I'm a professor of genetics at Rutgers University and I'm joined once again by Distinguished Viralogist Simon Wayne Hobson for another episode of "Indefense of Viralogy." Simon, how are you doing today? Very well indeed and happy to talk to you once again about the viruses and AI. Well, that's right. So today we're going to be focusing on something that is on a lot of people's minds, both for its potential good and potential harm, which is artificial intelligence. Simon, you wrote an essay in November for the Bio Safety Now Substack about a paper on AI design of bacterifages. Yes, this I think is, it's the first and it's going to be the first of a long line. So let's get it into discussion. The thing I'll be very blunt, I'm pretty sure it's going to degenerate and therefore it's going to need more attention. But the first this first paper doesn't raise concerned, but it sets the pathway and I'm sure people are going to go along that pathway. So you and I are a biologist and therefore we don't know a thing about all these technical variations and the black box that sometimes people say is AI. So we're totally incompetent in the AI, but we can perceive its impact when it's applied to biology and in my case, biology. So let's go to paper, go came on from a Stanford European Palo Alto and it was put on a pre-print server as of September. Now it's got this word that the me listeners may not know. They talk about bacterifages and they say you consider what the hell's bacteriophage. It's very simple, it's a linguistic oddity. It goes way back to the beginning of the 20th century. So in short, viruses and effect bacteria were called phages. Phages because they eat up the bacteria and they kill them. And then viruses, the word virus is tended to be used for viruses and effect animals and single cell organisms of the callumucarits, but basically plants and animals. And she would have this linguistic dichotomy. Phages for bacteria, the word virus for animals and plants, they are all the same thing. They are in the big family of viruses and this shows you this linguistic plasticity we have in Valor, do which I've mentioned in the past. It's lovely. So we're talking about viruses and viruses in effect bacteria. And there's one particular virus of bacteria that was isolated in a Paris sewer in 1932. So that's what, as I'm in France, I have the pleasure of reminding you about Paris sewers, a full of viruses and bacteria or phages. And this one is a bacteria phages being known. Everyone in molecular biology and people of our generation know this thing called phyx174. It was the first DNA genome to be sequenced and has a special case for those who know all about molecular cloning because it was the thing on which the rapid sequencing methods were generated. So our soul gents have a very soft spot for phyx174. So they basically sort of said, well, how can you design a virus and they went into this very fancy first part explaining how they can make phyras genomes. But basically they were just putting pieces together and it was just fiction. There was no reality. They were taking bits of viruses and saying, oh, we can come up with all sorts of genomes, but these were not genomes. These were just sort of things in sequence space. And then very quickly they got into the serocyte. Well, they said, well, to really see if we can AI can do anything. Let's concentrate on phyx174 like viruses. So they immediately put in a whole series of constraints and basically they whittled it down into a small group of viruses to train on. And we hear in the term I hear in AI as you train your data on something like 5,000 genomes instead of the first part which was basically fantasy. And then they said, okay, they get they whittled it down to 300. And then they said, well, we'll make these 300 genomes. So they do DNA synthesis. And we've mentioned in previous talks that you know, the information is enough for you to be able to make DNA and then make a virus. So they make all these. And the majority are not functional. We don't know what that means. It's a difficult to explain. There was an experiment that doesn't work, but they got 16 viruses that did work. And they then tested and did some experiments on them. And for me, there were two things. One virus grew faster than the phyx174. Now you can say, is that a bad thing? Not necessarily, but you could imagine that this thing could be more efficient at killing, yes, indeed. But for me from all my background in virus evolution, what was amazing that there were six viruses where the virus didn't change. So when you do an experiment, you come up with a design, you are guessing to some extent, you don't know exactly where you're going. And you can expect the virus to say, well, that actually wasn't a particularly good construction. And the thing mutates and gets improves itself. So I would have expected that the majority of those viruses would have accumulated mutations. And the sun did. But what it was amazing that six was stable, genetically stable. And that I wouldn't have predicted. I would not have predicted. If it had one like that, I would have said, gosh, you're lucky. And so when I see six like that, that sort of was probably for me the big surprise that these methods for whatever how they were and they were, can maybe these people can't tell us how they were. They are capable of coming up with things that are pretty solid. They're not little tweaks of imagination. They can come up with features. And that was a proff for me the biggest possible surprise. They they sold there in the discussion of the paper. They sold all sorts of things. But for me, that was the important point. And this is the first attempt. And on the first attempt at helping using AI to design viruses, they had one that grew better and they had six that were genetically stable and didn't accumulate other mutations. So for me, that was that for me, that was a shock. Are you saying this is kind of terrifying in a way of how successful it is already in its infancy? I would know. No, well, I think a good point, good points. The way of using the shock, the shock was for you to get all my experience. I wouldn't have predicted that I would have said, you've got one. Maybe then it was said you were bloody, bloody lucky. They got six. So then I start thinking I was sort of saying, well, you know, these techniques, okay, they just repeated a variant of this Paris page of 1932. But they did it and they got six pretty robust answers and one virus that grew faster. Now, then you start thinking in the second wave because you are shocked that you didn't anticipate. Your shock that the result went beyond what you expected. So this was sort of personal shock. Now, you can say, okay, these are viruses of bacteria. So who cares from a virus safety perspective? In the paper, they sell this as using possible use in treating bacterial infections. The semicool bacterial phage therapy. Say, if you've got a nasty bug in your tummy, your intestine, sorry, you give these phages and you can try and kill that virus in your intestine. This is being looked at by a number of people, large number of groups in front of way of treating severe bowel infections through a front bacteria. But if this or the second study is done on animal virus, we're on a human virus. And they get one virus that grows better or they get six that are just as stable as the one they're looking at. I mean, all my life, we've been working on HIV for 27 years. So HIV is just somewhere here in the front lobes. And you get a virus that grows a bit faster than what. And that's the other shock because this is saying that these, while it's a totally different method, the UNI have ever applied to our biological experiments. This is not something that is, it seems to be giving, was out so I wouldn't have expected. I wouldn't have, I wouldn't have predicted that sort of success. And therefore a few, if this is applied, say to flu viruses or, I don't know, it's just a little bit easier to do that. But it seems to be, it seems to be, therefore, that this technology, even though they themselves don't say, they don't say how it, they don't really know how it works, but they can, they can see the output. Then it's like, if you have a virus, say one virus is better, that's, is that getting a function? The virus grows better. So it could be mutations in a number of genes, but the result is the virus grows better. That was a human virus, therefore it grows better. And generally, generally, viruses that grow better are more dangerous. So if inside you, when you've got disease, we, we know pretty, pretty well that you'll have more virus in you than someone who has got a very mild infection or an asymptomatic infection. Pathology is linked invariably linked to a good first degree to the amount of virus. So if you make a virus that grows faster, that's like getting a function. This comes back to what, this is how even I got talking because we were both highly concerned about getting a function. And as I read on about AI, they said, well, you know, the first attempt is usually pretty poor. The second gets better. And then they learn from errors, they learn from progress. The third iteration is better. The fourth, fifth and sixth. And if I follow that logic, then you would expect this thing, that you applied to animal and human viruses could, could come up with improved which viruses, which is the equivalent of gain of function. So your reaction to this paper was one of concern that, yes, because I, I, I suppose if I'd be being blunt, I would say, I didn't expect that they, even though it's only a virus of bacteria, and we can say who cares about bacteria, what I can say about me, human beings or my dog or my cat. But I, I, so, but I would, I hadn't thought that they would be able to do that. But it wasn't just you that was concerned. No. What's interesting about this paper is it caused a number of interviews from people in the AI world. And to, sorry, to use a strong word, but they are shit scared. They understand the other side. You and I understand the biology side. So when I read an interview and was a by an interview, there was an opinion piece in New York Times from Joshua Benjou, who was one of these brilliant people in AI. He's just for, for people who just want to know how strong this man in here. Apparently he's the most cited scientist, living scientist. So he's not some sort of small player and he's in AI. And he's, he's said that you know, this absolutely terrifies him. And he understands the dangers from the AI side, which I don't, you don't, because we don't understand the technology. And, but we can see the output. And then from our side, from the seeing the viruses, that's when we get concerned. And so the concern is coming from both sides in New York this September. There was something called red lines AI, a group of people discussing, saying that we need red lines in AI. Serious people, lots of concern from organizations and media. The first paragraph saying they're going to have a lot of concerns. And the first concern is the genesis of new pandemics. All the other horrors that might be associated with AI like mass unemployment or, or, um, manipulation of human mind and thought processes and populations, they come off of the first thing they write down is pandemics. And these are people in the AI and you see the people led. There's not a, not many of them. And they move virologist. And they perceive this. And therefore, it seems to me that this is a new technology. And it's, it's going to converge on the things that have been bothering us for the last 10 years, which are microbes. To recap the most recent paper, it was titled, "Generative Design of Novel Bacteria Fages with Genome Language Models." And this is by Samuel King and colleagues at the ARC Institute in Palo Alto, California, which came out on bioarchive in September. It raises additional alarm bells because it shows that AI does a pretty good job at developing synthetic viruses. In this case, these viruses were just, you know, infecting bacteria, these bacteria fages. But as you point out, Simon, this is just, you know, very, very simple, a simple iteration of this. Step one, and we're worried about steps two, three, four, five being catastrophic, especially if you're applied to animal viruses. And the temptation to do that is, is too strong. And it will be done unless, unless, unless, people sort of say, you know, that's off chance. Sorry. So please, let's make an impassioned plea to the researchers. Don't do it. And to the science administrators, because they fund, a scientist can't exist without funding. And so you need to choke the past. If you really, as a scientific administrator, if you feel that this is not the way to go, and the presidential executive order says, you know, you don't make viruses more dangerous at all, or you don't invent new ones, or you don't resurrect old extinct things like 1918 Spanish flu, or whatever. You don't do it. You don't, you just choke the past. And we would hope that people in philanthropic private foundations, gates, foundations, and others who fund biomedical, so do the same. It's not complicated. And then your listeners will know that the far many more complicated issues are on the planet. How can we cope with nuclear waste? How can we do this? And this? How can we solve cancer? I mean, cancer is taking a pummeling, but you everyone knows some with cancer. It's not with. It's keeping, and some cancers are increasing in number. Other cancers are showing up in younger patients than before. It's anything, but this is, there's serious problems in all the neurological and metabolic disease. You know, we've got our as biologists, we've got our work cut out, and we have to get involved and distracted by people making novel viruses. Well, they say curiosity killed the cat. Well, you see the cat, if the cat started getting hurt, he would pull back his paw and sort of said, I shouldn't do that. But healing curiosity says, I know, well, if I do that again, will I get the same result, which is a totally different reflection to the curious cat? The cat will say, oh, I hurt myself. Stop. I'm going to pull away. That's not the how the research works. It's a different mindset. So whether you're cat or a human, let's not get killed with curiosity or kill others. I think we have to figure out a way to update that expression so that I can use it as a catch phrase, and we can sell mug assignments to support what we're trying to do here. Right. Right. What you're saying is we're going to have to do another indefensive viruletry and we'll come up with some catch formula, we can hope we can get across. Right, which communicates accurately what I guess I was trying to say or whatever. The key thing is, I mean, if any listeners have any suggestions, we're receptive. We are. This is not a two-man show. Do know harm is a good one, but I think if we integrate an animal into it, maybe we'll get a better traction. Like cats, dogs, dogs. Oh, dogs. Now now you've got it. Dogs, that's my area. That's my area. Dogs, dogs. I mean, yeah, I want. Yes, right. Fair enough. All right. Well, until until next time, Simon, thank you very much for joining me on Science from the Fringe's In Defense of Viralogy. Take care everyone. [Music]