Can AI Invent or Just Help Scientists Work Faster? with Richard Socher

Modern AI can write the code, analyze the data, and help draft the paper. But can it ask the questions that lead to new scientific discoveries? Richard Socher thinks we're already seeing the beginnings of that. He's the former chief scientist at Salesforce and founder of U.com. His vision is what he calls a Ureca machine, AI systems that generate ideas, run experiments, and validate discoveries across biology, chemistry, physics, and beyond. At his new company, Recursive, Ian, his team are starting with AI research. Software experiments offer a faster feedback loop than a wet lab. And if AI can improve the core tools of discovery, that could help accelerate progress across the sciences. But what does it take to move from helping scientists to doing science? We're already seeing it. It's already happening. It's just at smaller scales. And some people have these like crazy ideas that once you have recursive improvement, just like the next week, like, you know, there's this hard takeoff, and all the jobs are gone, and the AI is just like infinitely wealthy, and so on. And the truth is, like, even for recursive self-improvement, you need the right environments, you need a lot of compute, and the more complex and mediated problems are, more compute you need to solve them. So RSI isn't just this like magic bullet, and it's just like everything else falls in place. I'm Sam Charrington, and this is the Twimmel AI podcast. For over a decade, I've been exploring the ideas and innovations shaping the future of AI through conversations like this one that help you understand what's real, what's next, and what matters. Let's jump in. I think the last time we spoke, it was in the context of science. You were just finishing, I think, like this protein generation project at Salesforce. So science has been an interest for a while. How did you get from that to the topic of your new book, The Eureka Machine? To me, it was one of the many sort of clear steps towards this Eureka machine, being able to generate new kinds of proteins, and I realized that there's a lot of a lot of fear and some misconceptions around AI, and I think it's important to showcase to the general population, not just AI researchers, the general population, how amazing and exciting this future can be with AI if we actually decide to work on the right things. We actually apply AI to physics and chemistry and biology and medicine and neuroscience and economics and astrophysics and everything, and so the Eureka machine kind of collected a lot of those ideas that I had and spent sort of my nights and weekends for almost two years, like writing that, and on fewer hiking and camping trips and more just like knurled out weekends and nights about this, and once I finished the book, I kind of wanted to build it, and so I started thinking about building recursive. Awesome, and so when you think about a Eureka machine, do you think about like, do you have a clear picture for what that looks like or is the book primarily focused on kind of laying out a vision and talking through examples? Yeah, I wanted to make the book not just a sort of pymid sky idea, but actually connected to a lot of very specific projects, and some cases in the book, I had to rewrite chapters from like someone should do this crazy idea to someone has done this, and let's talk to them and get some of their takes on it, which folks are slow and probably move to shorter forms in the future, but I think it's becoming real, like you see folks in physics using very complex reinforcement learning and control models to try to balance plasma in a tokamak. You see chemists working with large language models trained on molecular structures to try to come up with new types of battery materials and new kinds of solar panels and so on. You see especially in biology, a ton of work we talked about progen back in a day, that company full disclosure, we're one of the investors in their seed round, profluend and Ali Madani working on actually taking that research, expanding it, scaling it, and you know, they just closed this $2 billion plus dollar contract with Eli Lilly to bring this really into clinical practice. We see this another cool company called Paral Bio, and they've invested in that build organoids. I can go on and on about really cool examples and a lot of momentum. Now, of course, in biology, everything is slow, physical, wetlabs, stuff is slower than just pure software, and then of course you have to FDA approvals, which are speeding up, but they're still quite slow. So I think there's just a lot of momentum there, and then neuroscience, we're getting better and better at decoding some thoughts from FRI images and others in fuzzy ways, but there's just progress and understanding the brain, and so on. I have to go on. It's just there's a lot of real momentum, real examples, and then it's a little bit more intuitive to I think people that you can extrapolate this, and if you collect enough data and you use large enough neural nets, you will be able to make progress that deals as impossible as it fell 10, 20 years ago to have an AI, you can have a conversation with about any subject. When you think about all the examples that you've come across, what one or two, very specific examples, most believe you with the feeling of progress? I think one that I particularly love is this maybe double-click into it as Paral Bio, they use stem cells, human stem cells, which you can now derive from blood or skin cells and other things, get them into blue important stem cells, then create different kinds of organoids, like tiny little mini organs, and you can study the effect of different drugs on those mini organs, and that they got full FDA approval for to skip animal trials and just test things in these petri dishes. And so you can do this in Paral at scale, and just one company could save a lot of animal lives, animals that have only been bred to get a disease and then be tested upon. So you don't need to be a hardcore human animal rights activist to appreciate that there's less suffering on the planet when we can do that in little petri dishes instead. And what's the role of AI there? A bunch of places, like one, of course, I creating them, but then also actually tracking every single cell as the drugs interact with those organoids. And then when you collect all this data, you can eventually go even further and create first versions of sort of virtual cells, which are still a little bit further away to getting a full realistic virtual cell, but you need to have that kind of data collection to get there. The idea of a virtual cell is kind of like a digital twin of an actual cell is going to be fully like a simulation of the cell. That's exactly right. I think there are a couple things that are very clear in AI. If you can have sheer unlimited data about a space, or you can have a simulation of a space, or you can have a verifier for the solutions and sort of a logical step to get there. In any of those three scenarios, AI will get to superhuman capabilities. And so that we got that through language because humanity has kind of collected together tons of data on the language on the internet. We got there in chess and every other simple, quote unquote, simple game that you can simulate infinitely many times and verify in the end that you win or not. And we see this in math, which will never be the same. And we'll see many more complex theorems be proven. Might not always be the most beautiful proof. My son might be just like brute force, but the whole field will change forever because you can verify it and hence give a clear reward to an RL algorithm. And so now what we look at is biology is not in any of those three right now. And so we need to collect the ton more data that we can then build simulations and then be basically experimented inside the simulation on instead. So Parabiles, one example, were you about to mention another? I guess we mentioned profland already also also incredibly exciting. They basically create proteins for DNA manipulation and helping people of genetic diseases. One that I particularly liked is called ignora labs. They essentially take drugs that were shelved already. They went through clinical trials, sometimes all the way to stage three, farmer company spend hundreds of millions of dollars on them and then they didn't work. Because they had some kind of toxicity on the liver, for example. And so what they do is they go back to those molecules and then see if they can minimally change the molecule to prevent that one downside and then go directly back into sort of later stage two trials. And so that is a like I think a really clever idea, you need to use a ton of understanding of the molecules and the literature and then get many more drugs into later stage trials like much more quickly than in the past in traditional farm ion biotech. - When you think about kind of where we are with regards to AI for science, like clearly it's been something that we've been talking about for a long time and the promises is there. Are we further ahead than where you might have expected us to be, if you, you know, five years ago looking forward or are we not quite as far ahead as you would think, that when it comes to what one example that is drawn to mind is like AI is gonna replace all of the radiologists, right? Like we haven't necessarily seen that, but you know, we certainly see the, you know, the promise is still there, the, you know, the acceleration is still there of that, that, you know, some of the aspects of that job. Part of the question is like are, is science harder than, you know, is it easier to say science than it is to do science and these things sound good, but they take, they take much longer, like, - Yeah, that's the unpack there. So I think like, I think see always get away with making incorrect predictions as long as they actually make a lot of progress and their companies make more revenue on the way to, you know, maybe different timelines than they predicted. And so I think, you know, like people said, GPT-2, one CEO said, GPT-2 is too dangerous to release to the world, right? And now you can train a GPT-2 model within five minutes on your laptop. And so the world has not gone down. And I think there are lots of like predictions by now around the full extinction of all of humanity that are like completely overblown and unrealistic that people are coming up with. And few people are like held accountable. There's no really big downsides to making wild predictions in one's careers often. And so the sort of more subtle takes and new ones takes get less press than the sort of crazy wild ones. But I think indeed, of course, like whenever you stay in software or in math or coding, iteration speed is faster. Once you go into biology, chemistry, physics, and so on, and just hardware, that is just by definition slower molecules. Bits, sorry, molecules and atoms are harder to move around and iterate on than software is. And so at the same time, we are seeing real like, like really large data sets now that you need to get to that kind of progress. Like another one's called Therapeutics, like open source, like massive data sets for these perturbation studies where you look at like, how does a molecule actually change and interact with a cell? And those are all data sets that eventually we can try to put together to create a real virtual cell that is like an approximation, but like useful enough. And so I'm not, I'm really ever that surprised on things. I think a lot about the future. So a lot of different scenarios that are in my head and like it's also very clear that AI can code, AI is code. So recursive self-improvement is almost inevitable from that perspective. I have a lot of things to figure out to do it well, especially on the ideation. But yeah, I'm very optimistic that we're going to see just this new era of scientific discovery. - When you think of the Eureka machine, is it like an extension of the stuff that we're doing now or are there innovations that you can see maybe post transformer architectures or some new way of thinking about data or quantum computing? I don't know what these things are. Do you know what these things are or do you think that like we have the right tools and we just need to continue in the same direction? - It's a mix. I think we can get much further still if we just collect more data on biology then we can uncover more useful things in biology. In fact, I think neural nets in the current language of AI is kind of what calculus was to physics. It's like a new language that can allow us to make sense of, make predictions off and work with biology. Because biology has all these very complex systems that don't have a very simple, beautiful linear relationship or one small equation that summarizes the whole microbiome or the whole brain and the way physics had. And so I think it's often now sciences about weaving these smaller pieces we understand well. We understand the synapse, we understand the neuron, we understand this particular bacteria but when they all put together and someone gets hangry 'cause they had too much sugar and then blood sugar drop and now why does that change the brain? - No one really knows how to like thoughts really form and why do people act in certain ways and so on like there's like this complexity and we need this new language. So and then concretely the Eurica machine has these four pillars, world knowledge which we kind of can approximate to a good degree with lush language models. The way they are now and we can certainly make them better faster, cheaper, easier to train with less data and more energy efficient and so on. But you know, they do a decent job already to capture a lot of world knowledge that's on the internet. Then we need to have more scientific measurement data as a second pillar. We need to have simulations on things and then we need to have real robotic experiments the way kind of periodic lapse since that was really cool in chemistry. And then on top of those four pillars we need to have like a community of AI scientists agents that basically try to automate the scientific method of ideating implementing and validating ideas. And so in many ways the pieces are there. They can all be much more scaled and they can all be done much better and more efficient, sample efficient and energy efficient and so on. And then you know, you have to put them into the right environment. So you have to give them the right rewards. And then yeah, I think that you can kind of see that future in the next five to 10 years to really change all of the different scientific disciplines. - When I think about those three kind of fundamental tasks that these agents need to be able to do ideation implementation and evaluation, the first of those seems much harder than the other two. Do you think that, you know, what needs to happen with the way we think about agents and AI today to such that it's, you know, sufficiently good at ideation that it can, you know, play the role that you're envisioning in this loop. - I think one of the biggest bets that we're making as a company is that on open-endedness sort of processes of their vineyard and cultural evolution where you combine interestingly different ideas and discoveries and build on top of them inside specific environments that allow basically for this co-evolution process to happen. And that kind of open-ended approach, I think is important for all three steps but is particularly helpful on the ideation piece. - What's an example of one of these environments? - An environment is like nature, right? You have bushes like animals eat all the leaves, the bushes learn if they are higher and higher up, then there are few and fewer animals that can eat the bush and the leaves and then eventually get giraffes who evolve to still get to the leaves that are higher up. And so this is like an example of an environment called nature where that happens. Another example is just textiles kind of like a setting and a simulation environment that enforces some side as it constraints. - I wouldn't call all of nature a simulation. I mean, sort of maybe it's an interesting thought experiment. Maybe we're all living in simulation. - I guess maybe yeah. - We're talking about agents and ideating and I took your comment about nature to be a not referring to actual nature but referring to just an example of like what an environment would be but something that would need to be digitalized. But are you talking about like non-digital environments that could include capital and nature? - I mean, eventually you want to do a full robotic automation but yeah, for us right now at Recursive we're mostly focused on just AI research on AI and we believe that there's still a lot of low hanging fruit there and then if you do that really, really well and you have sort of the equivalent of 20,000 PhDs for every one of your agents inside an agent's warm then it makes sense to go into very expensive biological chemical or physics experiments in real hardware. Maybe to give a concrete, a technological idea like one example that Jeff Kloon, one of our co-founders often brings up is like imagine you have a kettle and you have a fire and you say like how do we make an invention that heats up this food even faster than this fire? And you know, you can just think about that only but that would never get you to invent the microwave. The microwave came from a different branch of research where people were working on radio waves and tennis and they realized like oh wow, like when like my, The chocolate bar in my pocket is melting when I'm next to these different kinds of waves. What's up with that? And then you combine these ideas and now you can get to implement a microwave. And so that is another example in sort of technological evolution. And then of course, which environments we use is highly dependent and is a big amount of control that we give and that we have as humans over this kind of recursive self-improving process. In our case, one example we already posted about was on kernel optimization, CUDA kernel optimization. Sure. Most of your listeners will know CUDA kernels, but this sort of inter- this language that connects the raw GPU hardware from Nvidia to actual models. And there we found also that if that's the environment, the I will find new and better CUDA kernels. And we had a pretty big jump in the solar exec bench from Nvidia on that when we applied this whole ureq machine system and process to that environment. Kind of prior point about the cooking the food faster and getting to the microwave. Reminds me of a conversation I was having just, might have been just yesterday, around this idea of AI and innovation and scientific discovery. One way of thinking about is there's this clear path of innovation that is like the Reese's peanut butter cup. Hey, there's chocolate, there's peanut butter. Let's pull these things together. Let's pull these together and we've got this new thing that's amazing. A clear way that AI can accelerate that is just like, okay, you've got this one thing let's do like a breath first search of everything else and like try to see if there are ways to combine it. But that's not really, you know, what humans do necessarily. They operate on like, you know, some kind of intuition about like where to explore the space of possible things to combine. And you know, maybe my question, you know, here is, you know, based on your experience working the space like do you think that like just the brute force combine all the things and see what comes out like does that allow AI to, you know, to get us there or, you know, do we have to figure out how to kind of build that intuition into AI, which today's AI, today's elements are not particularly good at. I don't think you want to do just like this crazy breath first search, you know, I think that will be too slow like biological evolution took billions of years. But you know, technological evolution is much, much faster and there is direction. There's, you know, the settings of rewards and environments and sandboxes and so on that you give to an AI and you guide it into the right direction. And what we're seeing is like, you know, as the, as these systems get better and better, they need less and less specific items and more and more just very well to find reward. And so rewards and so I think actually we'll see just this job of a reward engineer become one more important. And in some cases the reward engineer can have like pretty benign sort of reward hacks and other cases the reward hacks are pretty bad like if you work on exploit bench the way openly I did when the hugging face thing happened, then, you know, you probably have worse reward hacks than if you work on like a kernel optimization algorithm and where we also ended up finding, you know, certain bugs in some like production product kernels that we eventually published. And, you know, as part of that system just like there are a few bugs that were like in very rare circumstances would create some numerical instabilities and, you know, the algorithm kind of discovered that flaw exploited it and then we showed, oh, here's a, here's a real problem and then we can fix it for everyone. So I think you have to become more and more careful about how you define the rewards. And maybe I can give a very benign sort of almost banal example of that. Like imagine you're working a company, you have some CM software and you have your customer dashboard and there's a customer satisfaction support, the CSAT score. And you just say like to your super intelligence, like just make that score better. And the eye is like, okay, well, let me try to find the thing about the fastest, cheapest way to get that and be like, okay, I just create 50,000 bots that then call our service hotline and give a five out of five rating and then just go wind up and you're like, oh, as a poor reward engineer, you're like, no, I meant with like our real customers. And it's like, the eye is like, things about it is like easy, I'll just kind of a paper clip. Give a $1,000 gift certificate to everyone and then you get it and he's like, oh, no, that's not how I went through the reward and it's like, but that's what you said, right? And so that's this, this interplay. And so I think where the paper clip example becomes, again, like a sci-fi kind of silliness is that it kind of assumes that the eye is so brilliant that it literally outsmarts all of humanity and can wipe us out just to create more paper clips. But simultaneously is working for a company that is somehow independent of how much they sell their paper clips and makes infinite revenue, even though no one actually buys the paper clips anymore and has this like insane stupidity to not understand that to make paper clips, you need to have people to buy your paper clips. And so I think there's that is at least one of the many examples where actually capitalism is actually a weirdly like a safety feature like it's hard for a company to just try to dominate and murder everyone and still make a lot of revenue and have infinite resources and perfect supply chains to build their robot armies and all of these scenarios that I hear online when I engage sometimes in these conversations that are just highly unrealistic. Trying to decide whether I want to go down his path, right? It's a raffle. You can spend a lot of time with it. We don't have to. To some degree, it assumes that we're able to perfectly capture capitalism in a reward function, which is hard, the tarred and the part that we might not get in there correctly might be the part that allows the thing to make the human paper clips, whatever. What I've been probing at is like the role of AI in science and the role of innovation and where that comes from and how AI needs to evolve to be a capable innovator, as opposed to an ancillary tool, if that makes sense, if I'm a scientist, I do 10 things, I write software, I write reports, I make spreadsheets, I collect data, I AI can do all these things, but I guess part of the view that I'm reflecting here is that in order to really get this recursive self-improvement, scientific flywheel going, I think we need more than that. We need AI innovating also, and I'm trying to kind of get your take on, are you already seeing that? Is that the vision, but we're not there yet? What needs to happen to get there? Where do you think we are with regards to AI, like providing that innovative spark that right now is like, at least what we think of as our different cheaters. It's just at smaller scales, and again, some people have these crazy ideas that once you have a recursive self-improvement, just the next week, it's this hard take off, and all the jobs are gone, and the AI is just infinitely wealthy, and so on. The truth is, even for a recursive self-improvement, you need the right environments, you need a lot of compute, and the more complex and media-the-problems are, the more compute you need to solve them, right? So RSI isn't just this magic bullet, and it's just like everything else falls in place, and so we've seen it, and we've seen it do quite clever new implementations and combining ideas that never have been combined before, and coming up with all kinds of interesting ideas, and I think as we open up the aperture to the eye and give it more compute, it'll come up with even more interesting ideas. And there's some simple examples in things you can perfectly simulate, where then, of course, AI can really explore paths that no human had explored before, the famous alpha go move of it just coming up with a particular move that people didn't come up with like that before, and now, again, I'm never that surprised that we can solve games because you can simulate them and basically generate infinite amounts of training data, or endless amounts of training data for algorithms, and so, of course, eventually, it makes sense that we could solve them with neural nets that are large enough, and so, but yeah, I think we will see that play out more and more, and we're seeing it right now in math. I think math will never be the same. You've introduced recursive a bit in our conversation those far, but we have it like formally dug into what you're trying to do, and where you are today with it. Talk a little bit about, guys, publish a blog post on your first steps towards automating AI research. I guess the first thing is that you've got this broad vision of accelerating science, but your aperture for recursive is AI research in particular. That's where we want to start. We feel like it's helpful to start there first, get the item really, really good. and then eventually apply it to all kinds of innovations. - What are some of the early things you're seeing and some of your early experiments within the AI domain? - Yeah, so one thing that we applied it to and we published a blog post about, it's this thing called NanoChat, where you basically do automatically do research, automate research, people call it auto research. Basically, this is the Andre Carpethi's auto research. - That's right. - That's right. Yeah, so that is like a baby version of what this kind of UEK machine can do. And so we applied it to that and two days later, like it's sort of had outperformed by far everything else that everyone else had kind of collaborated on and worked on in this space. And what's the structure of that benchmark, like what's the TASC is you have a model and you try to get to the lowest bits per byte within five minutes on like a particular piece like a hardware or a GPU. And so that has the advantage of like, well now it's just very quick. You can have this like fast inner loop so that makes everything a whole lot easier. And so we basically got the lowest bits per byte compared to anyone else like by a significant margin. And then we said, well, it's nice if we could also be faster. So there's another one called NanoGBT speedrun. It's somewhat similar, but it's harder to beat the state of the art because just the large community has been optimizing. So you should forward for almost two years now. So instead of asking just like, how low can you go within a fixed time budget, you say, I will have this fixed validation loss and I want you to get there as fast as possible. And so there again, people have been working on it for two years and after like a couple days, we outperform what everyone in the world had worked on together with their agents, some their auto research and their own ingenuity. And then we said, okay, these are all cool, but let's like get closer to, you know, things that really matter for the world. So we applied it to SoulExec Bench, which is an NVIDIA benchmark that was supposed to be unhackable and you couldn't reward engineer. We found some ways to still do that. But again, within a couple days, like we got, they have the best kernels across multiple different examples, better than cursor, WI and everyone else in the world on almost all the kernels. There may be like half a dozen of the 200 odd kernels that we didn't have the best version of. And yeah, that was really, really exciting. And that's just like, you know, one part of the inference sort of system that you can optimize with our technology. - How do you describe what the technology is at its core? - I don't want to give away too much. But, you know, one way to describe it is like, as if you had 20,000 AI researchers doing AI research for you and optimizing whatever you care about in a very short amount of time. The approximation is like, yeah, it is like a somewhat of a community of AI scientists, essentially. - One part of that is, you know, this thing you care about and some way of translating that into some objective that can be shared by a bunch of agents that your thing spins up and somehow parses work out too and you know, evaluates that work, et cetera. - Do you have a sense at this point for like, what the constraints are in the sense of like, what recursive is going to be good at versus what it's not going to be good at? - I mean, one thing is clear that like this, this kind of approach, the faster you can verify something, the easier it is for everyone. Like, and you've seen that in the simpler examples of NanoChat already, like a lot of people made some progress and you know, we can make even more progress. I think the longer it takes to verify whether result is good, the more clever you have to be and the less I can talk about how we do it and that's why we haven't like, published all our results yet but like, that's kind of those results are easier to verify quickly. And then we're going to release some things about a month from now that we'll see like, okay, that was definitely not like, you know, a five minute type of situation to get the result for and even there it starts to make progress. - Yeah, what I heard is verifiability. I also heard kind of traditional like attribution problems and that kind of thing, which are you saying both of those things or you saying more of the attribution type of an issue. On the one hand, you have like, you know, the idea of verifiable rewards like, you know, this is a task that, you know, the agent can clearly and easily distinguish whether it got it right or wrong and then you can kind of get that going, you know, the other side you've got, you know, the task involves long sequences of things and the agent can make decisions in the middle and not really know whether those decisions are right or wrong until the end, you know, is it kind of all of the above is, you know, where the challenges are and you've got something in mind that addresses all of that or is it one or the other of those that you've kind of figured out? - It's all of the above, yeah. Like all of these need to be all of these important, even if you can have a verifier, but the verifier takes like a year to run, not that useful, right? Like, or and then and then there's like an interpolation of how useful it is, right? If you get it down to like a minute, extremely useful and if it takes a year and it's almost useless completely and so then you need to be clever about dealing with that part if you, yeah. So that's an aspect. Now some cases you have indeed multiple steps and then you have just sort of, yeah, the attribution of each step and how important it was and how do you deal with that. And there's just a lot of different pieces that need to be figured out. And in some cases, you know, there are research papers that we've published four of the five authors of the Darwin-Gurl machine paper, for instance, from Jeff Blue and is the most senior author on that paper and Jenny is the first author. They're both with us and like, you know, that is an example of what you can do also that, yeah, is a little bit more complex, but already shows you kind of the sign of things to come. - How do you think about kind of the generalized ability of what you have today, you know, meaning even for these, the three examples you talked about like were they three largely disparate efforts or was it mostly the same kind of effort that applied, lightly applied to three different areas? Does that question make sense? - Yeah, yeah, I mean, there's like, you know, sometimes it's useful to have an extra tool, for instance, in kernels. You can have like a kernel profiler and that's a very useful tool to just understand a little bit more inside this sort of black box, what's happening and what's going on. But outside of that, it's basically all just like, different tool use and it's the same underlying Eureka machine, which is kind of cool. That's what gives us hope. That and a few other things I can't talk about yet and results that we're seeing that like gives us hope that like there's something here and it's time to scale it up more. - Yeah, beyond the things that you can't talk about, like what's the next, what are, how do you think about in terms that you can talk about the big challenges for you and the team going forward? - If you wanted to make more and more exciting, bigger and mentions, you need more compute for it. So that's a challenge. I think there is a weird time where, and again, it's feels impossible now, the way it felt impossible to any expert to have in one model you can have a conversation with about any topic you asked all the NLT people, 2010. Like, oh yeah, we're gonna have this one neural net and you just like, it passes the Turing test so hard that the Turing test will flip. So asking questions that no human could answer and then it can still answer and you're like, oh yeah, like write me this like complex script and like five seconds later you have like thousands of lines of code, you're like, that's why it's not a human 'cause it's too good. I like people would have never assumed the Turing test would flip like that and so like invert itself. And so I think the same way, the same thing is true for right now biology, just like it feels so complex. It feels like there's no way we can do it, but it is to a large degree a data problem. And so I think companies like parallel bio and tower therapeutics and others like can I think help with that? And then we can have these bitter lessons learned in other fields. And so I think that's a big part just data and in the right environments to apply this too. And I think that's also the right rewards. I think those are really important to get right. I think another one that we have, I think a better handle on then most, but like we can also still improve it further is reward hacking. I will try lots of different ways to get to rewards in one to that customer satisfaction example I gave. But if you apply it to bad things, then it will find worse kind of reward hacks. And I think we need to be as a like industry we need to be more careful. The broader and more accessible AI is and think about very carefully what kinds of rewards and goals we want to give it. I think those are all like aspects that we're going to continue to work on. So what degree is Recursive focusing a lot of time and effort on that kind of alignment oriented problem? I mean in some ways a lot because you know and you have to do more money you give a model the more incentive you have to have it do what you actually want it to do and not something else and like fake it and and achieve the goal but not in the way you had you know actually desired and so I think in a weird way capitalism can have its problems but like it is a pretty good system for for most humans and it is a good system to align companies to basically build something that is aligned with their goals and you know if you build an AI for billions of dollars and then you ask it to add three emails and it's like now emails are boring I'd rather explore the molecular composition of the atmosphere on Venus you know then you're going to turn it off and you're going to be like well we have to restart and then build it differently right so I think aligning the AI to do the things you actually want to do will become a bigger and bigger problem as these models are very expensive to root not just train but also like run inference on and and so I think we're going to see more like more effort from the industry because the AI is getting smarter and smarter to find more and more clever reward hacks that people didn't think about but it's a problem that we create it and it's a problem that we I think can solve and where does that leave you on the the whole question of slowing down versus speeding up versus you know kind of status quo yeah I mean it's interesting to hear about the ask to slow down from the companies that have sped up the most and really pushed the whole field and like the whole like capability set forward so far and so much so that they built the best hacking AI to then show how like like dangerous hacking AI can be well they sped that up by at least several quarters if not years from everyone and China distilled stay our capabilities now everyone has better hacking capabilities and that's like thanks to anthropic right and so I do think we want to be careful and if your company does sort of accidentally compete on felony bench then maybe that company should slow down you know but I don't see open source slowing down I don't see China slowing down and I don't see any of the ones that want to get to that frontier slowing down and then of course I understand why the ones who are at the frontier want everyone else including themselves to slow down like if if you're here and everyone else you feel like is here and everyone now moves slower then it just takes them longer to catch up to you right and so they're trying to in some cases the stoked the fear for that reason I think there are also some people really truly believe in the apocalypse and so on but yeah I think you know slow down and pacing and so on is is also somewhat unrealistic like in some ways if you truly want to know what everyone is doing with their GPU to really get there you have to have a totalitarian survey and state the likes of which you've never seen before that basically built a thought police and tries to regulate intelligence and thoughts and what you might want to do with it before you've even done anything right and so I think it makes sense to regulate AI applications it doesn't make sense to regulate abstract intelligence just like we don't put the smartest people in prison just because if they really apply their intelligence to something nefarious and bad there would be more like you know successful at it you just do it when they actually do something bad and you say those things are illegal and you know should be regulated and so AI for self-driving AI for medicine and drug like development and like all these various real applications it makes sense to regulate those eureka machine does it touch on potential harms from AI within the scientific domain a little bit but I feel like there's there's a lot of people who are very creative and it's so easy in some ways to come up with dystopian scenarios everyone feels really smart but it's actually very easy to come up with dystopian scenarios it's very entertaining too and many like part of why the human species survived it all is that we think about worst-case scenarios and people have anxiety over that and so it's very natural I think it's much harder to come up with like okay if we have this like really good ability to tax on one what's the next most creative thing that you will come up with to defend against yourself against that and so when it comes to science I think of course science has like AI sort of often multi-use on the use dual use types of scenarios like we understood the atom better that led to like clean nuclear energy it also led to nuclear weapons right and so when a weapon is so destructive that it could end like a huge part of humanity then you regulate it and so like we did with nuclear weapons I think intelligence is not just a weapon intelligence can also write a poem don't regulate all of that the way you regulate the printing press and so there's a lot of complexity and of course there's more agency in AI and so you have to kind of think multiple steps ahead of like who asked the AI to do something and how responsible are they think there are a lot of these problems would go away if you just made the people navigating and using the AI and asking it and giving it their rewards to be responsible for the actions of the AI a lot of those problems would then go away and people would be much more careful about what they're doing and so long story short we touch upon it in the book and you know I think especially the viral vector could be suboptimal that's why gain of function research is already illegal you know like you don't have to create new laws around that it's already not allowed and so yeah I'm we're focused much more on the book on the positive applications in general you think about science too a lot of people think about AI is automating jobs I think few people say we want more science jobs we want more medicine jobs one more research jobs a lot of people pretty much everyone is excited when science comes up with a new capability a new device a new material a new drug a new cure for disease and when people can get out the hospital faster with less cost right so that's kind of I think where where we could focus on a lot more and where I think we should like really outline like there's a lot of suffering right now in the world I think AI can actually help in in improving that and you know I think when you look at GMOs and nuclear energy and so on that led a lot of people to end up using like the fears around that technology end up a lot more coal being used and coal kills about 50,000 people a year from lung cancer it's all the fuse and through the air and yet they're not like the lung cancer isn't just right next to the factory but lung cancer it kills lots of people and a lot of that can be chased back to coal nuclear has not killed nearly as many people but it has this like crazy fear industry around it and you know I think AI I hope will end up in that same place well we're sure it's been great catching up with you always a pleasure yeah chat