Training General Robots for Any Task: Physical Intelligence’s Karol Hausman and Tobi Springenberg

The Mind-Blowing Potential of General-Purpose AI Learning Just like the fact that this whole thing works, it's kind of mind blowing. Yeah, right. Like you, you build this like loosely brain inspired thing that has very general purpose learning algorithm. You feed the data and it somehow gets it and gets it way better than anything we've ever had before. And this applies to robots and it applies to vision and language and sound and all kinds of other things. And like, I think if you stop for a second and just think about it, how it works and and that it works, it's just like absolutely mind blowing. Speaker 2 In this episode, we sit down with Carol and Toby of Physical Intelligence, a company building foundation models for robotics. Carolyn, Toby explain why the classical approach of breaking robotics down into perception, planning, and control was fundamentally wrong, and how end to end learning with reinforcement learning is finally making deployment possible. You'll hear how they achieved robust real world performance, getting robots to make coffee for 13 hours straight, and how these models generalize across radically different tasks, from surgical robots to drone flying in ways that we don't fully understand. We also talk about the technical insights behind π Star 0.6, which is Physical Intelligence's newest model that learns from experience using reinforcement learning. Enjoy the show, Carl, Toby, thank you so much for joining us here today. Thank you for having us. Excited to talk everything, Physical Intelligence, general robotics, etcetera. Maybe before we get into it, just for audience, can you share a little bit about what Physical Intelligence is and the mission that you're after? Speaker 1 Yeah. So at Physical Intelligence, we are building robotic foundation models. These are models that in principle should have, should be able to have any robot do any task. And over the past 1 1/2 years or so, we started building the we, we, we created the right building blocks that show how these models could scale. So we've shown that they're able to control many different robotic form factors, many different types of robots. We've also shown that they're able to generalize. So you can bring it to completely new environments and what it takes for them to generalize. And this, this last release that we just had called π Star O Six that we also wanted to tell you more about shows how we can get them to good performance so that they're starting to become deployable. And this is really important to us because we want to see this technology actually deployed in the real world, but also because we don't have the benefit of having the the free data on the Internet. There is no data of robot actions, so we need to create the datasets ourselves. So we are after the the problem of physical intelligence, after the problem of creating foundation models for robots. And we've made quite a lot of progress. Speaker 2 Wonderful. Addressing Robotics' Intelligence Bottleneck with Foundation Models And can I ask why the decision to build foundation models as opposed to, you know, there are companies that are building fully vertically integrated robotic products right now? You, you know, the, the Sunday lunch last month is in the back of my head. You can buy a cute little robot helper for your household. There's companies working on cooking robots. There's obviously the humanoid companies. Why build a foundation model versus build, build a robot yourselves? Yeah. Speaker 1 So I think if you look at the history of robotics, it's very, very clear to me and I think to many roboticists that we've been always bottlenecked on intelligence. We've had robots that are capable of doing incredible things, whether it's in the home or in industrial settings. We've seen robots more than a decade ago that Iftel operated. They can clean the entire house. And the, the, the really important caveat is Iftel operated. So if there is a human mind behind it, it's clear that the hardware is capable of doing lots of different things. And for a very long time, robotics companies have been structured the way you described where you kind of think of creating a specific robot that's designed to do just a single task or a single application. And instead what we thought would be would really help the field is to focus on the bottleneck, on the intelligence. So we created a company to focus on that bottleneck because we think that this is, if we address that bottleneck, we can actually make robots happen. And if you do it any other way, you're basically not making as much progress on the bottleneck as you could be. So we thought we would just target this problem head on, focus on the intelligence. And if we can do that, that would lead to many different vertical products. It will lead to, you know, robots being deployed in the home in in industrial settings, basically anywhere. Speaker 2 Can I just pressure that, test that a little bit? So on the hardware side, like I've seen the latest videos, for example, of the the optimist hand it, it's like it's exquisite. It's a it's a piece of art. And I hadn't seen the videos of people, you know, teleoperated robots cleaning houses 10 years ago. But I'm wondering if there's a set of tasks that's, you know, maybe now just on the cusp of becoming possible, for example, cooking or like being able to, you know, peel and dice an onion that like you couldn't have done with with hardware prior to where we currently are. So how much of AY now do you think hardware is or isn't? Speaker 1 So there's a lot of progress in hardware, especially in humanoid hardware like dextrous hands, for instance, as you mentioned, they're that that they, I think they're much better now than they were even a few years ago. Yeah, but it still doesn't address the bottleneck. We could have had robots operating, you know, chopping vegetables or doing cooking even with simple grippers before. The problem is that we don't have the intelligence to operate these robots and the more complex the hardware is it it doesn't really result that bottleneck, right? Like it, it allows you to do more, potentially, but you're still bottlenecked by the fundamental challenge of robots not being intelligent, you know? I see. Speaker 2 So hardware may raise the ceiling on what you're able to do, but like the the capability floor, we're not even there yet. Speaker 1 That's right. So even with simple robots, we are not yet at the level of a human operator. The Three Pillars: Capability, Generalization, and Performance in Robotics So the limit being the intelligence layer, what's the limit to developing the intelligence? Is that collecting data? Is it doing it cheaply because, you know, you've broken down the problem? We're going to keep asking you why, why, why, and just drill down further. So what's the next layer of the OK? What's the bottleneck for solving intelligence generalization? Speaker 1 It's a, it's a good question. So we thought about it in terms of three factors. We refer to them as capability, generalization and performance. With capability, our, our idea was that we want to get to the point where as long as you can collect data for something, for a task or for a robot, you should have a model that should be able to to replicate that to automate that task. This is something that we've gotten to fairly quickly. This was our PI0 release around a year ago or so showing that it's basically possible that if you can collect data for any task for any robot, you should be able to automate it and the model should be able to learn that. The next challenge is around generalization. And this is still an open challenge. So we wanted to get to the point where the robots can just work zero shot and you can just bring them to a new home, for instance, and they should know how to operate in that home. And this is a really, really difficult problem, right? Like if you if you put the robot in a new home, it needs to understand, you know, where different items are the the counters look different, the lighting is different that what you've seen in the past and so on. And I wouldn't say that this problem is solved, but I think we start to get a handle on how to, how to solve it and how it scales. And the only answer to generalization that we know in machine learning is through diversity of data. So if you see a lot of different diverse data sets, you should be able to generalize to, to a setting that's similar to the one you've seen. And this is something that we've seen of our Pi 05 release in April of this year that we, we got to the point where we can bring a robot to a new home that has never been to before. And it's able to, to, to operate in that home. It's not perfect yet, but at least it has some kind of common sense on how to go about simple tasks like cleaning up the kitchen and things like that. And then the last challenge of this also not fully solved yet, this performance. So how can we get these models to the point where the performance is good enough so you can actually deploy them, right? And deployments here are really, really important because as I mentioned before, we also need to gather data. And I think that is going to going to be the most scalable way of collecting data because you'll have robots out there in the world doing economically valuable tasks. And that way that the cost of that data collection is basically negative. And the more, the more broadly you can deploy this technology, the more data you'll be getting. And I think in the limit that will be, that will be the biggest source of the data you can imagine, much bigger than Internet data, for instance and. Speaker 3 How far away do you think we are from generalization or from a performance level that maybe it's a controlled environment, maybe it's a general environment in homes or offices, but not the whole whole world. If you could limit that, what what do you think generalization performance will will need to be before we can deploy these kind of robots? Speaker 1 I think we are actually fairly close to deploying these robots. We started deploying them ourselves already. We thought this was something that was gonna take something like 5 years to get to the point where the technology's actually ready to deploy a robot in a, in a commercial setting and have it do something valuable. But we've done it I think two months ago or something like that. So I think we're now getting to that threshold that, that the models are useful enough, they're perform performant enough, and they can do enough of variety of tasks to be actually useful. So that's a really, really exciting moment. We, I think we just crossed our threshold. I think it's still to be determined how wide is the aperture of where we can deploy. You know, there are some tasks where the failure can be really catastrophic. Maybe these are not the best tests to deploy just yet. There's some tasks that require a ton of generalization like deploying in the homes or that are, you know, have privacy concerns or safety concerns and so on. Maybe these are not the best places to deploy just yet, but I think there is that the aperture is growing as we collect more data, as these models get better, we can deploy them in more and more settings. So I think we're we're starting to get there. Speaker 3 Where's the current aperture that you're deploying right now? Speaker 1 So we're actually, this is, this is a really difficult question to answer because with these foundation models, sometimes you don't fully know. So kind of similarly to, to how with large language models, you know, you, you train this model, you kind of cook it in house, you try to make the best job possible. And then at the very end you get this artifact and you can't really predict how good the artifact is going to be. You kind of have to test it. And that's where we are with these models as well. So for instance, we open source them so that we are not the only ones testing it and we're not the bottlenecking, knowing what their capabilities are. And by open sourcing them, we see them being applied to actually many more applications that we could have imagined things like driving or surgical robots or agriculture and, and, and places like that. So I don't have a very good estimate of what the aperture is. I think it's wider than what I had expected. And I think it's also will be, it will be growing over time. The more data these models get, the more mature they get. I mean, the aperture will continue to grow. Speaker 4 I would add maybe like on the performance of like as you said, the opportunity is probably wider, the starting point is wider than we thought. But at the same time, of course, if you actually want each of those starting points that you start out for each of those applications to be at a level where people would want to use this as a day-to-day driving, you know their businesses. That's probably still quite a bit of hill climbing to do in terms of performance, right? So we've with this reason that we're going to talk about a little bit in a bit. I guess the π *6 we've made progress on like learning from experience data, getting back and making the models better when they are deployed. It's still for a lot of things that I can naively imagine that that'd be lots of scenarios where there's a really, really long tail of things that can go wrong or that you can encounter that we we don't yet have a great grasp on, like how to completely solve, I would say as well. Speaker 2 And you guys have been really great about publishing your results with a lot of transparency, releasing an open source. So whatever you're comfortable sharing. Vision-Language-Action Models: Our Core Robotics Architecture Can you can you talk about what your overall technical architecture, so to speak, is? And do you think that that the architecture is to kind of get to this promised land is, you know, pretty much baked and it'll be variations on the theme of where we are and we just need to collect a ton of data? Or do you think that you know the architecture is still still being figured out? Speaker 4 I would say so we, we can maybe start with like a little bit discussing like where we are at now and then we can like go into the details of like how that might change. So at the moment, you know, the, the architecture is very analogous to how, you know, VLMS and are are built today's that probably, you know, most of you interact with on a day-to-day basis, right? Type something in and put the image in and ask it to read what's on the image and and so on. And, and we've kind of like started from the same standpoint of, you know, there's a model that's trained scale data and it's ingested image data and text. And we're adding all this robotics data. And our training actually predominantly now is on robotics data, on data that we have collected ourselves. We have a little bit of that Internet data in the mix, but the majority of it is robotics data. The architecture is kind of this vision language model and we add something on the side, which is what we call the action model, the action expert. The, the part of the model that actually then has to drive the robot, right, That basically looks at the image and the instruction is getting and, and has to perform the task, has to send commands to the robot. And so broadly it's like a transformer model that is a fairly large model up to like some billion parameters at this point that we use that we pre train on our robotics data and on on Internet data. And it is trained largely in initially from human demonstration data. Carl mentioned this earlier a little bit and we we have this demonstration data to operate the data of humans trying to get the robot to do stuff. So that's the, the architecture that looks like now and like roughly the skating that we're getting is from is getting our data and we use models similar to what comes from the VLM world. How that might change, I think as an, as an open question, I, I think there's lots of opportunities in, in adding more capabilities to these models that we're also exploring, right? You can imagine that, you know, you, you might want more context in these models. You might want more, more, more cameras added to, to the robots that the model that needs to be able to, to use. You might want to have a better understanding of the physical world in the sense of, you know, understanding exactly what's in the room, what can break, what is easily movable and so on. So there's lots to be done, I think in those capabilities and also changing the architecture around. And I, I wouldn't be surprised if in like 5-6 years, we look back and we say, oh, you know, maybe the, the backbone of the model that we used at the time, which currently is, comes from this VLAN land has changed. Maybe we've moved on and we, we use something slightly different. I think that will evolve over time, but I think the the foundation of like the data and how we bring it into the model, but probably stay stay. Speaker 3 Like this got. Speaker 2 It and should I think about it as it's pixels or signals in and then actions out? Is that like a single single big neural net? Speaker 4 It's one big model, yeah. It's really just basically images in, text in, text out, and actions out at this point, yeah. The Shift to End-to-End Learning in Robotics Development And are you, I guess, do you have a separate kind of locomotion versus manipulation stack and maybe this might be a good time to talk about kind of just the historical evolution in robotics and the various different waves of learning and how it pertains to your stack? Speaker 1 Yeah. So for a long time, even before learning arrived here, people thought that robotics is one of these problems where you can, if you put enough people on it, enough engineers, they can think really hard about it and eventually write the code that will have the robot do anything in the world. And people have tried really, really hard to do it this way. And then it turned out that the world is just way too complex. Yep, right. Like you can't just write every single case you'll encounter in the real world. So that doesn't work. And also as we were trying to work on that version of the problem, what ended up happening is people did what they usually do. They try to break down this problem into smaller sub problems. So like, rather than working on the full robotics problem, you would say there was a perception aspect of the problem, there's a control aspect of the problem, there's the planning part of the problem. And this almost grew into a different communities. There's a planning community, there's controls community, there's they have their own conferences, their own problems and all of that. So then as we realized that, you know, it's not really possible to to hand write all of these rules, people thought that we should learn them. We should learn them from data, which just seems like a really good idea, right? This is how we learn too. But what ended up happening is that they started learning each one of those components. These broke down components separately through learning separately. Yeah. So you would have a perception layer that is fully learned. Maybe you'll have a control layer that is learned. Maybe you'll have a planner that is learned and that showed some progress. It was better than what we had before. But then turned out that breaking down these problems, this problem into these sub components, it actually is the piece that doesn't work. Because when I try to pick up this glass, I don't think about it in terms of perception and then planning and then control. I just go for it. I just pick up the glass and we're just all very natural. So it turned out that this pipeline approach where you have this predefined interfaces that like perception gives you the position of the object, and then the planner gives you the trajectory and the control executes that. Those interfaces are the pieces that broke down. So everything that we thought we knew how we work was always wrong. So then we then arrived to the next stage of this where we said, well, maybe just breaking down this problem was a bad idea to begin with, so let's just train the whole thing end to end, right. So we'll take whatever the kind of the sensory inputs as input to the network, and we'll have actions as the output. That's what we refer to as the end to end approach, where you try to go straight from pixels to actions. Yep. And we'll have the network figure out or the learning algorithm figure out how to split it into these different components, if it's even possible. Yeah. And then while we were doing that, we figured that it actually requires a ton of data to do this. And often it breaks when it requires some kind of common sense. And to gather that common sense through first first person action data sets is really, really hard because you would need to experience every single thing in the world to do this. And that's where we stumbled upon vision language action models, where we can use models that were pre trained on Internet data that already have pretty good understanding of how the world works. And we can utilize that knowledge so that we don't need to experience everything first hand. You can just add some action components on top of it and have a common world understanding and connected to how to actually perform things in the world. I see. And that's more or less where we're at today. I see Now at Physical Intelligence, we figured a few other things. So how do you scale? How do you start to scale these models? How do you get them to generalize? How do you get them to perform much better? How do you have them move much faster? How do you get them to the point where you can start deploying them? But I think largely we're still in this in this era of how do you bring some of the common sense knowledge from the Internet pre training, but how do you make these models very general so that they can work on any robot and perform motions? Speaker 2 And can I ask for something like reasoning, right? There's there's so much stuff happening in the, you know, reasoning side of the large language model space. Do you get the benefits of that? And as part of your VLA backbone, do you have reasoning kind of emerge as a consequence of what you're doing as you train these end to end? Or should I think about you know some of the benefits of what's happening in the LLM world? Do they do they benefit you or not? Speaker 4 I mean, I think definitely the mods that we have today, they are already planning actions not just at a what is the immediate action, but kind of what is the what are the next 50 things I I need to do right? So like the next 50 time steps in in some sense is is a very short horizon. 50 steps means like a like a second or or two, right? And then also additionally kind of decomposes tasks into subtasks in language space already. So when you when we ask it or clean the kitchen, the first subtask it might pick our to do is like, oh, I have to drive to the counter and then I have to like blah, blah, blah, blah, pick up the the glass, move the glass into the sink. So it already has those aspects in in some sense, right. So like it decomposes task into subtask because it gives itself its own top task and it predicts like a little bit of a horizon of how actions go. So, so some of this is already there. I think I think in the future there will probably be more of it. I do totally expect that, you know, all the advances, unlike our training for reasoning, all these things will will also make their way into robotics. And I think it's kind of interesting to think about because it's, it's maybe a little different than than the RL for math problems that people do, for example, right? Because I think those are very easy for easy for us humans to think of as like textural problems, right? You think through them in your head in like text. OK, if I change this formula this way, I will get this outcome and so on. And I think for the physical intelligence part of it, it will probably be a bit more than that, right? It's, it's going to be a little bit different when you try to learn a new sport. For example, when I, I recently started to try to learn how to play tennis and you know, I don't think through in my head of like, I need to now grab the racket. I need to move it here and I need to do this swing. But it's more like you think through the motion itself, right? I you, you think about it like how does your body move? How maybe maybe your plan in some sense trajectories of objects around you in your head. And so those things I think we'll see coming to the models more over time. Speaker 1 Yeah. I, I suspect that over time, right now, we're in a place where we benefit quite a bit from vision language models. I think it's it's very, very likely that that that's gonna reverse that. A lot of the, the shortcomings that we see in LLMS today are kind of baked in or because we are focused on on the text problem, on problems like math and coding. Yeah. And I think robotics would will offer this, this new Ave. Yep, where you need to kind of rethink how, how to think about reasoning. Reasoning should probably be happening some kind of abstract space where you know, you can reason a little bit in text, you can reason a little bit in images, maybe you can reason in trajectories or in all kinds of different spaces to arrive at the answer. And robotics provides this really nice test bed where you're grounded in the physical world. There is not that much data yet, so you kind of need to deal with some of the difficulties that come with that. But I think it will provide for new findings that will then be reapplied to the LLM world. Speaker 3 Speaking about data, give us a sense of I don't know how you measure the sort of magnitude of data you've already collected and how much you would like to collect to the next year. Not like I'm sure more is better, but like what is the magnitude that we're talking about? Speaker 1 Yeah, data is. It's one of those things that is actually fairly new ones. It's not just a matter of quantity. Yeah, quality obviously matters, but also things like diversity. And even when you think about the quality or diversity of robot data, these are not very strictly defined terms, right? Like if you, if you go for the same tasks in like 10 different ways, this is diverse data or not. Or how do you compare it to the diversity of the data if you go for like 10 different glasses, right? So this is something that I don't think we as a community fully understand, like how to characterize the data, how to describe diversity, how to describe the quality of the data, how to make it very, very rigorous. And we're also finding out that those there are some aspects of the data of the data that really, really matter. Like for instance, if you want to get to a certain performance on a task, you're not going to get there by just increasing the quantity of the data you already have. We've been working on these three different tasks for the Pi Π Star O6 release. And we've noticed fairly early on that if we just keep on collecting more and more data the same way that we've been collecting so far, the performance plateaus. They're not going to just keep on getting better. So you need to find either new ways of collecting it or you need to start thinking about what kind of data will result in better performance. And this is where things like reinforcement learning and things like this can really, really help. Unveiling π*0.6: Reinforcement Learning from Robot Experience Let's talk reinforcement learning and let's talk π star 0.6. Is the star a nod to to to Q star or Yeah OK. Speaker 4 Effectively trying to get to like policy star actually optimal. Speaker 2 Policy Star OK, wonderful. Maybe just say a word on what you guys are doing with π Star 0.6 and then we can dive into what RL means for your world. Speaker 4 Yeah, for sure. So I mean, I think the main, if we want to contrast it to what we talked about earlier, the main difference is that up to that point, basically all of the robotics foundational model learning that we've done was basically demonstration data teller operated going into the model. The model is trained kind of like just imitate that data, right? And now with this new model π star or six, what we're using is basically RL from experience that the robot collects itself by actually running a policy. So we start with the initial policy is this demonstration train policy. And then you deploy it. You try to actually have the robot solve the task. And then it additionally gets kind of reward signals given by humans. And it can also get corrections where the human intervenes and says, oh, actually, you know what, this is not right. Let's let's do this a little differently. And that data, that process, basically that data is collected, gets, comes back in and the model kind of uses that data to try and figure out which of the data can I kind of kind of like, should I reinforce, should I do more of and which of it should I do less of? And and basically improve itself over time. Basically. That's kind of the big distinction. And having that stream of real data coming in is kind of the missing piece that Carol was talking about that allows us to now escape this plateau that would otherwise we're finding we're kind of like getting to. Speaker 2 Yeah. And I guess in my brain, I think viral is, you know, you're hill climbing on your reward signal. And so how do you make sure you're, how do you make sure you're generalizing as you as you hill climb on these specific tasks? Speaker 4 The way we're thinking about this for this specific kind of problem is like you have this sort of general model and it achieves some performance that isn't, isn't great. And now your first, the goal actually isn't to further generalize. You want to kind of solve this specific task first, right? Like, so we deploy it and we have, we've picked like 3-4 tasks. So it has to generalize across tasks. Nonetheless, the method has to generalize. But when you're actually kind of deploying it and trying to start this RL process, you really care about, let's make sure I nail down this task and I kind of nail it down in a way where I can, where I can, I can solve it from many different positions and I can deal with all the long tail of failures that I will encounter, right? So, and sometimes the generalization and the performance here may seem at odds when you look at it from like, wait, but now you're just doing this one task. But, but really at the end of the day, what, what, what we want to do is we have the same method, the same process that deploys to each of these tasks and then kind of gets the performance high. And then we can have all of that data across all of these tasks and we can bring that data back basically, right? So in in in that sense, it's not actually at the odds, if that makes sense. Speaker 2 Yeah, that makes sense. How much of the RL are you doing? It sounds like this is a this is in real life RL. Can you talk a little bit about the approach to how much RL you're doing in SIM versus in real life? Speaker 4 So we have taken a quite like real world first approach as opposed to using SIM. We are exploring some of course, as well as as a research tool. But all the RL we've done for the Pi Star O6 paper is actually on real systems in the real world. And the reason for that is that it's actually really, really hard to model. Again, we can, we can get back to the long tail of failures that you see when you, when you do deployment. I can give you a lot of examples from the tasks that we've actually looked at for this release where there were failure modes that we, we saw that if you had just done, done a simulation of it, you might not have seen it. So to give you an example, we have this one task, which is you have to build a box, right? So there's an actual deployment task where the goal is we built these little cardboard boxes to put chocolate into such that they can then be packaged up and, and, and, and sent out basically. So that's building a chocolate box basically. And building this box initially, you know, worked great. And then there is new shipments of boxes coming in and they come in as like a flattened sheet of cardboard. And then these cardboards that came in in this new shipment were kind of not perfectly perforated. So they were sticking together, right? And then the robot starts like grabbing them, puts them on the table to to to try to build this box. And like it has two boxes suddenly on the table, right. And this is something that wouldn't happen in SIM if you had written like a nice simulator that where you would just get individual cardboards and like fold them. And so now you have to deal with this problem, right? And if you just learn everything in SIM and then try to deploy it, you wouldn't encounter it. So we encounter it and then our kind of method can kind of figure out that, oh, actually what I need to do is I need to separate this and I can, I need to move that that second piece back and and build the box. Speaker 1 Basically, and, and we see a lot of successes for RL being applied in SIM and transferred to the real world, especially in locomotion. Yeah. And we, we haven't really seen that kind of success in manipulation for for these kind of methods. And I think maybe one reason for that is that with with locomotion, we're trying to move around. It seems that the biggest part of the problem is modeling your own body. So if you can figure out how to model you yourself as a robot, you're basically like almost there. So you can do this modeling simulation exercise once because you only can do it. You only have to do it for you yourself for this one robot, and then you're basically done. If you do it really, really well, it should transfer with manipulation. However, the problem is not how you move your own body, it's how the world reacts to it. You're actually changing the world around you. It's not difficult to figure out how to move your hand from A to B, it's difficult to figure out how this affects the objects you're interacting with. And now the problem is no longer just modeling your own robot. You have to model the entire world, right? Like every single object that you might be interacting with. Every single task you can think of and that's where we see scaling problems. And that's I think why we haven't seen those kind of methods the as effective in, in, in manipulation. π*0.6: Boosting Robot Performance and Real-World Reliability What was the headline of the results from π star 0.6? And you know where where did you see the model get after RL on the on the test that you cared about? And what do you think that means about your overall training recipe going forward? Speaker 4 Yeah. So I think for me, the most impressive thing honestly for me personally to see was just have these models run for hours at a time, recover from lots of different failures and basically just keep going. And at the same time do that at a, at a rate that is actually much better than the initial model that we started with, right. So the headline figures where we increase kind of throughput of the policies by over 2X on, on these three tasks. So there's one task was this box building task I already talked about. 1 was the making coffee with an actual kind of industrial scale espresso machine. And the other one was kind of like folding laundry. And so for each of them, we managed to like make the base policy that was trained just from demonstrations much, much faster and also make it be able to recover from theaters much, much better. And so seeing that actually in action when you, you, you sit there, right? We have, if you go to our website, you, you can look at the videos, we have the robot serve coffee for 13 hours in a row or fold laundry for four hours, things like that. Actually seeing that life changes the way you think about these models, you know, changes the way at least I think about it. Actually being realistic that we can deploy them that that we can do it in a way where it's not just a toy demo which is shown once, but it's actually kind of doing the real thing fully. Speaker 1 And that's been really a challenge in robotics that I don't think many people are aware of. Yeah, you see so many videos of robots doing cool things, and we post these videos, too. There's basically like anything you want a robot to do, There's probably already a video of a robot doing that. But you can take as many takes as you want. You can keep on recording until you get the perfect shot. And the problem that I think everybody encounters is the reliability of these models, how performant they are, how fast they can they can go about the task, how, how, for how long you can actually deploy them without failure. And I think this is the biggest bottleneck in terms of deploying these models in the real world because you know, if you if if they break every every other trial, they're not really deployable, right? And this is, this is I think the most important breakthrough for us with this by Star O6 release, that we can actually start getting to a place where they are deployable. Yeah, where we use these robots in our office to serve us coffee, or we can give them to people at π to fold laundry in their home. Or we can deploy them and have them fold boxes for real. And that is really, really exciting. Speaker 2 Should we think about what you guys are doing with reinforcement learning as primarily a customer deployment reliability points then like you can now make sure that you can, you know, go reliably deploy the the coffee making model out on a customer site. And it's, it's going to be fast enough. It's, it's not going to fail over long time horizons. So it's, it's, it's more of a customer deployment innovations versus like a fundamental kind of capability innovation or is it both? Speaker 4 I think it's both. I think, I mean, Carl, you said this a little bit earlier. I think to some extent the robots that we really, really want, right? The robot that you want as home, which can do your laundry, do your dishes, cook for you, drive around. And also the robot that people want in these smaller businesses may be solving a real problem that they have that they don't want to automate to the classical way because it's too expensive, like building a chocolate box. Those are things where the robot has to be reliable, it has to be good, and it has to have the capability to do a new task that it hasn't seen in initial training stages. I think it's unrealistic for us to assume that we can just go with like more and more human data collection, go bigger, bigger, bigger. We will do that. But there is always going to be a limit to how good and how much data you can get and how good the initial policy is going to be. So I think it is that what you said in terms of we want deployment, we need this. But also I think increasingly over the next years we will, I expect we will see that we will do this deployment and that data will actually become really valuable as a source for pre training, for making our models better themselves. And we'll rely more and more on autonomous data collection is my prediction, at least over the next coming years to kind of build that host of data, that convex hull of all the tasks that we want robots eventually to do such that the models like and ingests this and becomes good at doing them and interpolating. Speaker 1 And I think of it as a new capability. We haven't so far figured out how to learn from your own experience or there's been many attempts, but I don't think we've seen it done at scale to like a very, to the extent that actually shows a convincing result that allows you to deploy something. Yeah. And This is why this result was really, really important to us. We wanted to get to the point where they can learn from their own experience. Yeah, because similarly to how we learn, you can learn a little bit from watching videos and and maybe learning from others. But at some point you need to learn on the job. You need to try the thing yourself. You need to see how your actions impact what you actually want to achieve. Yeah. And make your own conclusions and try to learn that way. The Path to Continual Learning and Robot Self-Improvement Yeah. And I think this is the first step towards that. Speaker 2 You're reminding me of the Do you guys read the Rich Sutton Age of Experience for this year? I love. I thought it was very profound. Do you think that this unlocks kind of continual learning in robotics for you all? Will this be part of that? Speaker 1 It kind of depends what people mean by continual learning. I think it's definitely more continual than what we've done in the past where you have like a big pre training mixture and maybe like a post training mixture. And you like, you know, you, you, you sit down, you work really, really hard and then you come up with an artifact and like, that's it. Yeah, right. Like the artifact is done and there's not much you can do to change it. Now this is a much more of a living thing, right? Like we, we start with a process similar to this, but then you deploy it and then it keeps on learning, right. So it's much more continually in that sense that that it tries new things that it tries to learn from its own experience and it keeps on getting better. Yeah. Now I think there is still room to for it to be much more continual where it can acquire new skills that way or it can be even much faster in doing this. Yeah, it can probably reason throughout this process. So I think there is a spectrum of like how much you can learn on the job. And this is really promising because it shows that you can do it. But I think we can make it much, much better. Speaker 4 Yeah, I would agree. I would say we're at the very beginning of of this, right? And it's not it's not, it's definitely not continued learning in the classical sense that people would have thought about it of like data streams. And then the the whole thing turns and it just ultimately leads, leads all the way to, I don't know, AGI or something like this. Yeah, but you know, it's, it's a first step I I would say, and we're moving in the right direction there and there's lots more to be done. And I think I will say from even from this release, like I was personally impressed and to some extent, you know, shocked how good these models actually are at picking up little things that you put back into the data. I was surprised that even with just like human corrections for there was one example for for tamping when when we do so tamping is a specific part of making an espresso, right? You like put the the beans and then you have to tamp down. The best part. Yeah, the best part you have to tamp down. Speaker 3 I don't get a coffee myself so. Speaker 4 There you go. See, I'm not a, I'm not a coffee. It's a. Speaker 3 Skill issue. I'm gonna get it just right. Speaker 4 That's right. And so our robot in the beginning, like tamped way too hard because it's just happened to be the case that, you know, the initial human demonstrations were just making sure that, you know, let's make sure the coffee grounds are flat so we can put it in. And then the robot was like tamping really hard and like almost lifting itself off the table when we looked at that's, that's a bit much. And so with just, I don't know, it was, I think 34 to 50 episodes, there's a really small range of Corrections that humans did. And we fit that data back and the model actually starts like being much more gentle and doing the correct thing. And I was really surprised by that because you you think, you know, this model has been pre trained on this millions and millions of episodes and now you're just doing a little correction and that actually works. So seeing that happen was was a thing that I think is pointing towards this continual learning part, which I find impressive. Speaker 2 And can I ask though to? And then the thing I'm still hung up on this generalization. So as I learn how to tamp better, does that make me better at folding boxes or not? Speaker 4 In this specific case, no, But the mechanism is the same that you can also employ to fix the oh, I have two boxes in front of me that are sort of stuck together and I need to pull them apart, right? Because you can get 30 corrections for the stamping part. You get 30 corrections for the pulling boxes apart. Yeah, bit you get 30 corrections. For all you know, this box wasn't like neatly folded together and all of this accumulates together to then give you this more generalized improvement. Speaker 2 OK, so it's a repeatable recipe, but they don't necessarily cross cross following. Speaker 4 Yeah. I mean, we, I would expect that as we scale this up, we might see also things actually kind of transfer from A to B if there is motions that are kind of similar across tasks. But at this point, yeah, I would say it's more like a repeated recipe. Speaker 1 Yeah. And we see a lot of generalization from pre training or you train on more and more tasks, more and more data, you see that it's much easier to onboard any new task or you see tasks that appear zero shot that you didn't expect before. And this keeps on improving. We kick off a pre training run at certain cadence and every single time we start seeing that the model keeps on getting better because there's more data being put in, there's more improvements that we're making to the pre training process and so on. And I also suspect that as we have more and more of these models deployed doing all kinds of different tasks, they also bring data back in. And I think one way where we where I'm quite certain where we'll see more generalizations from that process that as you deploy these models, the data comes back, the models get better, you can deploy them more and the models get better, you can deploy them more and so on. Speaker 4 And I think maybe it's worthwhile for this point that you brought up, we haven't really talked about one like crucial detail aspect of this five star 06 recipe, which is that the model has kind of two parts. 1 is the policy that is trying to like improve right via corrections and and RL feedback. And the other part is how do you actually get this RL feedback right? So we've talked a little bit. I've mentioned like, you know, humans might correct. And that's the human correction part. And the RL feedback part is it's a little different and it's kind of, and already has some of these aspects of, of generalization that I think you're, you're, you're like trying to search for, which is that the way we do this is we, we first basically get humans to, to, to tell us basically whether a specific attempt of making the coffee or doing the box was successful or not. So there will be like human labels provided with these episodes. And then we train something, which is called a value function to try and predict basically from my given point of where I am in my, in the task, will I likely be succeeding or failing basically. And this value function is then used as kind of a baseline to to decide whether for this data point, should I like bump that up or should I bump that down depending on whether I expect that I will be moving towards success or or more likely to move towards failure. And one thing that we saw when we trained these value functions, so those are trained basically from the same kind of backbone, the same kind of model, but they're pre trained before then actual policies trained that that actually runs the task. When we trained these value functions, we see that adding more data from different tasks actually helps there and the model starts being actually really quite good, at least for certain tasks and knowing when it will fail beforehand and before. It is obvious for me, for example, when I look at a video of it trying to insert the, the the Porter filter. Thank you. See, I'm I'm I'm not good at making sure and it's trying to insert a Porter filter into the the coffee machine. It kind of knows that it doesn't quite have the right angle before that happens. So like 30-40 steps before that actually happens. The value function kind of if you look at the prediction drops and and saying, oh, this, there's not good in this specific episode, so I I shouldn't This gets better with more data and more tasks. And so this is an interesting counterpoint to the the. Speaker 2 Carpathy like slurping bits from a straw thing, right? Because you're not, you're not waiting for that final bits at the end, you're you're actually giving a lot of signal along the way. I think a rel is just like a. Speaker 1 Such a vast field and there's so many different approaches to it and people often associate A REL with something like a policy gradient method or you know, with very specific on policy learning approaches. And to me, RL is more of a problem definition. And there is many, many approaches that get around the problem that you're referring to, which is that, you know, you only get the reward at the very, very end and it's not really scalable for very long horizon tasks. There are things like value functions, there are things like temporal difference learning that try to get around this problem where you constantly make predictions and you do it in a sequential way. And this is maybe another one of these, one of these things where I think robotics can really help the the broader AI community because we don't have the advantage of having a perfect language simulator where you can run as many simulations as you would like. Instead, you need to do it in the real world. So you need to make more efficient methods. And therefore you need to learn value functions and things like this. And I think this will this will be really valuable everywhere. Yeah. From Bootstrap Data to Scalable Robot Deployment and Commercialization Can I push a little bit on? I'd love to. Speaker 2 Understand, you know, Internet video seems like it's part of the recipe, but not that huge focus right now as I see it. Like, do you think that there's gold left to be mined in Internet video? And then if you look at what's happening in video models right now, world models, to what extent do you think? Speaker 1 That's going to be. Speaker 2 A, you know, discontinuous jump in model capabilities and you know, an important part of your your model pipeline. Yeah, I think maybe there are two. Speaker 1 Questions there. One is about the data like how do you bootstrap yourself to the point where you can start deploying? And the other question is what about video models and kind of the world model aspects of it. So on the data point, I think we are now in this bootstrap phase where basically anything goes. Whatever you can figure out how to add to the model to it's benefit, I think it's good whether you can add SIM, whether you can add human videos, some kind of handheld devices, human Tel operations, I think it kind of doesn't matter. You just need to figure out some way to bootstrap yourself to the point where you can deploy these models. Because I think in the long term, there's going to be this bootstrap phase, but then there's going to be the deployment phase. And I think deployment phase will be, we'll provide much, much more data than anything you could do in the bootstrap phase. So we're in this kind of weird spot right now where we tried many different things straight to see what sticks to just get us to the deployment threshold. I see. And once you can deploy, I think that will vastly be much greater than anything you can do before that. So, so that's also what we are sprinting towards. That's why we want to start deploying these models. That's why we want to do this up, you know, with many different tasks in, in, in many different environments so that we can just have this very powerful data engine. Now on the on the world modeling side of things, I think the world models and RL approaches are kind of targeting at the same problem, the problem of counterfactuals of how do you or a credit assignment problem, right? Like how do you figure out which actions were the ones that actually matter for your success? And how would the world have evolved had you had you taken a different action? And one way you can do this is by predicting what would have happened, right? Like rolling out a full video of, you know, if I, if I put this port of filter a little bit differently, you know, where would I end up? And would this be a failure or a success? Or you can do this through reinforcement learning and does it through a slightly different mechanism, a little bit more implicitly, but it fundamentally targets a very similar problem. We are exploring all of those approaches and try to see, you know, how to, how to really solve the counterfactual problem. I don't think there is a, an answer yet, but we see, we see a lot of progress with, with reinforcement learning that we that we've just shown with, with π star with π star 06. But I think there is probably room for for many, many other approaches too. Awesome. Can we talk about once you guys? Speaker 2 Get past that bootstrap phase. Let's talk about customer deployments a little bit. What do you bring to a customer? What do you sell them? And then how do you imagine that's going to evolve over time? Like are you selling them a fully vertically integrated robotic solution? Are you selling them a model that they have to figure out how to integrate into their operations? Like how does this all work? The the the real answer. Speaker 1 Is we don't know yet. Yeah, we are. We are still figuring that out. Yeah, We are still quite early in the technology. As you can tell. We're just starting to even get to the threshold where we can start deploying these things. So we believe we should focus on the technology first to figure out how to get it to the point where it's actually easy to deploy and expand this aperture that we're talking about initially. And robotics, the history of robotics startups is, is very often gets to this point where you develop a technology for for some period of time, you started with this grand vision of what it should be able to enable, how general purpose it will be. And as soon as you pick an application that you want to apply it to, you're kind of stuck. You start cutting corners, you start figuring out very special purpose solutions just for this application. And very quickly you become, you know, an application company that just focuses on, let's say, warehouse pick and place robots and that's it. And we really want to avoid that future. We think we have a chance to really solve physical intelligence. And the benefits of doing this will far outweigh any single applications that we can focus on now. So we want to make sure that the technology is as general as possible, as easily deployable as possible. This aperture is as wide as possible. And then we'll start figuring out how to commercialize it. And as you said, there could be many different ways of doing this. There's probably ways that we can think of just yet because they'll depend on how the technology goes, whether you can be a model provider or fully vertical solution or resolve robots or whatever else. But I think it's a little too premature to answer this question. It will give you a lot of comfort just to like pick one of to give Alfred a lot of comfort. Yeah, I'll probably. Speaker 2 Be happy with us, but I think it's just too. Speaker 1 Early, No. You guys have a grand? Speaker 3 Grand vision. So thank you for working on physical intelligence. It's a, it's a wonderful, wonderful improvement just for π Star 06. It's just a huge sort of breakthrough. And so congratulations on all the success you've had. Thank you. General Robotics: A Different Path Than Self-Driving Cars Can I follow? Speaker 4 Up with a spicy. Speaker 3 Question sure. So as you said, this vision is so grand, so broad, you're doing all these different things. If I'm sure you've you've studied all previous robotics like efforts and they've largely, as you said, applied an application to an application and they get narrow and narrow and and one of the most successful cases of a large application is self driving. And Waymo or Tesla have done enormously well. But if I had to go back in history, you know, I learned about self driving when Sebastian Theron was on the stage of Ted and I think 2002 thousand and nine 2010 and he talked about the thing where they won the DARPA challenge. That was 2007 and we're in 2025 and the thing barely goes from San Francisco down here. They kind of can do it now, but they take local roads. They can't even get on the freeway. If you do such a generalized job, how long is the runway or the timeline that you're thinking about to build for generalization and performance? Yeah. So there are some aspects of the problem. Speaker 1 That make it easier than self driving and some that make it harder. Yeah. One thing that makes it easier is that we don't need to deploy it only when it's 100% reliable, right? There's many, many tasks out there that even if you're at 95% reliability, you're totally fine. If you have a robot in your home folding your laundry and every 100th bite and you know it doesn't fold it perfectly, you'll be totally fine. You just call your your child to go fold the that's. Speaker 3 Right, that's right. But somebody has still we still we still need chores. How about Yeah, exactly? Speaker 1 And with self driving, that's not the case, right? Like if you fail every 100th time catastrophically, that's that's a big problem. Yeah. So I think in terms of deploying this technology, it might be easier. Now we also benefit from the fact that this is a different era of technology. Sure. We we are at the era of vision language models of foundation models that that have some some common sense and we learn a lot of lessons between what was it 2009 and 2025 and we can benefit from all of those. So I think that also really, really helps. And these are much more general purpose solutions than what we had in the past. At the same time, there are some things that will be very challenging, right? Like there isn't just a single application. This is a very general purpose solution that can be applied to driving, but also to manipulation and locomotion and flying and all kinds of other things. And I think it's to be seen how much harder this is so far based on what we've experienced, it doesn't seem to be that much harder to be honest. It seems that if you if you tackle this with with a very general purpose kind of mindset from the get go, it turns out that it can generalize fairly well. And there is something about physical intelligence that we don't fully understand that allows these models to generalize between driving and making coffee and flying a drone and operating a surgical robot, even though they seem so far apart from each other and it seems that these should be all different models and different applications, these models somehow can make sense out of all of that data. And that gives me a lot of hope that maybe the problem is not that much harder and it might be actually easier. So I think it's a fair question, but I also don't want to draw the wrong conclusions from what we've seen from from self driving. That's beautiful. Congratulations. Speaker 3 What results is impressed you the most outside of results? That's a great question. Yeah, it's a good question. Speaker 1 Actually. Speaker 4 I can start. I've been. I've been really impressed by the. Speaker 1 Video models what you mentioned earlier, I saw them few years ago. I worked on on aspects of them few years ago and I didn't expect this trajectory to be the improvement to be so steep. Like they're basically indistinguishable right now from reality and they can do incredible things. So that's been really, really impressive and really surprising to me. Yeah, I would say I'm still. Speaker 4 In awe to some extent that we've gotten to this place where we do seem to get models that do seem generally intelligent to a level that I really didn't foresee coming out of out of just next token prediction I'm I'm still amazed with this and like every little advance that I see you know winning IMO mass challenges or you know applying it's to finding new stuff in science to me yeah there's so many things this year where I thought like wow there's still there's still a lot of progress to be made even though it felt like at the beginning of the year maybe this whole pre training business of LLMS is kind of maybe petering out a bit yeah realizing that there's like this whole almost second breath of yeah fresh air basically coming in yeah I was maybe add to this just like the fact. Speaker 1 That this whole thing works, it's kind of mind blowing. Yeah. I don't think we, like fully realize how ridiculous this is, right? Like you, you build this like loosely brain inspired thing that has very general purpose learning algorithm. You feed the data and it somehow gets it and gets it way better than anything we've ever had before. And there's a place to robots and it applies to vision and language and sound and all kinds of other things. And like, I think if you stop for a second and just think about it, how it works and, and that it works, it's just like absolutely mind blowing. Like the fact that we can have robots, you can put it in a home and it kind of knows what to do in a home that it's never been to before, or it can make coffee for 13 hours straight or, you know, things like that. And this is from this very general purpose thing that that trains fully end to end that we don't fully understand, but it seems to start to get it. That to me is just mind blowing. We're in a simulation. Speaker 2 That's what. Speaker 1 That's what Sonia. Speaker 2 Believes that. Speaker 3 We're living in a simulation. But it is interesting, right? Like in science, they teach you to take a big problem and break it up into smaller and smaller problems. And then basically somebody realizes that's maybe not the best way to train machines or robots of any kind. And to be honest, the whole machine learning like. Speaker 4 AI field made that same mistake actually, to some extent, right. We were working for a long time, people were working on solving individual problems very deeply, basically, right. And then over time there is this like notion of oh, if we can put it all together like do multitask learning, if we could do that really, really well, we'd do much better. And then, but then the fact that that all happened just because we switched to this, you know, general pre training objective and then it just all falls out. That's the part that is the surprising bit, right? Do you think it's like an accordion where we go from? Speaker 3 One framework to the other framework. We take big problems, break them up into small and small and small, small ones that work for a period of time. Then it stopped working. And then we're like, alright, let's go back to the big problem and try to solve it. More generally, go back and forth. I don't see us going back. Yeah, I. Speaker 4 Don't see us going back. I think there is. Speaker 1 A lot of approaches or a lot of people saying that, you know, you need the best of both worlds and you need some kind of way of incorporating the rules that you already know about, like, you know, Italian physics. You don't need to learn that. We already know how it works. Or can you just like put it somehow into the weights? But from what we've seen so far, it, it doesn't work. If you try to do this, you kind of limit the ability to to learning new things. And I don't think there was the best of both worlds. I think we just go all the way learning. And it's kind of interesting to, you know, how similarly to how we learn, you would think that if there was a way to pre bake all of the intelligence, the evolution would have figured this out. You would have just been born, you know, knowing everything there is to know. And we see this with some other species, right? Like I think deer, when they when they get born, they're basically like as smart as they'll ever be. Like they don't really learn much throughout their lifetime. But for intelligent species like like humans, but also I think crows for instance, they have these childhood periods, the adolescence period where they're not very smart to begin with, but they have to learn from their own experience and it doesn't come pre baked. You kind of have to earn it on your own. And I think there is something, something to that. You need to just experience the world and learn from that. And I think that's the lesson we're learning in machine learning as well in AI, that we think we know how we think, but we actually don't and we just need to let the algorithm learn it from data. Same thing with raising a child I. Speaker 3 Think I know how my son is thinking, but I don't Yeah, I. Speaker 1 Have a small daughter and yeah, it's just so surprising. Like they learned so fast. They learned so fast and you don't know where. Speaker 3 They get it from. Speaker 1 Hopefully from the parents, hopefully she definitely knows some things that they didn't teach her. Thank you guys so much, it's a really beautiful mission. Speaker 3 You're building after. Thank. Speaker 2 You for coming this year. Thank you. Thanks for having us. Thank you.