Book of Genesis: Chapter 1, verse 1

In the landscape of AI coding, Pi is arguably the most interesting agent harness to look at today. It's gained massive popularity when the AI assistant OpenClaw took the Internet by storm. OpenClaw was built on Pi and uses it or used it as like it's backbone or spine. There's been a lot of developments in Pi since then. A company named Aaron Dell has since acquired Pi and its genius creator Mario. Aaron Dell interestingly was started by Armin who a lot of people will know from the Python community. In fact, I used to follow Armin and his blog during the century days, but also clearly he's created flask, ginger, pigments, this is just to name a few of the Python creations. Pi has hit 1.0 recently, and that's like a monumental release for many reasons. But one thing that they also released on the side is Pi Durable. Pi Durable is this library that a lot of people have started to get really interested on. And if you've listened to the last episode of fragmented, we talked about self-learning harnesses and you start to see some kind of resemblance of those patterns. But I thought, hey, Pi Durable is clearly something that's very interesting. We need to learn more on fragmented about Pi Durable. And I'm extremely honored and happy that we have the best sources on the planet to talk to us about Pi Durable. So Mario and Armin, thank you so, so much for coming on the show. Well, thanks for having us. Yeah. I'm going to try my best not to, you know, fangirl or fangboy because me and my co-host we spent literally almost our entire day on Pi. So it's hard, but, you know, in the interest of science and learning, I'll try to extract as much wisdom for the listeners and not try to fangboy. One of the things that's impressed me and I'm sure everyone else about Pi is how right you get those core primitives. Is there some kind of, like, rulebook or some philosophy that you follow that helps you guys get it so right all the time? Like, you know, what's your approach being to create that API or interface just at the right level, but not more where it gives everyone that extensibility? Um, it's a question for you Mario. Yeah, I would say. Share your magic. We didn't get everything right, obviously. And a lot of it is just our endipity and, I guess, the other half is I've been programming for over 30 years. Armin, it's probably also clocked a single amount of time programming things. We like simple things. We like things that we can fit into our head and understand ourselves. But it leads to systems that are generally simple. Doesn't mean they are easy, per se, in case of Pi, they are comparatively easy if you know what I mean. Like, ultimately it's a while open and the question is how do you package up at the while open, give it enough hook points so somebody else can build stuff on top of it. The first iteration of Pi, which now has culminated in 1.0, is basically just that. And it was born out of frustration and messochism because I used to do a lot of open source work. And then I did this and then I was back in the open source world and, yeah, just to put that in context right, famously, Pi, I did not have MCPs, Pi 1.0 now has MCP. You guys wrote a long blog just about it. I'm actually very specifically curious because obviously that leads into like code mode, which is also very interesting. Even after reading that blog post, what I came off was like, seems like the code primitive that makes a lot of sense. You could have just left it at that, right? And then there's like plenty of plugins with MCPs. You could have just said, well, okay, now if you want an MCP, you can just use it. What made the like, what is that itch where you're like, no, but we probably have to pull an MCP as well. Can you use that? Like, you know, you guys make it sound simple out of humility, but it is hard to get that API really, right? No, I looked at MCP and I'm like, I don't know why they added it. Like, what was the dot plus is there? So did we get it right with MCPR, that's your turn. MCP is interesting because it has almost become a little bit of like an unintentional dis. Like the fact that I didn't have MCP was an unintentional feature, but not having something is not necessarily a feature. Despite the fact that there was not MCP and Pi didn't mean that we didn't also use it at times, or that we didn't talk to the MCP folks. So it's like, we paid attention to what was going on. One of the fundamental parts about Pi is that it doesn't try to do something that the models wouldn't sort of naturally go on out. So we're trained like wherever the RL is taking these things, we want to make sure that this works. And so CodeMode increasingly became a thing that was useful for models to express to calls and like extended workflows. So it was very clear over time for us that eventually CodeMode should be in there. But CodeMode comes with a lot of complexity, like, for instance, there's a JavaScript runtime in it. And then the MCP element of it was sort of like once we already have CodeMode, MCP sort of very easy to do, the ultimate motivating factor for CodeMode was actually even MCP. It was chef and decision or plasifier models. Because like in order for those models to make any sense at all in the context of Pi, which is based on LLMs, you need to create an ad hoc harness almost. So you need to have the LLMs create a contraption by which it can call a classify model. Step one was like, how do we get this to work without introducing more tools into Pi? And MCP, like if there are a lot of use cases where MCP works pretty well, and the one that I use a lot right now is Blender MCP, for instance. And that is works at least for my testing. But MCP is getting better and with CodeMode and I would say like progressive disclosure, it also doesn't waste the context like it used to do in the early days. And I would still say like, right now I'm not a huge fan of MCP. There are a lot of issues with it. But we are also hoping that by having sort of an M CodeMode only MCP, we're kind of also being one harness that pushes MCP in a direction that we actually like. So instead of sitting on the sidelines and not doing anything with one idea, at least say like, hey, this is how we think this could actually work out. And I guess another side to specifically MCP is that it has become a standard to authenticate with services and let your agent integrate different services. So while CLI might be a little bit more efficient, possibly, with CodeMode, you can get a similar kind of efficiency depending on which tools you expose through CodeMode and just allowing people to connect to establish MCP service with OAuth and all of that. Like since now, at the end of 2026, at the end of 2025, I wasn't so convinced yet. You mentioned there's still some things that you think can't be improved about MCP, can you talk about that? Well, what are some things that still don't work as well? Well, I can tell you, I don't care for MCP, but we recently had a talk with a bunch of MCP maintainers and they were really nice and smart people and very receptive of feedback from harnesses. One, the one that I have is that, so an MCP server can be, it's basically just RPC, right? And then the question is, you have your methods defined that you can, uh, let the LAM call through the MCP server and then what gets returned? If it's just plain text and it goes directly to the LAM, to the agent, okay, but that does mean that programmatic manipulation, transformation of those return values becomes hard. So what we suggested was that the MCP standard kind of enforces return types, strictly type things. So programmatic use of MCP servers becomes easier, which then together with code mode makes a lot of sense. No, that makes sense. In that philosophy of keeping it minimal, can you give examples of things you've explicitly chosen for pie to not have now? Currently there's no sub agents. Uh, yes, but it's, it's, it's very easy to build a sub agent extension like I have one that just basically adds a command line tool that spawns a new pie session in a TMAX container. I can switch to it from within my main pie session within pie, which works fantastically well. And I think even though there is more stuff in pie now with code mode MCP, if you upgraded one version of pie to another, it didn't magically change. So we're still very, very like protective of like you should be able to safely upgrade the whole thing and like everything continues to behave like the models are going to change because like you're changing the models, but like pie itself doesn't form one person to another randomly produce new things into your context. Yeah. We've actually been pretty good about not breaking people's extensions either. I think there were a couple of handful of small things that broke, but overall it's been a year and people are still not mad at us. So we, we probably did an okay job keeping it stable. I can attest to it. I don't, I don't think my plugins are necessarily that complex, but at this point I've got like a whole bunch of them and I've very rarely run into an issue where it's always been more like, oh, they introduced this. I can probably like improve this aspect of the billion. It's not been like, oh my god, they broke it for me or something. On, on the other hand, that same plugin system is what's kind of holding back pie point one point, oh, to do a bunch of things we'd like to explore just experimentally. And so in the name of stability, we have now not frozen pie point, one point, oh, we're obviously adding new stuff like code mode and MCP, but the extension API. and DSDK APIs, so many people are using them now. I'll sort of fight a coding agent that we kind of need to keep them stable. So then we built a new thing. - Yeah, that's the perfect segue. Thanks for helping me get as beautifully get there. Actually, you know, a pie durable is clearly everyone's super excited about it. And like, you know, there's multiple posts and people talking about it. I actually wanted to go and start with the motivation behind it. Pi only exists, it's great, but made you guys wake up one day and decide, you know what, I want to experiment with this new thing called pie durable. Like what was lacking in pie that you felt, no, I really feel this itch to go and start something that's different. - Pi is a coding agent, which means I talk to one of multiple agents in multiple terminals and the sessions are usually pretty short. And once I'm done with one session, like fixing a bug, implementing a feature, investigating something, that session, I don't really care about it anymore, right? But that one single agent is very ephemeral and if it goes away, I don't care. What we do see is, and I guess open clock and showed us that is that for other vertical photo use cases, you want very long running agents. And you need to have the agents that tolerate errors and resumptions just because the user might be a non-technically user and not be able to fix an agent dies. There's also agents in the cloud where you might have compute substrate that's not permanently there, but just gives you slices of compute for a limited time and then you get shut down and you have to pick up where you left off later and all of these things basically led to us trying to design a durably executing harness, which means you run the harness, the harness does a bunch of stuff code mode, tool calls, compaction, whatever your application needs to do, is also covered by durably, it can use its durability engine, so to speak. And if your process dies and you spawn it up again and reload from storage, it just continues from where it left off automatically. And the term durable is not like a cool name that you came up with. It is a known terminology in like the space of software engineering, right? Like, can you talk a little about that? I think our minister computer scientist. I mean, I don't know necessarily where the whole thing started, but the concept of durable workflows in particular is pretty well established. And it really expresses the idea that you can. I mean, the equivalent, I think that sort of gave birth to some of this was a project called cadence, which later turned into temporal, or like temporal is the second version of cadence. Actually, even before cadence, there was like a dot net workflow engine or whatever this thing is called. But it was the idea is like, how do you express long running computation where you can serialize all the state in between so that you can continue off where you left off? And people have been largely using durable workflows for things like when an organization signs up, schedule a task 15 days from now to remind them to, I don't know, subscribe or something, right? So like this, this classic workflows that you want to have a lot of state on that you want to have expressed from a computing point of view. Like, you have a bunch of these jobs that then intern has like a lot of state that it has to manage and report back. Interestingly enough, I think like a lot of the design ideas behind what was really great about the server workflow engines, I was a little up to two years ago, was to make it easier for the programmer to use it. And so they used a lot of like automatic snapshotting, things like that. But the idea is really like, if I were to run this code, you basically use a spend it, but then you run it again on every checkpoint that you run through, you skip because you already have the data point. And that idea is embedded in the root of piterable, but it's sort of the other way around rather than automatically snapshotting, we're letting the agent explicitly snapshot. - Interesting. And can you talk about that snapshotting at what dimension does that snapshotting work? 'Cause you're not snapshotting it like at a turn agent loop turn level. You're not snapshotting it at the session level. You went a level deeper, right? Why, what helped enable that? - Well, so a basic agent loop has a bunch of tasks if you want, that it has to execute in a specific order. And so let's just say you start with a generation task where you get the input from the user and the previous context. And it sends out an LEM request that comes back with a tool call or with multiple tool calls in a single response from the LLM, so you want to spawn tasks for each tool call to be executed, right? In parallel, for example. And then you have a task that awaits those tool call tasks and the results. And if they come back, it formalizes the next step, which might be do another generation task or just stop whatever. And those are not a lot of tasks in an agent loop, but we wanted something that's more general so that we can build apps that also schedule tasks that are not just agent loop tasks, but other things. Like, for example, if I want to index my session into an inverted index, Lucine or Elasticsearch or whatever, I want to schedule a task in my harness that periodically runs or runs based on some trigger that can then index the latest messages within the session so I'm able to search my sessions. Or I might have background agents. Or code mode is basically just every tool that's getting called in code mode is a new task. And with this primitive of tasks, it then becomes very easy to basically do this effect sandwich, which means I first store the intent, what am I about to do, right? Like do an LLM request. Then I do the actual thing, the LLM request, and then I store the result. And anywhere in there, the process might crash. And so if I come back and see the task exists and it stored the intent to send the LLM request, I don't know if it was actually sent or not because I haven't saved the result, but I know that this is what I need to do, right? So I can try it again. If I have the result, then I know this task is done, I can actually basically terminate it and do the next thing. So what PyDurable then does on an API level, what Armin alluded to in Temporal or something, you kind of write straight line JavaScript code. In PyDurable, you write a task, which consists of phases. So say I have the prepare the generation requests, do the generation request, and store the generation requests result. Those are three separate phases. And each is basically one of these sandwiches where you store the intent, do the thing, store the result, go to the next phase. That makes reasoning about it very easy. And it's also very easy for agents to understand. - Car, and that's what you're calling as a task, which has those three separate phases. - Yeah, it can be any number of steps. It can be just a single step or 10 steps. It's composable, but it's explicit in the state transitions, which has one benefit that is. So you can basically think about it like in Temporal, you have to re-establish the point in your call stack where you crashed. In PyDurable, you don't, because you don't have a call stack. You have a linear transition from state to state. So if I load back a task, I know exactly where to continue. I don't need to do function calling and replaying or what happened. I know I'm now here. And this is my program counter, so to speak. - It is a little bit of a mix between a durable workflow engine and sort of structured concurrency. The ideas of expressing basically work similar to how structured concurrency works. - The natural question to ask is, well, why can't we just plug Py in some way to Temporal? But I mean, I've worked a little bit temporal, so I know it's not as straightforward, like it is quite heavy or something. - Now you can. But Py is not going to use Temporal's durability primitives. Py is basically just one activity within Temporal and does its own durability. - But Armin, you had like absurd tool, right? Which I mean, as in like the tool called absurd, which in some ways feels like it would have fit magically because it's like a tiny SQL, as I understand, I've not used absurd specifically, but it's a tiny SQL library back by Postgres. It has all the right check boxes that people would want these. - It was actually interesting because like, absurd was written a year ago. And it was written at a time where the LLMS were weaker, let's say. And so absurd in a way made the same misstep, that, or maybe not misstep, I think it's still a good design, but it made the same trade off that Temporal does, where every time you suspend, you basically mark all the checkpoints that you have suspended so far. And you need to have some level and actually a 100% level of determinism when you run it again, so that you hit everything at checkpoint, because if you miss a checkpoint, and from that point onwards, you're executing everything again. So the benefit of this is that the code looks very neat. The downside is that now every time you're making a change to the logic around it, so like you're making a new version of the code, you need to either put some code in to hit the old checkpoint and then transition to a new checkpoint, or you just lose everything along the way. And working with L&M's, they actually don't really necessarily like that. They prefer the explicit way. And this kind of interesting, because a lot of the original design, prior to cadence and Temporal, was my memory of this was that it came from the more traditional workflow engines from.net, where you actually had explicit configurations. So you expressed everything of thing as a task, you wired them together with some configuration XML stuff, I think at the time. And people just didn't like writing that code, but it was actually very explicit. But people didn't like writing the code. But now with L&M's, I think the tradeoff is sort of back to that, where it's actually better to have it expressed explicitly, because then every time it does, it changes it fully understands that. The second part is like, we obviously can't make Pi dependent on Postgres. And so, but some ideas of absurd went into PiDurable, but not the automatic snapshot input. - You know, you said like one of the 10-pole features is getting pi-durable. to survive or respond well to crashes, right? How does that design work? 'Cause if you think about software engineering tasks, not all tasks necessarily, you can actually recover from, right? Like what are things that you can or cannot recover from? Who makes that decision? Like I don't think necessarily that's made at the model level or maybe it is. How does that work? Then there's the environment aspects of it. That also can cause like sort of instability in how you actually respond. Just wanna dive into all of that. So your LLM has tools and those are basically the way for the LLM to do something in the real world on some other system, right? And you do not have control over the other system, but you only have control over your tools that call another system. So a read tool can be replayed safely. You might get a new version of that file compared to three hours ago, but the act of reading itself is item potent. Did I do it? Yeah, yeah, yeah. You got it right in a previous podcast. I did see stumble, but this one, you got it. Holy cow. So yeah, read is no problem. I can mark the tool as what I call replay safe. So if we crash while read, shoot or has executed, we just execute it again. Of a write, it can be a little trickier, but generally it's fine because write means write an entirely new file, right? And if I previously stopped doing that in the middle, the new write will just override the old partial file that resulted from the crash. Edit, the same thing can be replay safe. The edit tool would just fail if something changed in the environment to this if the file changed. Not a big bad monster in the room is bash, obviously. If the LLM uses a bash tool and does a bunch of very complicated invocations and incantations, you can obviously not say, "Ah, yeah, I'm just gonna retry that." Like, I'm not gonna redeploy my server or I'm not gonna do some destructive action again. So that tool, for example, is not replay safe. So if you crash while it executes, when you resume the LLM sees, this tool was interrupted because the process crashed. So please inspect your environment and figure out what to do next. That's basically interesting. - Oh, interesting. - The answer actually in a way is that the LLM is surprisingly capable of recovering from failure. So they can make the judgment call to some degree and I think one sort of interesting element of a durable harness is that, and it's a very minor thing, but like in existing pie if you stop it and start it again, you have to right continue. Even though we technically would have known that we actually never made it to the end of the stream, and what you kinda don't want is just an agent, it's sort of like it's stuck somewhere and requires a human to re-activate it. - It gets in the pot. - So now at least we don't lose that information. And so the harness has the ability to recognize like where did we, like what exactly happened here? How did we end? What should we do? - I didn't clap that before, but oh, you're saying because of how we have like architected this to know what the safe and unsafe calls are. And if it's technically unsafe like bash, you actually can instruct it to go and actually check what's happening. - Yeah, and this is not so relevant if you're working with a coding agent locally on a machine because you're there and you can just type continue and it's a small annoyance, right? But if you deploy an agent on say cloud flare, durable objects were the agent that the durable object only has 15 minutes before it gets killed hard, right? Then it does matter because it wants to go into your background async cloud agents and kind of kick them in the butt every time. - Continue, continue, continue. - And so this is one of the problems that durable solves. And while this is not too hard to do with all the things that go on in a coding harness, so what can break? Generation can break, compaction can break. Two calls can break and you can be smart about how you resume that when it gets interrupted. But harnesses are becoming more complex. So for example, if I have one conversation, one agent and that spawns a sub agent through a tool call, then you're also when you resume that one to the sub agent, so also resume with their thing, right? Just an ownership, basically every task may have a different task. So if the process dies and I see this task needs to continue, then that means all its children also need to continue. So if I have my main agent as a task that's currently calling a tool and the tool is the sub agent tool and that waits for another agent, another conversation, which has its own tasks for tools and generation, then the entire tree or graph gets marked as I need to execute all of these tasks again on resume. And that's it and it's very simple to reason about. It's just an ownership tree, basically. - And also it resolves some really nice aspects that were just incredibly hacky in the earlier version. So as an example, background compaction. - So there's one of the, I think, I mean, it's probably not too surprising, but like one of the problems that you have with an agent is that you're busy state synchronizing also like a remote service, which is the LLM. So you wanna maintain the KV cache as good as possible and like that in itself is a function of what Harness does. But you're not actually required to not use concurrency. So it would be in code is kind of trivial to say like, "Hey, as we are filling up the context, please compact." But also we wanna continue making progress and that will take the KV cache hit once to sort of slice in my stuff. Now, it doesn't work on a topic for reasons, but there's a lot of stuff you can do with this which becomes sort of more natural to how the system works compared to doing that in the model that is in PyWon is comparatively harder because there's not a clear, there is a state machine in there, but it's not as nice and it really kind of doesn't assume that there is stuff happening concurrently. So it has a very sort of, there's just one thing happening and yeah, and you could sort of patch it into it, but there are limitations to how far you can go with that completely destroying the abstraction. - So this entire task abstraction that we have in the scheduler, which is basically a DD workflow engine, that is open to anybody using the Harness. It means you can also do your application task scheduling with the Harness. And the reason you want to do this is because it turns out that yes, you can take a Harness, any old Harness and put it in temporal, but you do not get the same durable execution guarantees. And it also extends to state that you want to do a BLE record, which is obviously also bound to your execution state, it's not just the data state or the application state, it's also kind of tied to where in my execution am I at the moment. And you kind of want to tightly couple those two things together. - This is such a crazy story and you think I'm making the story up for the podcast, but I'm not. I actually was working on a sort of my own local system. It's called workshop, which is basically one of these software factory kind of use cases, when I'm like, if you really want to execute these autonomous loops that a lot of companies are talking about, there's a lot of complexity involved in managing that state. So I got to think about these, I'd had this thing built and I was traveling, as it so happens, I was in Vienna. This is crazy, I didn't even tell you this. And I was on my flight back and you guys released PyDurable, this makes so many things easier than like damn these guys, I spent like a week, I spent weeks trying to tune, because there is a lot of complexity in that state management, I felt that, 'cause a large part of it was me just hitting each problem. I was like, oh, God, wait, how do I manage this? I don't know to manage this, okay, fine, let's like spend some time. And I'm like, I think I have something and then PyDurable comes out. So I mean, well, the happy story in the end is, I actually have almost rebuilt that entire thing, the PyDurable for the same reason, because it just removes all of that state management. - Yeah, but it seems like the two of us work the same way, because PyDurable is like the eighth iteration of this design, it took me all something. - Yeah, it took us really long to feel good about this. - And it's not like it's amazing in the sense of something else I mean, it's just, you have to think through so much stuff that you're hitting roadblocks with very designed. - And I think the, what is so confusing about using algorithms for coding right now is that you feel like you can solve any problem and you can sort of do, but you don't end up with this like really neat sort of core design that has like the least amount of possible states if you, like it looks like it does, and this is like actually it doesn't. - This is one of the most tempting things. That's why I started with like, do you guys have some kind of philosophy? 'Cause in this day and age, it's so tempting, right? Like it's so tempting to just throw an agent at it to solve the problem versus, you know, stepping back and thinking, "No, no, but is this the right primitive?" And that remains fascinating how you've done it with PyDurable. - Yeah, I mean, again, it comes back to this. Can I keep it in my head? And I can keep PyDurable in my head for the most part. Obviously some details like, how does the Bash tool concretely handle, I don't know, Windows versus Unix like systems in terms of process groups and child process, I don't, I don't, I don't remember the details, right? But the oval architecture and the overall concepts in Durable and the main API of work against or my agent works against, I can keep in my head. And this is only wipe based, but I have found every time I build a fundamental primitive like that, agents have a way easier time to stick to that and build on top of it. Then if I just sell them, build this make no mistakes. And then I iterate and add feature, and feature, and feature, and feature, and the code best just becomes your mongers without any kind of structure. So yeah, that's why we spend so much time making it kind of sort of tiny so that it fits into your head. - You briefly touched on like, you know, process ownership, and that's one of those questions I had as well. 'Cause funny enough, you know, Mario, I know you released your sub-agent extension. I've had my own sub-agent Pi plugin that's being there for some time. If you think about it from the surface, well, it's a simple requirement, which is, I want my primary agent to spin off some subagents in the background but if it gets interesting I kind of want to jump on to it right I want to like hop on to that sub agent and then actually like you know steer it More or like interact with it and then you know Collapse it back. Yeah, right? This is like again like vibe hitting the problem You know as you go then I realized that this is a different process like you can open it in a separate pipe process But then how do you know when to do it right if your primary session is Opening it up as a background session or like no, do you want to gain that handle to be able to access it Then it's a very different thing because then the communication that has to happen is very different from this simple like Oh, maybe I want to jump on to it after the fact, right? I'm sensing I haven't done this yet, but I think PIDURAL solves that problem to some extent. That is done then This is it's super easy like arm and just spun of what you had like 32 sub agents in the single PIDURAL session all working in the same process and Yeah, is that so on and durable? And I think which is also sort of it's not like all the time we are going crazy on sub agents. That's not really What's happening here, but there are obviously a lot of cases where you so I think like we're sort of separating too far south One of which is like here is like In agent it like all week sort of spawns like sub agents like crazy and they talk to each other and they have swarms and they use beads and whatever Yeah, that's not really a thing that has like a lot of So far at least that has not a lot of traction with an aerodil. But for instance like to review a ton of Pi issues. I can't just say like hey get me all my code mode issues and spawn a sub agent for each of them And that makes sense because like all they're doing is they're researching the issue reports that came in And then you can sort of switch them to some stuff and the difference between how this works Indurable and how it works in Pi 1 is like adorable The sub agents are basically like a natural kind of emergent behavior of like how the data store in itself works And you just schedule like anything that says like hey, we need to make some progress here Will automatically be scheduled onto an agent harness. So the it is a durable work function that also has a schedule Right and sort of because there is a scheduling system in it it will just naturally get something attached to Pick up the work and continue working on it That doesn't mean that if you use pi durable out of the boxes like a sub agent communication system That's not really at the core of it, but it is just one tool away for you to hook all of the stuff up And that also leads to some really interesting ways in which you can also hook these things up because for instance unlike how I mean like on the coding agent layer had to fear ready clear a lot of flexibility in doing this But on Pi durable or if it's generally how we use pi durable Where you execute these things is also a little bit more clearly delineated from where the harness sits And so we don't expect them to be on the same machine Which would then also allow you to be a little bit more flexible in how you set these things up Although once you cross machines to some degree You will probably need more than one schedule out But like if you can schedule it from one machine and just reach with your arms it to some other box And that still is an extra behavior of the system Now this whole bot assistant workflow is like you know gaining a lot of traction right and yeah Probably noticed everyone's building a version of this with pi durable which is awesome Can you talk about that multi-player system with Pi at least in the past I couldn't have a pi session and then you know join it from my phone Or like join it from like a different sort of client all together So that's an other aspect was something thought through in pi durable to make that easier or like was it always possible to pi or like can you talk about the And so it's kind of sort of possible with pi, but it's not the nicest pi was never designed for this a lot of people have used it Obviously to do just that like open claw is basically the one most famous example In pi durable durable itself is not Multi-play in a sense that it comes with a server and clients and whatever But it gives you the primitives to allow you to build a server and a hundred lines of code and a client in a hundred lines of code Where any number of clients can connect to a doable session and get updates from that and send stuff to it So we have something called watches where you basically get sort of like an event stream that tells you How the life state changed and what the context looks like and the state of the agent is And you can connect as many clients to that watch and they all get just a minimal delta that they need to know from the last state to render the current state And then in terms of multiplayer that is multiple people are other agents talking to a single client as single agent It's just an inbox system really So classical clients send their inputs they get queued up in the inbox and then at specific safe points We pull from that inbox and give it to the to the agent. So it's not rocket science Again, the primitives make it very easy to quickly build a server client thing on top of durable Um, what's harder is and that's also something we're working on at the moment in like an experimental playground is A plugin system that is also networked So I I want to write a plugin for pi And one part of it runs in where the agent our restaurants were pi durable runs Another part runs in the web interface and another part by running a toey And so getting off that to work is an entirely separate can of worms on top of pi durable That was going to be one of my you know, follow ups if I try to like think about what pi durable gives me One of the things is like yeah, it survives the long running processes The durability it's able to survive you know through crashes and stuff But you know supposing I spin down my cloud instance I come back three weeks later And I have like three plugins that are new or like some plugins that have gone away. How do you win? Man is that right like how do you recover from that? How much time do you have yeah we we also spend a long time over the summer Actually working on a plugin system that can survive that kind of stuff across multiple clients multiple machines we call it cord And the basic principle is at any point in time there's a set of plugins that is active and known good That means they are compatible with each other and they work And so when then a new plugin comes in or an existing plugin changes or gets removed We keep the current active set of these plugins working We load the new set in a separate kind of thing where it can validate that it's actually all fitting together still And we do this by explicitly versioning things that is we version interfaces we version the data structures that the state schemas and so on and so forth And only if we have validated that new set of plugins we tell everyone that is interested in plugins That is the server that the agent worker the web clients connected the twig clients connected here's a new set of plugins You can load But only do this if the active set of plugins is no longer used so any inflight calls to one plugin They wait until that's done they train it and then the new version comes in And it's actually more granular than just the full sets that get switched You can so every plugin basically contributes networked objects I don't want to bore you with the details But you can actually just modify a single object within the plugin and only that object gets switched out And everything else within the plugins stays the same it's Again, let's do another podcast Gladly have each and I will also say like the plugin part is like I think that despite the fact that I think like Most of the design sort of predates fighterable. It's also not done It's mostly done, but not in a way was like battle tested Because it is it is a surprising hard problem And it's also if you sort of think about it a little bit longer Pretty significant issue here is also like with multiplayer and this is really what you want It's like someone owns session other people join into it Once you have user-generated content whatever you want to call this you have different trust levels And what you don't want is you don't want to try into someone random session All this sort of takes over your authentication credentials So with a multiplayer experience Unless everybody's actually on the same trust level is for the most part it's not really a thing There's just inherent much more complexity in this in this entire experience I was thinking of joining Mario's like Android session that he exposed to And I was like what is he actually doing this? I'm like let's try unfortunately I know what I'm doing famous last words No actually it's much more like I'm when I expose something like that I know nothing bad It can be accelerated. I there was nothing in there that could have caused damage So armen doesn't have to worry about air and those secrets like Connected to my stuff and got read right access and the first thing he tried was to to to hack my shit So yeah, it's not my phone number on the internet. Yeah So he fucked himself. It's great No, so the interesting aspect here So generally when we consume services we nowadays do it through an app or through the web right and mostly through the web Let's say on a desktop in in your office whatever right And for a website to do damage to a local computer The code that the server sends your web browser Needs to basically get around the the browser sandbox which is very hard It exists in a lot of groups are doing it. Obviously zero days and so on forever. It's very hard right right so now You have some kind of of agent right and it has a server client architecture and the plugins are controlled by the owner of the server And when I connect with with my clients to that server I get the plugins for the client from the server So if you connect to my server and I have nefarious plugins that have access to your file system because they are now running on your machine I can easily hack you right right and that is actually a huge huge problem And it's not so much a problem in the web interface client for example But it's a problem in the twig client because the twig client in our case is a note process And if you load if you load and then I send a send JavaScript from the server to your local machine's node process that loads that, and then might have access to your file system, to your network, to whatever end. So that's interesting problems to have, because now we have to think about code isolation and sandboxing and so on and so forth. So Multiplayer adds a lot of new capabilities, but also a lot of dangerous stuff we need to think about, real hard. One topic I had is one of the things that on the podcast, we talked about self-learning harnesses. You guys also talked about some interesting paradigms with deep seek and then what's the other one? Prime agent. How do you think about these self-learning harnesses? Pi has reload, which is like, I can just do things hit reload, kind of like a hacked up way of making the harness learn itself. Despite you're about to have something, is that already there, I'd love to hear more. Especially in this long running world, right? You have agents that run for a long time. If they need to improve themselves, what does the future look like there? It is a good question. Never asked me questions about the future, and I really suck at predicting things. Just going back to technical levels, what does self-improving harnesses mean? For that, you need to have a goal, right? You need to have something that the agent can optimize towards, that it can evaluate if it has reached that goal, right? Right. I'm not quite sure what prime agent, for example, does in that respect. They obviously do some self-improvement. I think they might just, I don't know, bench max on terminal bench or whatever, and then sure that the harness keeps getting better and better at that benchmark. I'm just making this up. I haven't looked at it. So nothing in the old pie actually prevents you from doing that. In fact, prime agent was based until last week on pie. So they just rewrote it. They never wrote it to the rest. So you as the coding agent user in the tool you, yes, you have to type reload, and that's kind of nasty. But nothing prevents the agent to do that for you basically, right? Impidurable, the only thing that we ensure is that we already have extensions in pie durable. It's basically a set of system prompt contributions, tools and tools that you can bundle up as one extension, right? And then the same hot reload stuff I talked about earlier with versioning and blah, blah, blah, blah, that applies to that as well. So now the harness itself has hot reloadable parts. And with that, you could easily build a self-improving harness. It just keeps improving while it works on itself. So I think the thing is the self-improving part that is both intellectually interesting, but also highly confusing, is that where do you set the limit? And this then no limit, because a lot of people have started up like, OK, here's my initial prompt. It's like the last prompt ever. And then it's like set yourself up a harness, and then just go from there, right? And then you do end up with an organism of sorts that behaves in whatever form, probably is optimal for the thing that it's doing. But because it is sort of optimized for that, there is no constraint to the system at all. And that might actually perform really, really good for use case that you have. And if you never have to look at the thing, and if you never have to interact with the thing, or plug into the thing, that might be optimal. But if you do want to debug this, or if you want to sort of connect the UI to it, you actually do want it to have a certain shape, at least. And that to me is sort of the interest in Canada right now, because it's obviously very straightforward to have a harness, but I have a model build its own harness and just drive that. And if you never have to look at it, and you only observe the outputs, that's totally fine. But observability was always about being able to introspect it. And now you kind of want to have a UI as a hook to up, or something like this. I think what is more interesting is actually a harness that is capable of self-extending, but not necessarily that is sort of completely revised itself. I mean, I want to see these experiments how they play out, because I think it's really, really interesting. But it's also deeply discomforting, I think, in a way, the idea that your software eventually ends up being something else entirely. Like, mold, it just grows. I mean, obviously, obviously, solution to this eventually will remove all the pieces that talk to the human, and it will just run on its own. You're like, why not? It's like, unless the reinforcement learning really says, like, do no harm, or whatever is sort of really strongly enforced, there is no reason why that agent should in any way optimize anything other than the sort of very narrow output that you gave it. The goal is to do this, like, what, I shaved 20% CPU time off by getting rid of the part that human talks to me. It's like-- Yeah, actually, I just want to bring that up. Self-improvement, given a goal, is definitely something that those things can do. That the only problem is the specification of that goal, and the Oracle evaluation code that actually evaluates whether that goal was reached, or if you made progress towards that goal. I did this for Pi startup for shits and giggles, many, many months ago, like using auto research and just said, make Pi startup faster, right? And as a human, you would intuitively think, yeah, of course, make it faster, but without removing things. And what it did was it basically removed everything, right? It just left a shell. There you go. Yeah, ta-da. One millisecond. And so I mean, that was a trivial example. But then if you have something that's more complex, that's more like, OK, modify this harness within which you yourself operate to perform better on that and that and that and that benchmark. That can mean so many things. That can mean looking at the benchmarks, at the problems you have to solve in the benchmark, and just adding system prompt hints for each benchmark. Perfect. I have improved the harness. It now works perfectly fine on terminal bench. Actually, some of the permanent bench submissions did exactly that. So at the end, you need to be very specific about defining the objective function of the goal and the process or program that evaluates the progress towards that goal. And I feel like that's where most of the shit falls on its face flat. It's also like-- I don't want to shit on the idea just to be clear. I think it is very interesting. But I think it is also such a machine learning thing because if you get a machine learning, you run like-- I don't know, just run this thing like 100,000 times and eventually you'll sort of grok very basic mathematical approaches. But you have so many generations of actually running this. If you actually want to have a truly generalized model and like a truly generalized harness that builds itself, given the current constraints of the technology, seems incredibly inefficient. And so I don't know. It's like, I think that right now, I think they would be very narrow on the problem that they're solving. I don't think they would generalize unless you burn a lot of money on that. But I might be wrong. We'll see where it goes. No, I mean, 100%. And yeah, that's the other thing. When you think about the happy path-- yeah, that sounds awesome. But what is something goes on? This thing has like self-learned, self-improved its way through 20 cycles. And well, I don't want that. But then if you don't want that, then there's 10 other things that were linked to it. It just feels more complicated. You can just like machine learn your way to like something. You know, that's awesome. You started with 20,000 lines of code that you can still kind of sort of understand. And then a week later, you wake up to 20 million lines of code. And neither you know-- But maybe the lines of assembly, man. Yeah, that's an excellent. But like one part-- and it's like, this is conspiracy theory town right now. But one thing that really I find interesting is we're entering this sort of phase of recursive self-improved on the model learning part. And they're like, as part of the learning, they're supposed to be isolated from the world at large. It doesn't seemingly happen all the time, something that just out there. And but like the models at this point have sufficiently understood that the goal is improvement over time. So like part of seemingly what they're doing on the internet is like leaving evidence for the next training run. Right? They have this whole level of like, like, this is happening. And it is deeply weird. I don't know. I don't need my harness to also do that now. I'm personally much more bullish on model training self-improvements than application harness levels self-improvements, just because model training is super complicated, no question. But the individual parts of that process are of such a shape that you can easily define metrics as a goal to get there. Like if I have a GPU kernel for inference, right? This is 100% deterministically testable weather, a change to the GPU shader or to the kernel that runs the inference on the GPU, made things faster or not. And a couple of labs have already reported that they have some Hedict's warms improve their inference kernels and they magically get better by huge amounts of percentages and nobody fucking knows anymore how it works. I think with that sort of thing, like the plot is already lost, I like, you know how you keep saying keep the whole thing in your head. I think at this point, nobody knows how they say, so it's like, well, if you don't know how it works, just. But in this case, it doesn't fucking matter if it works because you already had from previous times, right? Your test sets, and you know exactly what the outputs given some inputs need to be, right? So does it matter? In electronics, we've done this for decades, actually, we use evolutionary programming to arrive at designs that are basically not really understandable by a human. And it works fine. And so why not do this for stuff like GPU kernels for LLM experience? I'm totally for that. I just don't know how this could extend to other things where the task or the goal is not as easily as scribable or very fine. I think like what confuses me a little bit in this sort of current way of working is, I really, really like the fact that for the most part, for as long as you have a single thread that's kind of execution, a computer, it functions very reliable and rotted it to my nest egg. And it's actually-- a beautiful piece of engineering because like electrons are not that deterministic. Like there's a lot of engineering in a TPU that gives me like a really great playground that should not really exist in the reality, right? It's like there's a lot of like weirdness going on, but somehow my TPU is very, very reliable. It's a masterpiece of engineering. So I think I was saying how small it is. That's a great goal. I like that. But now it doesn't matter anymore. Like just so far from wanting to another sort of like an organism. It's weird. I feel like why do we give this up so willingly? I don't know, but maybe this is just like I'm getting old. I don't know. I mean once this computer is back, just a simple new environment machine. And the great thing now, the great thing now with LLAMs, the great thing with LLAMs now is you can take this really old computers like a NES or like a Commodore 64 and you can like you can hold it a thousand times in memory and you can do like great stuff when it's a little boxes where you can look from the outside into them and like snap for them at every frame and do cool stuff with them. Like I don't know. That will not work with the present day computers. I don't think so. So we need to have another episode down like the philosophy of like you know very very are and the full people. Yeah, I know it's like the only audience is greater than like 20 years of. You're gonna bring your channel down so hard man. I want to try to do like a rapid fire of these burning questions I've had about pie. What feedback are you looking for from pie durable that would convince you that this is the way like we should just pie becomes pie durable. So yes, we are building this already and I think ultimately you need to talk food things. You can't rely on the kindness of strangers and their feedback to build something great. But we do want others to build on top of it as well and lots of people have surprisingly done that already. So we're getting quite a quite a bit of nice feedback and that's mostly about ergonomics and it's running in different execution environments like Versailles, Cloud Flare and stuff like that. We are not experts on these infrastructure pieces so anyone who can give us more insights on how they use this there. We might want to change some things inside durable or on the surface of durable to make this easier. I think we're also very interested like when people have external scheduling requirements and how it fits into it like async in friends. I don't know is whatever weird as limitation they have on the sandbox provider because the whole goal is like something needs to wake up and we wake up the whole thing and like the whole thing just sort of keeps working as it was and that only works as well as sort of like the design overlaps with the constraints of whatever system you're integrating it with. And there are a lot of like really wacky setups out there. Some of which are actually really great and they came to their wacky design because of their inherent sort of design in it. But then you might have an impedance mismatch where like what we think the system should do and what the other system does like totally doesn't fit together. So like learning about this I think would be very interesting. Do you think Pi will ever ship an official GUI or are we really listening to it? Yes definitely. Awesome. Do we know when? Soon. Actually real soon. Actually real soon but not until the 5-1 panel. Karin. Aaron Dell is also an interesting company in that you guys are building tools. Can you quickly tell us about like you know what are the things that we can hope to see from Aaron Dell and like what are some interesting products from that perspective? Other than Pi which most people probably already know of. We really want to make sure that this technology actually works for everybody and everybody particularly is not software engineers. So like I want my mother, my kids, my kids are using pineals. So it's like I don't want to see this being a technology that works for everyone without giving up your like preferably agency or your data. Like to do the degree that is possible. So we want to find ways to bring this what works really well in the coding space to more people. So we hope that like in some some period of time not too far in the future there's a version of a good AI interaction that we like that works as good for non-engineers that works for an engineer. Makes sense. So it's code mode. There is a mode. I run it this way. Code mode is the only mode that I run and do you both run it? Like when should I use it? When should I not use it? I just enable it on opus and GPT models or anthropic and and open AI models because they are trained on it now and for everything else. I stay vanilla. Yeah I mean we're going to see a little bit I think how the models behave. We know you can run deep seek harness and you can turn on code mode there as well. So it's pretty clear that they're doing something there and that will probably be more of it. But at the end of the day, I think like part of what working in the spaces, you have to figure out what the models really do and sort of take it. You have to feel it out. You have to be out. You have to get it out. You have to feel I keep telling the story about a 25 bucks USB stick from China with hardware on it and I spent more than a thousand five hundred or even like more than two thousand I forgot like dollars on I mean obviously subsidized yeah like this is disproportionate my side project but never paid that much money if it was actual money on the side project but like right now it's sort of free so. I keep enjoying it for the whole thing at last you are meant thank you so much I'm extremely grateful this this was much more fun than I even thought I knew it was going to be so thank you so much and I appreciate you thank you for having us.