9 Commandments for Building AI Agents

At any given point in time, there will be a bottleneck or something that doesn't work. It's like, "Oh, we need an authentication solution. We need a better rag solution. We need a better embedding store. We need the better model." Yeah, I don't think there's a simple vector where I say, "Well, in that space we'll never buy again or never build." How do you make sure that your agent has this idea, okay, as this is ambiguous? Actually, don't know what to do. So, I need to go back and force with a person to actually understand what the problem is. We always prefer to have this clear borders between things. Reason, act, observe, plan, act. It hardly ever happened like this. Today, we're talking with Paul and Dimitri about building effective AI agents and the design principles that go into that. Paul is the VP of AI at Process and Dimitri is a senior director of data science. I, myself, and Dimitri Os, the host of the NLOPs community podcast that you are listening to, let's get into the conversation. For all this talk of AI workforce, I still haven't seen enough agents helping me with my work. Your email has got to be basically your whole time. When you opened your laptop, the notifications that you had were overwhelming me just in the first five seconds. That would be helpful if they actually started to take away some work so far. They're helping a lot, creating more work. But it goes to that idea of the cognitive load and then how you can get this overload because information is so cheap now. It can be so noisy. I continuously referenced your idea on we plugged an agent into GitHub and we turned it off about six hours later because it was just constantly pinging and pinging. Super verbose, like giving all sorts of commentary that just then requires you to read that. Process it, judge it, and act on it or not. I call it like the agents or the LLMs in AI in general is disrespectful of my time. It doesn't recognize that you have a finite amount of time. It will be verbose and it will give you all of this information or it will ping you and just kind of like about nothing important. And so that discernment, I'm not sure if you figured out a way to get through that filter. Oh yeah, so I think there's a lot of interesting lessons from building agents for e-commerce where you show the initial experiments we were doing. We're just building, let's say a chatbot for, you know, shopping and ordering food for buying a car for looking through real estate. And it was sort of like you get people coming in and they say, you know, they're looking, let's say on OLX, right, which is our second-hand marketplace. And they type in, you know, iPhone 15. That's their start of the conversation is iPhone 15. If it's a search engine, enter. And then comes back like 10,000 tokens of, yeah, you know, we've got this one over here. And then you're, yeah, instead of just scrolling, you realize that that's suboptimal. So how do we deal with that to ask you a question? We're starting to go into much more of an intuitive UI with Gen AI, which is you give people some information, of course, but, you know, which is tends to be more visual, much more structured. But at the end, you have buttons. Is it, listen, you're now need to choose, do you want more options, different options, cheaper options, options, near me. And so you kind of pre-select what the next thing is the user wants to do. And you make it a clickable thing as opposed to then putting that sort of high cognitive load burden of them. I'm actually having to like describe all the other things that they want. So anyway, so this feels like one of the principles in building out agents. We want to talk today all about the different principles that you can have when, and you need to think about on a technical level with Dmitry. But then here, just like from your side of the fence, how do you enable the team to go out and make sure that they are fulfilling these different principles? And so when I think about this, I reference the last talk that we had with Flores and how you all are doing hackathons all the time. And you're trying to really push the boundary of what are some wild ideas of where we can throw in agents. Through that, I imagine you learn a lot. Yeah, there's a lot of different things here. There's a big open question. Yeah, we do a lot of different whatever thons, right? Vi-vathons, cotathons, hackathons. Oh, vi-vathons. We did do have a vi-vathon recently. They sound different than what I thought it was going to be for a minute. And so the goal here is, you know, the role of our team, right? So it's to basically figure out where can we move AI-interproduction, agents-interproduction, into real products, into the real teams that where is it ready? And so the only way to do that, because the field moves so fast, the tools evolve so quickly, is to continuously, you know, roll up your sleeves and try them. In an AI team, you may assume that we all know and see and use all the tools every day. And we try to, but if you don't actively spend time on exploring that, you also fall behind, right? We tried Devon a year ago, we tried, you know, Manus, the moment came out six months ago, we tried deep-seek all these things. And if you try them again today, they're significantly better, right? And many other tools that we tried didn't get significantly better yet, right? So having this sort of continuous drive to experiment to try to hack around and see if they work is something we have very much in the DNA of the team, to make sure that we also understand what's real and can go into production, the end our goal in the team is to make sure we build real things that are useful and aren't just a demo. And so to have the distinction of what's ready and what's not, you need to get your hands dirty continuously and try it all the time. I want to ask you about how you think through forward compatibility, because like you said, A, you test tools at a certain point in time. Those tools are how they are, but the teams continue working on them, the technology continues to get better. That's great if you come back to it and you keep revisiting it. That seems like a lot of work, but also there are things that in my head are continuously getting better and you can try to make them better and brute force this capability from the agents or the LLMs or you can wait and maybe it'll get better in six months. So thinking through the forward compatibility and how that looks in a future that's moving so fast, like the six months down the line, is there going to be a longer context window and then you don't need to think so much about trying to hack together a solution or is it something that you need to invest time into building yourself? Yeah, it's a great question. So at any given point in time, when we build agentic systems, AI systems more broadly, there will be a bottleneck or something that doesn't work. It's like, oh, we need an authentication solution. We need the better model, right? Maybe the model isn't performing at the same level. And you can think about well, to solve the model problem, like in theory, I could just go and train my own model, right? But I'll tell you, the fastest high value depreciating asset I know are LLMs, right? Like, companies spelling hundreds of millions of dollars on training these things. And then three months later, there's a new one. And they switch, right? It's a commodity in many ways, like whatever there's a new one, everyone forgets about the old one, right? And all the traffic goes there. So you don't want to end up in a situation where a small team, even though we spend a lot of time on building the right things, like, you know, we will train models, but only for areas where we are very confident that, you know, there's nobody's going to solve that for us because it's a specific domain, a specific area, or we need it to be efficient. So it needs to be fast and so on. In terms of social forward compatibility here, like, again, going through the list of problems, could be, I don't know, maybe the model we used today is too expensive. It should be there for not, like, work with that model, or should we just assume that in six months it's going to be, you know, 50% cheaper. And we actually operate under a couple of principles that where we believe that, you know, technology is going to kind of move in that direction no matter what one is models will get cheaper. That's a certain rate, and we make some assumptions on that. Second is, you know, model context will continue to get, you know, bigger. We've seen that happening. That's so true. And so you don't want to continuously optimize too much on the rag side and chunking and so on. If you know that eventually, you know, you have, you'll have an infinite context. You know, on certain modalities, we have some expectations of this voice or video or imagery that they will be able to get better over time. So you can already start to build solutions that today are 80% but soon they'll be, you know, 99%. So with that sort of intuition of where things will move, we're fairly comfortable to start building already and sort of take that in six months it'll be cheaper, it'll be better and so on. But there are some areas where we actually need to build a solution like software to authenticate as part of a rag pipeline to give you an example, right? We need to retrieve documents that are inside the enterprise, inside the company. And we need to make sure that only the, you know, certain types of agents can access that and use them for answering a certain question. We had that problem two and a half years ago. And, you know, we're pretty sure everybody is building these things has that problem, right? Because you need to also give agents that are permissions and so on. So then the question is, do you build for that yourself? And you're, so that is kind of the framework that you're working under. And that's why you continuously are revisiting the different tools and you're seeing, did they get there yet? All right, we've got our eye on these set of providers like the composios or the arcades or whatever it may be. You know you need off. And so you're keeping an eye on the space, you're testing it, then you're waiting, and then you're testing it again, and you're waiting and you're seeing, is it advancing in that rate that we're looking for? Absolutely. And, you know, there's just so many of them. And the problems are trying to solve are real, observability, rag authentication, you know, logging, and so all those things are real evals, real problems. And they all try to solve those real problems, but many of them aren't there yet. And so we try it whether it's a language, an arcade, a composio, and sometimes actually most of the time, unfortunately, we still have concluded that once we move to production, the tools are not there yet. But again, we're a small team, so I very actively encourage everybody in the team to make sure if there's somebody out there that solves this general problem, let's use that tool as opposed for us to build it. Unfortunately, often still, the tools aren't ready yet, but we're starting to see with arcade, a good example, which we know we're now adopting, because it's good, you know, good enough for what we want to build. And there's several others now in the tool suite that we'll talk about, but there's another thing, like actually what's in our benefit is that it's much cheaper to build software for, you know, much easier because we've got all these tools now. So in some cases, we're actually faster, better off, just to build, you know, our customized solution, with, of course, the devins, the curses, the win serves of the world, because it's much quicker than to go out there and evaluate five vendors and test their solutions, which are never sure, perfect for your, for your situation. I get legal agreement and do price negotiations, and so on. It isn't just only a economic, like, is it what's cheaper? It's like, what's faster? Right? And something we just need to move fast. And so we'll just put something together, and some parts of the software that's fine to build our own. But what we see is it's much easier to use a cheaper to build your own than it used to be two, three years ago where you just don't have time to build whatever, right? Your own authentication solution or a content management system, whatever, right? So are there vectors that you feel like have been completely demolished that you never want to build, or you never want to buy anything in that space because it's so easy to build and that time to production is so fast? Well, it's never say, like, always or something, or never, or whatever. I think you, you know, as engineers, I think we always have the natural propensity to build yourself, right? It's fun. We think we prefer to do that, and so we understand it. So that's kind of where it's been. And then, of course, as a team, you need to sort of make sure that you choose wisely, build or buy. And building is becoming much easier and cheaper. So that's that's one reason where for small things, we will build and ourselves more frequently. And buy hasn't been ready for many of the cases where concerned agents in production. But buy the space of options where you can buy, of course, in two years from now will be much richer, right? You have all these startups that will mature the solutions, will mature communities, open source projects, standards like MCP. And so we build basically like a solution for MCP a year and a half ago called tools controller. Right. And now we have MCP and we're grateful because now, you know, that makes all of our work much easier, right? Because there's an agreement standard to this. So I think there will be some areas where, yeah, I don't think there's a simple vector where I say, like, well, in that space, we'll never buy again or never build. It really will depend a little bit on also what others are doing with compatibility is with the models with the, you know, all their parts of the software stack you're interacting with. You've mentioned a few times different things that you're encountering and then later tech comes out. Yep. To or a open source project or the standard like MCP. Do you see the folks that are on the team building tools and specifically becoming more adept at the tool building process or is it more in the vein of I just want to have the agent and I'm going to build this agent that goes and does something for me. So right now we are definitely thinking about most of the things we build as a, let's say a set of building blocks that the end user can quickly stitch together to solve their need and I'll be giving a specific example. So our internal, you know, AI assistant token has been around for a while as we talked about in other episodes, you know, answered millions of questions for, you know, tens of thousands of employees around the world. And now people have, you know, lots of use cases as part of their daily workflows. Hey, I get this Excel sheet. I need to kind of upload it and write a memo around that like a monthly report or something, or I've got some customer calls that are coming in and I need to transcribe them to figure out, you know, what's being said in the center. So there's, you know, examples where people are starting to basically, you know, integrate this as daily updates, daily reminders of things that they're doing, right? Give me a plan and give me actually an agenda for the day, generate a.ICS file based on these things that I want to do that are going to upload to my calendars. I've got all my invites there. That's cool. So, but those are, you will be doing something even to me and, you know, the next person in the team will do something else. And so we are now with token building spaces where people have basically ability to create their own tools, integrate into their Salesforce environment, their Monday.com environment, their Databricks environment and have this general agenda capability that then they can point to those different tools to solve a certain problem. And many of those people are not, let's say, native AI practitioners, right? So we need to make it really easy for them. Almost, I think of it like GPTs, but, you know, custom to, for us to, to process, to commerce to the, our world, right? And that means that we have a certain set of systems that we integrate with, like I mentioned, Databricks, but also other cloud environments with data and them or financial system, SAP. And so on that, our standard tools that we make available. So anybody in finance can connect their part of that database or anybody in HR or the legal teams or the data engineers and so on to create their own agentic system. So then it's about offering people a general agentic capability with a set of tools and giving them a no-code way to describe what they're going to do. This agent or this space is workspace and this token workspace is going to be meant for as an HR support bot, right? And then basically this HR employee points to what data that HR support bot needs to have access to, how it should behave, they describe that, click and now it's published through everybody in process. And they can now get, you know, questions about paternity leave, about a performance review process and so on, created by somebody that has, you basically know, and didn't have the program to understand any of the underlying agentic systems to build it. You feel like as you are using agents more and more, it becomes clear that that is the way forward where these tools are going to be built with the no-code solution, but then you have the engineers on the back end that are making it as easy as possible for the subject matter experts to put their magic touch on it and do their stuff. That's exactly right. So we actually have this theme across process, AI workforce and Sean will talk about that, where we want to make sure that everybody's got access to their own AI workforce, you know, consider that everybody should be able to have, you know, a team of junior analysts, some interns that are agents, right, that they can create and have them work alongside them. And so we have very specific initiatives, you know, objectives for the coming period to actually measure a number of agents that are created across the company, how often they're being used, how many questions they're asking to actually push the adoption of these basic capabilities for anybody in the organization that, you know, doesn't need to be a software engineering, it's actually should be anybody. And we do that through token and many other tools, of course, as well. Okay. So going to these 10 commandments that Dimitri is going to talk to us about, I wanted to get specific on one that potentially isn't understood the same way by every person, which is the memory piece. I know there's almost like two ways to think about memory. One is I know that you like buying a certain type of shoe or I know that you like these movies, traditional kind of recommender system. I remember that you do these things, and so I'm going to know that for the next interaction with you. And then there's another memory, which is that agents remember how to do things. And so they have this tool in their tool belt, like you were saying, when you're connecting the different systems together, it would be great that an agent learns how to do something within a system. And then it always has that in their memory, so that it's not just guessing when it is trying to accomplish that same task again. This is such an interesting topic. I think the memory piece is, you know, it's an obvious area that we need to build that once you start to think about, you know, the types of memory that these agents need to have to accomplish certain tasks. They become, you know, it becomes much more granular. And in a way, nuance complicated, what you need to build, short-term memory, long-term memory, you know, like character, right? How do they behave over time and so on? So there's lots of things to be said. Let me focus on the piece of the agentic memory, which is related to getting better at a certain task, which I think was where you were going, right? Because it's also such a other memory, like memory about me, because you talked to me 25 times and so on, but let me focus on getting things done for you better and faster, whether it's, you know, helping me with the PR and the software world and the agentic system like Devon or Manus or the e-commerce agents that we're building with the large commerce model and so on to help you find what you need. I think that overarching sort of thing that we're seeing here is that these agents will are now starting to learn from their experience with the world. And what does that mean? If I go in and use an agent in a commerce setting like we do with iFood and OLAGS and take a lot in e-mag and the companies of the process group, this agent, let's say I come in with a search query and, you know, you're going to have a conversation with me about this, I don't know, let's say an apartment you want to rent in Polznan in Poland and it will help you find what apartment. You'll say, "Well, I have this kind of requirement. It needs to be 60 square meters. This is my budget. It needs to be in this area." And you sort of refining that and eventually, you know, you'll know whether or we, you know, the agent will know when this person contacted the renter and said, "That's a successful event." Right now we've helped them something that they thought it was interesting and in some cases they also know when they actually, you know, made the agreement to move to that apartment. That journey is now something we can see that that was a helpful, agentic journey. There's also other cases where the person doesn't find or it doesn't succeed or we ask the agent to help browse the web, right? But it didn't find the information it needed and so on. That's also useful because that's a negative example. Like in this case, it wasn't helpful for the intent of the user. Those experiences, if you store them, you can actually train your model to do better reasoning, right? Because all of these are reasoning agents now, right? So they have a model that they, the first thing it does is it reasons like deep seek. We've all seen it and now a 1, 0, 3, 0, 4 and so on are first making a plan and thinking through how should they approach this problem. Now that's generic reasoning. It's great. It's generic reasoning for answering a certain question that you, any question on Chatchubt, Claude, Gemini and so on. What we're trying to figure out is what does it look like if you use the experience of the agents storing when it was successful or not successful? Those experiences to fine tune the reasoning. So now the reasoning becomes really good to help you build a certain, you know, type of software in your coding environment, right? Because you always care about having these standards or it becomes really good at taking a query in food delivery, search query and helping the user find what they actually were going to want to order. And when you say fine tune, you mean actually renting GPUs and fine tuning. Oh, yeah, absolutely. Yeah, but training is not just throwing you into the training models. It's not even just only fine tuning. There's all of the training variants you can imagine that we're working with now to basically create models that understand what the best path to success looks like. And what we're able, the reason we're able to do that is because we can actually very quickly gather data, although it works or not because these are, you know, real experiences, real products in the world we've got. And so that allows us to create that fly. We'll collect experiences, got to the objective or didn't get to the objective. And sort of a, it's sort of a reinforcement learning, not in the traditional technical reinforcement learning sense, but you do store the successful path. And that becomes input in the next training round. And you're saying that is creating more memory where that is, it's like inherent memory in the model now. Right. And so that is, it's a form of memory because you're storing all the successful paths to a destination, right? Or a successful action. Because we have got, you know, hundreds of millions of interactions with users, those paths are store their memory, right? I actually learned how to help somebody find their shoe, right? I actually learned where is the best way to help advise somebody on finding an apartment to rent in Poznan. I'm sorry. I thought you meant find your shoe, like you lost it. Oh, no. Yeah. Sorry. Like buying a shoe, right? So finding the shoe that you want for your next race. I lost my lesson example. Yeah. And I needed to find it. The model helped me. Yeah. And so yeah. All right. Or somebody says, Hey, I want something healthy, quick for lunch at the office. Yeah. Like that's a search query that, you know, surprisingly, I guess, surprisingly, we're very bad at like answering today, right? What's healthy for me? What's, what is quick? What's, yeah. And does it matter if you're at the office? Does it matter if it's you versus me? Like all these different things that by, like I said, being very deliberate about what memory you're creating and giving the agent access to can make a big difference. So I hadn't heard or thought about that way in thinking about memory as you're taking successful journeys, you're also taking the non-successful journeys. And then you're giving the agent, you're spending cycles on fine tuning it so that the agent has that part of it. I had always looked at it as something that you do after the reasoning model is there or after you've set up your agent. And it's just like memory that you're bolting on top of it with caching it, for example, you can do that. So you can definitely do that. So some areas where we've done similar things is, you know, we've been playing with web browsing agents like many others. So you give it a task, go on, I don't know, find me the cheapest iPhone 15 seller today, right? Or help me order, you know, salad for lunch. Let's say in the food running space. And then this agent goes, starts to browse, it goes to the various websites. And what we saw initially was, you know, wow, like we tried this 50 times. And it only succeeds, you know, 10 times. And maybe sometimes five times because it got stuck on a capture got stuck on the didn't scroll. It didn't really find the items because the search term it used wasn't good. So, but what's cool is that you give it a simple task, go and find me a cheap healthy lunch to order to my home. Then it goes to all the food delivery providers. It looks at the restaurants that are open. It goes and browse the web, right? Like your I would. But that search space is actually pretty broad, right? It will take all these steps in these paths, if you will. And out of the 50 tries, let's say five are what you would have liked or you would have done yourself, right? And we then store those five. So that next time somebody comes in and searches for food in a general sense, it will access those five paths as a reference successful way to get to that. There were slightly different things because maybe I was ordering in a different area or in different country, or you know, you gave a different preference for whatever you wanted. Or the website got updated. Well, the website got updated then yeah, exactly. Then you screwed. You're gonna, well, you're gonna find, you know, five more ways not to do it, right? Not to succeed. So, and that can be accessed perfectly as it sort of, you know, in a cashed way. It doesn't need to be trained in the model, right? We also try those things. But I think the important piece is once you start to be delivered about storing what's successful and what's that isn't the path to get to successful outcome, you can then feed that into the model either as a cash or through rag or eventually if it's large enough, like in our case, we actually also train models on that as well. It really goes back to what I was talking with Yone's about yesterday and he said, "Oh, my take is that e-vows are your mode." And so the better that you can get with your e-vows, the better you can expect your agents to perform. It's a really important realization to that if you don't know whether what your model or your agent did was good, very hard to improve. Like, that's simple, right? I think that notion everybody understands. But then what does that mean in the real world? That means you need to actually get feedback from the users or know what they were trying to achieve or end know what they were trying to achieve to know whether it's successful or not. And because we've got, you know, basically two billion consumers that we serve across various parts of the world interacting with the platforms in different ways, we do have ways to know whether they got to their outcome. You know, did they find what they were looking for? Did they buy something? Did they contact a seller if they're on the second-hand marketplace and so on? That is an evaluation, right? E-vows is a technical term we would use as AI development. Basically, was it good or not? Was it successful or not? And that flywheel is indeed a moat because that means you can then feed it back into your agentic systems. They can use that to get better the next time the similar question comes in. And I like how you're saying there's many different ways. Once you know if it was successful or not, which is step one, then you can figure out how to incorporate it into the technical side of the system with caching it or updating your RAG system or fine tuning it. I also like that you are hyper focused on certain tasks or certain verticals per say, for example, if it is any commerce and you're just trying to figure out the path of one of these food delivery systems. When someone is using that, you know there's only a select amount of things that someone is trying to do when they get on to app and it's probably along the lines of ordering food or ordering travel and so on. And we found that especially the generic agents, the broad ones, they're very bad at some of these specific things. Like if I actually wanted to get some help in booking a flight, the agents get stuck. Things that we think are very easy, these agents fill it. So an example is scrolling is not something they automatically do or understand or think about. If I go and select my departure and arrival destination airport or look at the dates, typically every travel website is a little pop up, drop down, yeah, scrolling, pick up calendar, whatever calendar you pick up the dates, the agents suck at that. And so we were trying to do, you know, those help these do those tasks and sure, you know, you could perform at, you know, 50% on OS World benchmark or whatever other, you know, web browsing benchmark, these agents are being benchmarked against. That actually doesn't translate very well to the use cases we were helping, we are helping our users with. And so then storing this memory, right, and say, actually, this is the way that you help the user find the flights from A to B and you need to scroll, right? Or you need to actually translate these dates into an action on, you know, a grid with a calendar and so on. Those are the kinds of things that we need to build and be very specific on to get to a certain level of accuracy for the user. Because if it's not like, I mean, this case, 95% successful or useful, people won't use it, right? If it gets stuck all the time, it, you know, and then it was the wrong flight. Yeah, exactly. It doesn't accept cookies. Yeah, if you're in Europe, you need to accept cookies. I know that. So they haven't been trained on enough European data. Yes. I want to bring in the experts now, though. So let's get to that conversation. Action live. We are here with Demetri in the studio. It's great to have you. I want to talk all about this unified rule set for building AI agents that you have put an enormous amount of thought behind. When I saw it, I thought, man, this is so good. We need to have a full conversation and podcast and break down some of these. Hopefully we can get to all of them, but I want to hit the most important ones to start. That being said, AI agents is a contested term. You have a special definition for it. Give it to me. Yeah. So, well, thanks. First of all, thanks for having me here. Yes, I think that for me, AI agent is a solution that is able to achieve a task by selecting by itself the path towards the goal and also by defining itself what the end is. So where to stop. And I know that it's not complete and you can have so many corner cases where you would point to a simple piece of software and say, well, it's kind of does that. And I know that some people say, well, what would be good to add here is memory. So it can learn from the past. It can learn from its mistake and do better. Yeah, I think it's a fair point, but again, there are so many cases where you say even without memory, you can build a successful agent. Yeah. It does give you that spectrum of it's autonomous in a way. It also knows when to stop. It knows when to get more information. It's not a workflow that just gets kicked off. And it's not something that you have to hard code and say, do this, this, this, then this with a little LLM sprinkled in there and then you call it an agent. Exactly. So not everything which uses LLM and a flexible execution path is an agent. So we need to be careful here, but also not everything which has certain rules, predefined rules is not an agent. Yeah. There are a few fundamental principles when building agents. And I think you with this document, you put the most fundamental at the top. What is that? Well, for me, the most important part is how does a cycle of an agent look like? So when we started building agent, by the way, we didn't know it was called agent yet. It was couple of years ago. And whether it is a piece, I believe that you talked to some of my colleagues about data analysts, right? And we essentially implemented what is now called react cycle. So reason act where we say, you know what most of the problems that you need to address, they will be implemented by doing to separate parts. One part is what we call comprehension and reasoning. And another one is execution or acting. This is why react as a name is still a very popular concept. And depending on a task at hand, you will have a very different distribution of complexity between comprehend and between execute of big win reason and act. So let me give you an example. If you think about data analyst type of agent, so something which takes a problem and tries to extract information and provide you with an answer, significant, the most complex part of this would be comprehension. So how do we take what user asks and bring it to the point when we know what we actually, what data we need to retrieve, what report we need to write. It's very difficult. My favorite example, it's real one, somebody goes to data analyst and says, what is the fastest growing company? fastest growing by profit, revenue, people, sales, fastest growing in last month's, last year. What do you mean, right? And how do you make sure that your agent has this idea, okay, as this is ambiguous, actually don't know what to do. So I need to go back and force with a person to actually understand what the problem is. And then once you arrive to understand what person the person actually means, what is the fastest growing company from our portfolio based on a revenue in the past year. This is very easy to give to LLM to say, okay, please translate this into SQL, for example, get this information and then I'm going to combine it and present it in a report. So for this type of task, comprehension is extremely alone and difficult and complex and sometimes almost impossible. An execution is somewhat simpler. You can also look at a simpler task, let's say, you give your tool a document and say, you know, extract all information and tell me couple of insights about A, B and C. In this case, the comprehension of a task is very trivial. The execution still not as complex as in a previous example, but bigger than comprehension. So the distribution is different. And this is the first thing that we did and I think react was the first kind of a conceptual approach that we saw from most companies implementing when we start talking about agent. And then we start having more discussion, okay, but what else is there? So what I like to add is observe, think, act and then reflect. And it's more or less taken this initial things, so comprehension reasoning and splitting it into two parts. One is observe and this is understanding user and circumstance. Let me give you an example. We did a small demonstration where we created an agent that can order food for you. So you sit in the office, you basically click a button and you say, hey, I'm an office, I'm late. I won't mind my usual. And this is it. And 20, 40 minutes later, there is a career at the door and basically you've got your usual stuff. So significant part of initial agent work is actually just looking at understanding user circumstance. So saying, hey, this is the materials. He is in process office. It's late. So it immediately shrinks the space where solution can be found. And then you come and say usual and basically just solve it. So it knows what to do. It handed over to execution part and then basically after it's easy. So hopefully it illustrates a bit of the observed part. Reflect part is something that is not usually implemented. But at the moment, you see it in very specific scenarios. The best example to give is if you've got a code in agent. Because code in agent, you've got an assignment, it's executed, writes a code and then you reflect on a code whether it's by trying to execute it or trying to write an execute unit test or whatever. It sees the result of execution. It says, okay, I'm not doing well on one or two or three matrices. So I need to repeat the loop. And this would be a part which is difficult to do without reflect because then you more likely to create solution with errors or suboptimal solutions. So the observed part is really this filter of do I have enough information is the ask clear enough. If it can pass through that filter, then you can reflect and say, can we set something up so that if I do not have some piece of information, I go back and I ask that. And so it's almost like you're getting through the observed part. There's one filter. The reflect part is almost like another filter on top of it. Reflect part is at the end. Once you execute something, like what did happen during this execution? I see. Yeah. The observed part starts at the beginning. So when the stand, what the circumstance we are in and then think part is do we understand what needs to be done and how it shall be done. And think part is more often connected to discussion about ways that we've got static planning, dynamic planning, we can touch upon it later and so on. One of the things that I think people find confusion, we always I think prefer to have this clear borders between things. So we say a reason act observe plan act observe think it hardly ever happened like this because quite often when you think about what you need to do or you think about context of request, you actually also need to do something. Allow me to again illustrate it with examples that I mentioned before data analysts. So what helps if there is an ambiguity in the request that analyst can go read some documentation about company, maybe go to database and look what data is available. So it doesn't go come back to you with this range of random questions, trying to decrease ambiguity, it basically come back to you with more precise, but do you mean this or this? Like this is defined like this and so on and then it drives your kind of communication with agent. So it creates this loops. Yeah. And sometimes the loops can be very small other times they can be much bigger. But the key is that you have various loops happening throughout your agent journey. Yeah, absolutely. What I put there as a cycle is not prescriptive in the sense you always have to do it. So there are some use cases where you say, okay, like reflecting is not critical. So it will not break some other cases. You say without reflect, you actually never get to this top performance. In some cases, you say, you know, observe, it's more about personalized and whatever. Like if you've got a very simple task, like gives this recording, transcribe it and translate it. How much context do you need to understand? Probably zero. You just need to think, okay, so I need to actually call transcription tool. I need to call translation tool if it's separate from a lemon, then I need to basically call something else and I'm done, right? So observe and reflect kind of these additional things that depends on circumstance could be make or break stuff, but the core element is still. But are you looking at that as the agent will figure it out or you are trying to understand the use case and then at the moment, I think that based again on our experience, it's still a bit with a developer. So people who build the agent, they say, okay, this is what we need. But of course, the most sophisticated you've got agents, the more you need this freedom indeed. Now, there's another design principle that you have, which is extracting all capabilities as tools. And I liked this because it's not just other systems or other tools that you can use, like a Gmail tool, et cetera, but even humans can be a tool that you use if I am understanding it correctly. I think that you are more adventurous, let's put it this way, than me. I did not think about humans as tools to be honest. I was thinking about other agents as tools. This was a bit of limit of my risk taken, but now you're absolutely right. So I think that basically if you look at agent, what we again discussed at the beginning, it's important that agent has all various tools available because it will need flexibility to decide how to execute a task. And I think that in the simplest way, everything that it can call, everything it can use, where is a mechanical or human, it's a different story, could be and should be abstracted away as a tool. With some simple interface for input output, we recently had some ideas from various companies of how it may look like, but essentially, if I call a specific tool, it's called via API. If I call another agent, by the way, it could be also done as a tool. If I ask human for help, in a way, you can abstract it in a such way that it basically looks like a just a call in a tool. And then you've got a lot of freedom. I think. Yeah, I was thinking humans just because it's it's almost like if you think about human in the loop in a way that is kind of a tool because you're getting the okay approval, which can be a tool. I absolutely agree with you, so I see both sides of our argument. So you're basically saying look sometimes when we give a request to agent, we already in a way provide our self as a tool. Yeah, and I can relate very simple example. Sometimes I'm using one of these agents to help me to create a documentation. And what I put in instruction for my request and saying, if you need more information, ask me. So this is explicitly saying, okay, use this interface. So send a question to me, and I respond to you. And then away from agent perspectives, there is no difference. Send in me text and getting text from me or sending it to some API text and getting it from there. In the same way, if you think maybe another similar example, so you use operator-like system, quite often they specifically say, look, you give it an assignment, it will execute, but if it gets stuck, it will ask you. Exactly. And it could be in a specified point, for example, when you need to provide a payment or it could be in a point where it says, you know what, I don't know what to do. So it's like error resolution. And in this case, again, I absolutely agree with you. It uses you as a tool. That's it. So the greater theme here is, how can you think about abstracting as much as possible a way to make everything a tool called? Yeah. Tell me about the sandbox idea. For the code. Yeah. So I think it's also very important. So what we saw over and over again, and again, it's not only us, you can see the same idea when you are using other products like Antropic or OpenAI is that you've got tools, but sometimes you've got tasks that cannot be executed by a single tool, specific tool. And this is where you want to have a flexibility to just write a code. And you see it's happening. You basically have a code executor with limited capabilities to actually write a code if you want to create graphs. If you want to run data analysis, if you want to ingest data, I don't know, many different options. And I think that this is critical because it addresses this gap in, okay, there are so many small tasks that you cannot foresee. First of all, and build it into standalone tool. And second of all, you actually don't want to overload your agent with a list of 1000, let's say tools it can select from. It's much better to say, okay, you've got these core tools that you often need. And then whatever other, what 5% of scenarios you can cover with doing code. You build it yourself. Yes, you build it yourself. The agent builds it. And this brings up this question of when is enough enough? I think we mentioned how right now there's a very popular thing of long running agents. And the more time that you give it to think and to act, the better the outcome. And you had said there's a scenario where someone gives an agent a lot of time to try and execute a task. And it realizes there was a library that it needed or it needed something merged, PR request merged in a library so the agent went and sent an email. Well, look, this is anecdotical. It popped up in my ex-fit, I think. So let's take it as an anecdote, but I absolutely believe it when I read it because this is also something that we saw in some instances in our work. If you give a task to an agent and you do not limit execution and it's dynamic execution paths. This is, by the way, probably where we need to talk about static plan versus dynamic plan. Yeah, it's a good point. Yeah, yeah, bring that in. Yeah, so one of the things that people work on, a lot is when you have a task and it's as a task is given to an agent. The agent has or think about the task and then there are multiple options on how to go about it. One is you create a plan, execution plans in a case. So this is my data. I gonna call this tool. I gonna take this out of this, call that tool, that tool, that tool and then I come to an end. This is more or less referred to as a static execution plan. It's actually very nice for simple, predictable tasks. Again, going back to one of my previous examples, let's say transcribing and summarizing the meeting. Well, it's very clear what needs to be done. There is no surprise between a function calls about outcome. So you can do static planning. It's really good in a sense. It increases chances of convergence. So the agent will reach some result. But of course, for longer tasks, there is increased risk of running into error in one of the intermediate steps and it cannot correct because the plan is fixed. Now, the alternative to it is a dynamic planning. So you basically say, okay, I understand what I want to do. I know my first step. I gonna call it and then I gonna observe result and I gonna call it again, something again and so on and so forth. Dynamic planning is great in a sense that it decreases probability of getting stuck on a particular error. But it increases probability of agents never convergent. So it can constantly go and go and go and this is something that what I say, how do you explain to agents this very human concept that better is a worse enemy of good. Right. So because it's in theory, it can go forever trying to improve on your task and trying to get better and better results. So now going back to this anecdote that you mentioned, let's say you've got agents that want to write a code and it can write a code, it can test, it can see if it works or not and then improve and so on and so on. And this agent has access to internet. So it's not a big leap to understand that at one point it's a you know what I can actually google stuff. And we see already now if you use I know or three, for example, it does a web search if it runs into an error, it cannot resolve from its brain. And it does web search and then if you give it also permission to, I don't know, write an email, it can contact people asking for help like package developers. If it can go on web beyond web search, so with tools like operator and so on, it can go on Reddit on forums, it actually post questions, it can. So again, back to what you mentioned earlier, it starts seeing internet and human as tools to achieve a goal. And if you do not create any bounds on what your next step could be, then anything can happen. And again, it won't be stretched to imagine if you give it credit card, it will hire somebody to do a job for it. Yeah, it will go on fiber and then say, yeah, can you do this for me, which is a fairly interesting piece to look at because if you would like the best result possible, potentially you're okay with that and all of these outcomes. But now like you're saying is there's that fine line of how much better can we get it if we let it go and we let it hire someone on fiber and we let it post on Reddit, all of that type of thing when it gets stuck or maybe it's just that it needs to incorporate you the human in sooner and say, I'm stuck here. Do you have any plans for how I can make this better? So in that design principle, thinking about when to incorporate the human in or how to say enough is enough is something that I think is still is that an open question in your mind or have you figured out a way to do that? No, I think it's pretty much open question. I think that when to stop is an open question. I think that if you just leave it to the model you observe to behaviors typically, one is it's overconfident it's achieved the goal. Generally, if you look at the lamps nowadays, even on a step of reasoning comprehension, it's very difficult to get a model to acknowledge it doesn't know something, right? So you ask, okay, do you have enough information more likely than know it will say yes, even though it's obvious like it's not enough information. And the same way, if you ask a model, okay, you execute it, do you think you did well and get to a good result? It's more likely to say yes. An opposite side is and again, it depends on what kind of instructions and guardrails you put in place. It will say no and just continue. So it's very difficult to strike a balance between this overconfidence and a constant doubt. And I think it's still an open question. Well, it's funny because that was one of your pillars of what an agent is is knowing when to stop. Now, talk to me about the memory piece. We talked with Paul for a minute in the beginning part of this episode on memory and how you have these two different types of memory paths. You also had mentioned this before there's the memory of being able to complete a task in a way that it can do that reliably each time. And Paul was saying, well, this is where for us, the evals are so important because if you can say what the path that got it to success is, then you can update that information in the models like abilities or toolkit in various ways. Maybe it's through caching or maybe it is just updating it in the rag system or it's going as far as fine tuning. So for you, how do you look at memory in the space of design principles for building agents? So when I look at memory, I think about a couple of things. First one, you've got a short term memory. I borrowed a term from somebody that I like. It's called scratch path memory. And it's a memory that you use within the task. So basically, you start with a blank list and as you execute the task, as you call different tools, you interface with a user, you basically write down what happens, right? So this is short term memory from within a task. If it's a conversation, it's a conversation memory and so on. And then you've got a long term memory. And a long term memory is basically what happened overall across many interactions, across different sessions, across different tasks. And for me, this memory, I also looked at from two different angles. The first angle is more or less personalization. So you've got an agent and it executes tasks on your behalf and it learns something about you. Remember going back to observe, like, understand user and circumstance. So we're talking about getting intelligence about user. So next time you ask my favorite, it actually knows what your favorite, it knows your communication styles, it knows a lot about you. So basically preferences. It's preferences a bit. People sometimes call it like profiles, memories, whatever. But it's learnings about you that needs to persist and needs to be used later with your execution task. Now the second type of memory is, okay, I did this task. What did go well? What I need to do better. And this is not related to you. It's related to the ways that agent executed the task, allow me to illustrate. So again, going back to something like creating a report based on some business data, an agent looks at conversation, says, you know what? I spend a lot of time trying to search for city which didn't exist because I didn't know that there are unique code or that the city name could be encoded in a data storage differently. So next time when there is a task like that, I first need to understand what the spelliness, if it's correct or not, what encoding is, and I need to verify that the city is known before I start pulling all the informational source looking for something which is not there. It's almost like the agent is doing a retro on its own. Exactly. Exactly. And this is also a part of reflect. So when I was talking about cycle, I was given example with code writing and then reflect was really within the loop, but you can also have reflect a bit outside of the loop of you will or at the end of overall execution, just looking at everything that happened so far saying, okay, these are my learnings. And I think that this is a very important part. You remember at the beginning I said what agents are and one of the elements is memory and memory of in a way of learning. And this is a part of what contributes to learning. Reincorporating that learning into the next time that the agent does that, how do you have this retro, the retro, or the agent now understands, I could have done it better in these ways. What are you doing to update its understanding so that it does do that? So essentially what you do is you put this information into context. So you essentially change instruction to your agent. So you've got a play saying, okay, if you, so you've got this task and by the way, and this is also where rock comes into play. So you need to, eventually you will not be able to put all memory in, where is its personalization memory or execution memory or whatever. You need to find elements pertaining to user and to task at hand, which are relevant and you pull them from long term memory and you put it in a context and you say, okay, this is what you need to take and so on. Because basically you're saying, hey, by the way, the last time you did this, you said that you should have done these things. Make sure to remember to do that. Yes, essentially, not, not phrased exactly like that. More prescriptive say is saying like a good practice in this case to do this and this. You verify for this and this, but yes. Okay, fascinating. And you also go through the memory. So it's not necessarily part of agent execution. It could be part of environment. It could be something outside where you say, okay, so you've got this learning's learning's learnings, but you need sometimes to compress them to generalize them because you cannot have like 100,000 execution and then you've got 100,000 learnings 90% of them are the same, right? So you just need to be able to generalize and compress. Yeah. And so then it's like this reflect part plays into another one of these ideas. Tell me how it's different from the internal critic. Is that the same thing or are they separate? It could, it could be again. So I was trying not to separate internal external reflect in a way. So if you think about agent writing code reflect is in fact internal critic. So it looks at the execution. It says, okay, this is what this is what the test shows within do well, right? It could be also a look at something in other scenario to look at a document written and say, you know what the instruction of user was to summarize this document but convey this emotion. Did we do it or not? And it says now this emotion is not really properly conveyed. So we need to do another step. So is this is defa in fact internal critic, but you can call it also reflect, reflect. But it's reflect within execution and then you've got another example which is reflect at the end of execution or on outside. And you can mix it obviously. Yeah, there's a diagram somewhere in there that maybe we'll have to create where it shows these loops that you're going through, especially with the way that you're thinking about it with, you know, the observe and then the thing and the other one was act and then reflect and maybe reflect is at the beginning, maybe it's at the end, maybe it is at the beginning and the end and it's continuously reflecting after each step. It does sound like that can get expensive or it does. It does. And this is why. So in principle, this is reflected at the end, you can you don't need to do it at the end, you can do it on every step as well, it's just extremely expensive. So many solutions that I've seen so far and many solutions that we've built, I actually doing it at the end and it's simply because it delays it at cost. But yeah, the more you can reflect the better one, you just need to make sure that eventually you converge and not reflect, reflect, reflect and again, better it's worth a minimum of good. One thing that I wanted to say just based on your question because you refer to what Paul says, learning from previous execution. So one of the things that you can do with a memory is actually optimize your execution path. So not only go for trivial things like the one I mentioned where you say, okay, to avoid this mistake, like this is a good sequence of step of whatever, but to actually optimize execution path altogether. An example of this could be you spoke to Flores, right? So he at one point was working on an agent that uses web to browse websites. And essentially the way that we implemented it is if you got a task, your execution is to go website and start basically browsing it. So finding out what is clickable, what the fields are that you can fill information and so on. And then you basically said, okay, so I need to click on the menu, I need to look for restaurant, for example, in foot or ordering case, and then I need to click here to add to a basket, then I need to clear, so all the steps. But of course, once it's done and successful, it looks at the whole execution and says, you know what, I actually don't need to wait. And at this step, I don't need to wait until the whole web page is loaded. I can just put a trigger saying, okay, once this element loaded, I can activate it. And then I can do it next, next, next. And suddenly instead of spending two minutes browsing, you can do it in under 40 seconds, not wasting resources of website, also not waiting resources of yours. So we moved from slow execution to fast execution. And then another thing that happened, it looks at the slow execution, say, you know what? Actually, if we do this sequence of steps, we can predict what where we end up on this website because actually website is a limited amount of pages of use essentially. And this place where we end up has a specifically formed URL that I can form already at the beginning. No one where I need to go. And this is what we call reflex. So next time I've got a task, when again, I'm going to say in order me usual, it says, okay, I just go there. And in parallel, I'm also trying to do this fast. And in parallel, I'm also trying to do this slow. If I'm good with reflex execution, I'm good. If not, I fall back on the fast. If files fail, I fall back on a slow. So slow, an example of this would be website changed, element changed, right? But this is also an example where reflect is very important. It's not critical in a sense that your application will not work without reflect, but it's very important in terms of optimization of performance. And this is why again, memory in relation to execution is important. Yeah, there's so many things that I want to comment on there, especially because if you think about the way that that happens, if it gets done with reflex, then you, how do you call off the rest of the actions? Like how do you call off slow and fast or? It's to be honest, this is this depends on application and it's just implementation of detail. So the concept itself is actually not new. I saw it many years ago. I believe Siri was working on Apple this way. When you remember the early version of Siri, you would say like call mom and it would call mom. Yeah. So what would happen as opposed to now? It doesn't do that for some reason. It's been five years and it got worse. Model change errors don't. But essentially, if I understand what happened is it would take your voice and it would start processing it locally on the phone and at the same time, it would send it to the server where you've got much more powerful processing. And then if the phone says, okay, a process, I actually understand what needs to be done. It just communicates back to the server saying, okay, I don't need your support a stop. Yeah. And it's a bit of waste of resources, but it's significant optimization in terms of how quickly you can get resolved. And the accuracy of that result. Yeah. Now speaking of wasting resources or budgeting, I wanted to touch on the idea of how you think about budgeting every action. And I'll bring this up because I spoke with Zach probably three, four months ago. And he was putting together different agents and an agent builder at his company. And when folks build agents at his company in staging, they then have this little number that says, if you are to push this to prod, we estimate it costing this much money because of the scale and the amount of LLM calls that we're going to be making and the resources needed. You were looking at budgeting in a different way with long running tasks and being able to say, if you exceed a two-dollar budget, then just stop because I don't want this all of a sudden to get racked up to a $10,000 open AI bill. Yeah. Yeah. So I think budgeting is extremely complex questions, to be honest, because one thing is to say, you know what? We just want to limit the cost of execution. And I think it's not particularly difficult. You can estimate cost of execution again, as you just said, based on number for LLM calls or maybe some other things that it does. We actually did it again going back to one of these things with a food ordering agent. We actually can very precisely say, okay, so it did this food ordering. It looks for options. It reached out to restaurant. It looks the way it's easier to deliver an order via platform or via restaurant directly. And it selected the best option cost-wise. And this is the order that was made for this amount. And from there, we can also say, okay, but it runs so many calls. We know on average what infrastructure costs. So we say, okay, it's actually a saved user like $8 and it costs 1.5 to run. And you can put a very easy cap saying, you know what? On average, we are bringing this value for user. So if you exceeding this value, please stop. It's business discussion, it's not technical discussion. This is one thing. Where budgeting becomes extremely difficult, in my opinion, is when it becomes a part of trade of discussion. And unfortunately, this is where I do not see a lot of interesting solution. And please tell me if I just miss it. So remember, agents have tools. And if you look at most of the cases, now in the market, these tools are not overlapping. So you cannot replace one tool with another. What happened if you can? Very simple example. You can have three image generations. Each takes different execution time, each slightly differs in the quality of delivery results, and each cost very different. Did you actually see a good solution that is able to navigate this trade off? What is a task at hand? Which tool is the best option in terms of speeds of answering quality of answering the cost? So this is where I think budgeting will be extremely important moving forward because we build, again, we build agents with more and more tools. And internally, I already see a couple of use cases where we've got two tools doing more or less the same. So how do you let tools choose? How do you communicate what is important? What if the importance changes over execution of your task? At one point, again, going to human example, how do you explain to the agent to an agent, sorry, the concept that it's better to have an answer now than a good answer later tomorrow or next week. Exactly. So this is where I think budgeting will be important. And this is where I still see a lot of opportunities for development. It's almost like you want an urgency knob that you can dial and you can say, I'm cool with this one going as long as you want or I need this as soon as possible or somewhere in between. Yeah. So if it cost me less, it takes me a week and it's not critical task fine. If I need an answer now, how urgent is my now? You see how it's very difficult. Also, if you think about a human, you work in a team. How easy it is to convey and also understand the sense of urgency. And people say, oh, I pay whatever, if it's done, but then whatever is, whatever it's done, and then you realize, well, it's actually not whatever. And it's not really done. Yeah. And whatever is different for different people and definition of done exactly. It also reminds me of the paper that I read back in the day called Frugal ML. And there were a few different ways that they were exploring how to bring down the LLM cost. And some of that was by throwing in two questions in the context window to similar questions and getting the output. And then another one was being able to have a dynamic router. And so if the question was simple, it would go to a simple open source model that was a smaller one hosted, self-hosted. If the question was more complex, it would kick it off to back in those days, it was on a reasoning model, but it was something of the sort, the bigger model. Now, what you're saying, though, is imagine if tools had that same kind of router or it just was a capability that an agent could understand these are my options. And considering I need X amount of urgency versus Y amount of budget, I'm going to choose this tool. Yeah. Exactly. It's exactly that. And I'm really eager to see solution there that actually addressed it. But yeah, we're still in a phase when we've got just a handful of tools. And again, you don't really have these options where you can trade one characteristics for another. That's all we've got for today. But the good news is there are 10 other episodes in this series that I'm doing with process, deep diving into how they are approaching building AI products. You can check it out in the show notes. I leave a link.