AI Coding Agents Can Do So Much More (ft Kiran)

your goal with your whole dev setup is that you have a pipeline that is very clear about did this work or did this not work models like to roleplay a lot. And so if you give them a strong narrative then they're really effective and if you give them a weak narrative then they're not as effective. You can actually get like completely correct code as long as the code base has very consistent rules baked into it. What you're asking for is like a factory for software, but you still have to design the thing. Coding agents are taking over. Every research lab seems to be putting out their own CLI tool for coding agent. IDE's have AI baked in now. But is this the best way of doing it? On episode 60 of Tool Use, Brought to By Tool Hive, we're talking to Kirin, one of the core contributors to Slate. Slate is a research-driven approach to coding agent and it does things little differently. So today we're going to learn about what makes Slate different and how to set up your environment to make the most of coding agents. So please enjoy this conversation with Kirin. In order to get the best out of any of these tools, you really want to make sure that your dev environment is set up in a way that the model will actually benefit from it. And so at this point, it's like I think a known thing out in the open, but all of the models that are trained now need to kind of have some sort of feedback mechanism about the correctness of the things they're doing. Otherwise, they end up being and making pretty stupid decisions. And so an easy way to do this is to provide it things like, well, to give it awareness of things like the build commands or the, you know, dev server commands or practices for how it should do certain types of debugging, et cetera. And so you have the ability to do this through things like rules files. Or you can just structure your code based in a way that it like logically makes sense. But rules files will probably be your best friend there. I think like, let's say in your rules file, you describe different debugging approaches. And you describe how you can build it, how you can run it, how you can install everything, how you can set up the environment. All of those things are really, really important because you're taking like essentially all the reasoning that it would have to do around setting up the environment and just caching it somewhere for it to use. And what that means is that the model then has a like, wazzy tool, right? Like if it can run those commands, if it can set up the environment in that way with its existing tools, you've essentially given it basically another tool that it can use for validating its work. And what we found pretty consistently is that if you get something like slate in a loop where it gets feedback from its environment about like what it did and whether or not what it did was correct, then you actually can get like way better performance. And this should, this should be obvious, right? It's like, okay, yeah, you give a person a task and they go write some code, you know, like some top percentage of engineers are going to be able to write that code completely correctly just from the mental model they have in their head of everything. But most engineers are going to need something like the ability to do NPM run dev or you know NPM run compile. And that I think is super important. Same thing goes for tests. So you want to have probably a good testing set up so that you can ensure that nothing broke when you add new things. It does get weird when you ask the model to write the tests for you because you then have to verify that the tests themselves are correct. And the only way you can really do that is via inspection. So you literally as a person have to walk through the tests it wrote or you have to write them yourself in which case you should probably actually focus on making your testing infrastructure as good as it possibly can be and as complete as it possibly can be so that the model has this like hill that it can actively hill climb because it's just, it's trained to do that. So you may as well just take advantage of the like natural behavior that it wants to exhibit and just let it do that. You just need to put the guardrails in place so that the feedback that it gets from your system is designed in such a way that it makes it easy for it to recognize where its code failed. And another thing here is a lot of people have talked about how Sonnet was very reward hacky. What this manifests as is it's like deleting tests, deleting code, right? It'll, oh yeah, it just deleted my entire code based and now all the tests pass. Cool, great, right? But it's because there's no tests. And so that's the one thing that you have to kind of be careful about here. And so you can also do things like ask it to verify, you know, like whether or not there were changes to X file or Y file or you can put in pre-commit hooks to make sure that there were no changes to that file. So something you could do is like you could take your agent and you could have it like do all of its work run all of its tests to make sure everything builds and do a commit and have a pre-commit hook that you manually bypass as a user where you can use that as a mechanism to check to see like, okay, like the agent didn't mess anything, right? It's just another form of code review. It's just easier. And so then you could probably do things in the form of like two commits where you do like one commit where you do the test related stuff and then you do another commit where the agent does all the code. So that's an example. Another thing is that you, if you are running multiple of these in parallel, you kind of need, you can't really do locking that well over files. Like if two agents are writing to the same file, you can get conflicts there. And so really what you want to do is probably something like work trees or you want to use like some remote agent that has like an isolated container or a VM or something, right? And then you can get like the best of both worlds where you can get the change sets for thing A and the change sets for thing B. Where it gets interesting is how do you do things sequentially, right? Like how do you get these, how do you get agents to do large tasks sequentially that are dependent on each other? How do you get them to do them well? And this is I think actually an open problem still to some degree. I think the anthropic models are pretty good at this. But the problem, the main problem is that code contains quite a bit of, yeah, code contains quite a few implicit decisions about what you should and shouldn't do. And because the decisions are implied, you need to have the code itself in the context. And so if the model doesn't get all the code in its context when it's trying to implement something, you'll end up with a bunch of like issues like the model will re-implement something or it will use the wrong decorator or it will do, you know, it'll do use the wrong parent class or it'll extend the wrong type interface. And then it'll use that type interface in the wrong spot because it, you know, when it searched, it was like there were two type interfaces that came up and it picked the wrong one. And then it was like, okay, well now I'm going to just use this for the rest of my execution. And then you go and you look at it and you're like, well, my future task is dependent on this thing. And this thing is broken. What do I do? And in that case, you're kind of stuck, which is why you want to make sure that your code based patterns are very clear. You want probably as few types as possible. You want as little abstraction as possible. You want everything to be as modular as it can be without being overly modular because as you start to break things down even like to a certain degree, you can actually increase the coupling by trying to break things apart when there actually is like code that's supposed to sit together. So let's say you have a big function and they share like half of the variables are start being used halfway through the function and the rest of them are used at the top. And there's a very clean line where you could split it into two functions. So you have a tradeoff to make when you're writing your code, you can either say, oh, actually I'm going to output this type and I'm going to take make a second function that takes in this type, but really what you're doing is you're just increasing the complexity of your code base because you are creating two black boxes instead of and you're creating two two points where you have to track and maintain structure, I guess, like you have to remember the correct type structure and what each thing means rather than one. And so really again, it's like a couple, couple guiding principles, which are kind of standard or like if you have to repeat something more than twice, consider refactoring it, you know, code that like changes together should stay together, etc., etc., general architectural and like good code organization. and principles will apply. Something that is important to note though is file lengths should be shorter. And that's simply because when you go and have a model read file, like the length of the file will introduce context, right? And you don't necessarily care about all of that context. And so if you can afford to break up the thing, like you don't have just one giant class that is like 2000 lines long. And it's actually just like a bunch of different things. If you can afford to break that up, you should probably break that up. And you should probably move all your utils to a utils folder, import them, et cetera, et cetera. Same thing with building good components and doing good state management. If you get your state management right or your event handling right manually, and you move all of the system guidelines and complexity into one spot and you create patterns around these things, models are really good at propagating those patterns, but they're really, really bad at creating them themselves. And so if you have poor state management, initially, the model is just going to write, well, maybe it'll write better code than you. And in that case, you're good to go. But assuming that you're doing like prod work at a company, if you are doing poor state management on your front end, the model will just replicate the quality that you have. It's not going to really save you. And so you need to establish, like I was saying, there's implicit decisions in the code. If the model sees these kind of patterns, it'll just choose to replicate them. And so your whole goal is to set the patterns up in a way where it's easy for the model to work with. And then the agent builder, so like in this case, us, it's our job for us to surface those patterns to the model so that they make good decisions. All that makes a lot of sense with the architecture of it. I'm curious when we get to the usage phase. What type of constraints do we need to give agents when we're prompting them? Agent rules about how they can verify their work, architecture decisions that make it easy for them to do certain categories of work, I think, works really well. So let's say you have a certain pattern in your code base that allows you to extend the functionality of different components in your code base. That's really useful because the models will be really good at extending those patterns or repeating them for different cases. But the constraints, I think, exist more in how you choose to accept the code. How many times you review it, how you want to review it, a lot of people say that GPT-5, GPT-5 code X are good code reviewers. And that-- you can build your own workflow where you have a GitHub and you have GPT-5 do some review task. And you have Slate, Cloud Code, whatever, do the actual implementation work. And then you can just run that workflow in parallel. So ideally, what you want to think about is how do I build a pipeline here? Where I get all of the information I need to make a decision about whether this code is good or bad. And that's your goal with your whole dev setup is that you have a pipeline that is very clear about, did this work or did this not work? So that's number one. Number two is then also creating the guard rails or patterns or guidelines that make the code more likely to work. But first, you have to be able to tell if it works or not. So again, tests are really important. To be honest, when I do things, I actually don't rely too much on tests because we have to do a lot of end-to-end work. So my personal workflow is something like actually making sure that the model is aware of the right files. And then this is why I was saying patterns are super important. It's because you can actually get completely correct code as long as the code base has very consistent rules baked into it. And you can just get basically perfect code without having to have tests. I think the thing is that most people don't write code in a way that is very-- what's the best way to say this symmetric? There's kind of consistency across how things are written. And the more consistency you have, the more likely the model is going to follow that pattern. And therefore, if that pattern is correct and is useful for your current use case, the more likely the code is going to be correct and the happier you're going to be. And so it's important to figure out how do you want this code base to be built. And this actually leads me into a different, more interesting point, which is like, is software engineering dead? And do new grads have jobs? And I actually think that the answer is yes. Simply because you have to make decisions about how you want the code base to be built. And that requires a lot of skill. And it requires a lot of practice. And you're going to constantly need people who are able to understand how to actually make good decisions about code organization, code structure, implement-- implementations, specifics, abstraction, decisions, even build lifecycle or the CICD decisions, all that stuff. All of that stuff is super necessary to make what is essentially what you're asking for is a factory for software. But you still have to design the thing. All we have right now is we have the robotic-- like, arms, and we have the conveyor belts, and we have whatnot. We have all of these individual things. But you still have to make good decisions about your factory. Because what happens if your factory is like, shape like this? Why would you do that? Most factories are squared. Why would you do this? But the thing is most people with their code bases, if they don't think about it too much, they actually end up creating a code base that looks like this and is not actually designed to be a factory. And this is also true of code bases before AI, right? You write a good code base. And it means that you can extend it very easily. You can change its purpose very easily. If you need to adapt something, you can do that. If you need to rip something out and reuse it in a completely different project, also super easy to do as long as you push the complexity around in a way where it is contained in a way that you can understand it well. So just like generally good software engineering principles, I think make it easier to use AI tools. And the less you know, the harder it is. There is also something interesting here, which is people who are not super strong developers, but still use these tools also get a lot of benefit. But it seems to me like they spend a lot more money working through these issues. Coding agents are able to take real work off our plates and deliver real value. In order to maximize that value, you have to give them access to real data and systems using MCP. And that can be kind of scary, which is why I've been testing out ToolHide. ToolHide makes it simple and secure to use MCP. It includes a registry of trust MCP servers. It lets me containerize any other server with a single command. I can start an client in seconds and seek her protection and network isolation or built in. You can try ToolHide as well. It's free and it's open source. And you can learn more at toollive.dev. Now back to the conversation with Karen. Why coding agents? This goes all the way back to when chat GPT came out and stable defeated and came out. And I got really interested in why models are unable to reason about code bases. So if you were using GPT3 or GPT3.5 at the time rate, and you asked it to do pretty simple tasks coding wise, it would generally fail, even if given all of the context. And I thought that was really interesting. And so something kind of weird happened when stable diffusion came out because diffusion models are able to actually understand the kind of structural relationships between things. So if you think about an image scene, an image scene actually has a bunch of relationships between different things modeled. A classic one was the mug in the grass versus the grass in the mug. And models for a while had a hard time distinguishing between those types of relationships in language, but they could actually represent them in images. And I thought that was really weird because code is much more structured than language is. And in theory, you should be able to teach a model to-- understand those things. And so, went down a rabbit hole there for a while, got really, really interested, read a bunch of research papers, then ended up building some early agents based off of the Voyager paper, right? That was a really, really cool paper to see. And then, at that some point, we were like, well, we know how to make these agents. And we were working on infrastructure projects at the time. And we were like, why don't we just make one because Devon just launched one. So why don't we just like hope and source Devon? And so we did that and it actually worked pretty well. We launched a CLI in early 2024 sort of as an open source thing. Not really, really, like March 2024 approximately. And that was, that worked out pretty well. Got a bunch of GitHub stars, a bunch of people used it, ended up going down this giant rabbit hole of like, how do you build a product around this stuff? And then we went off into the coding like bunker and just like build a bunch of different things for a while. And recently we decided that we wanted to just launch the agent. So we launched the agent. And that is, I think that's basically catches us up to today. So do you mind telling me a bit about slate? Because there's so many different coding CLIs coming out now. Everyone made your players dropping them. What makes like different? Why should we check it out? Just a little insight there. Yeah. So slate is primarily for engineers who are working on like more production code bases. It's designed for longer, harder, more edge case tasks. I think I mentioned earlier that you can do like systems debugging. You recently posted a blog about how you can do like partial migrations using slate. And it's designed to work with you through these problems. So something kind of interesting with respect to that blog post is like, we purposefully didn't give it that much information to just see how it would perform if we made it as autonomous as possible in this scenario where I was interacting with a user. A lot of learnings distilled from a lot of experiments in making long running successful agents. So I've run experiments on things like how long can a model propagate the state of an automata or a set of automata over a finite tape? So how many states can it simulate forward in a set of automata? Another example is like, I was building some reasoning benchmarks and one of them was like for continuous learning. And it was how can I get this thing to navigate a maze without being aware of these surroundings? Like just based off of knowing its position, how do you get it to actually map out like the state of the environment that it's in? Right? And there's a bunch of really interesting problems like this that you can use to model failure modes in more complex environments like coding. And so a lot of the learnings around general reasoning in agents is baked into slate. It's, they're, yeah, they're generally baked into slate, but it's kind of funny. The end result is actually just something that is more simple. So for example, to do lists, right? To do lists are actually a really, really effective way of getting a model to think through a problem because you have this reasoning, reasoning problem where the model kind of has to predict what issues it's going to run to and run into and like what the shape of, you know, its solution or approach is going to look like ahead of time. And it actually doesn't know what is going to work, right? So assume like golden path, it explores in this case your code base. And it is able to, you know, gather enough context and then create a good plan. And most of the time what happens is it'll create its plan. You'll get like five bullet points and then it'll go do something. And this happens with almost every coding agent. And part of that is because the model providers have trained this behavior into the models. There was a period of time where getting them to track things like task state was actually really, really hard. And that has since kind of been as far as issues go that has since been kind of solved to some degree, especially like, you know, anthropics talking about their memory tool and all that. So this state tracking behavior is really, really important because if the plan does not go according to plan, right? Implementing your plan for a solution in a code base doesn't go according to plan. And you learn something. Let's say you run a shell command and you something is broken in the environment. And now you have to go off course. And instead of just going on this like happy path, you actually have to go this way, right? And solve the environment issue. How do you go down that path and come back to the original like golden path once you've solved that environment problem? The same thing goes for like whether or not, you know, certain code already exists in the code base. It's like, if the model misses that, then what happens? And so those are the cases that you have to build for. And so I've tried like, I don't know how many different types of solutions for a lot of these problems. And generally, the simplest and most generic solutions seem to be what works best, which is really annoying because you know, you'll try to build like a memory, a memory system, right? And almost all compressive memory systems that I have tried or seen generally fail. And I think people saw this a lot with like the compaction in Cloud Code. It like, it just kills the session. So a lot of people will just kill the session and start a new session. And so when you like run into these problems, you end up finding solutions that are probably more scalable. And so the way that we do context management I think is probably better. The way like our requests generally don't go over like 50 to 70,000 tokens, I think, on average, at least that's what I've seen historically. And that seems to be like a sweet spot for context. It's, once you get past that point, a lot of models start to grade. So I think anecdotally what I've seen is something around like 120, K tokens seems to degrade a lot of the performance. And the same thing goes for like, you know, how do you do context engineering? How do you pull in the rights snippets at the right time? What do those snippets look like? How much is enough context for the model to like latch onto and move onto the next thing? All of these things are like really subtle small things, but they add up to make the performance on any given task like way better or way worse. And so when you're like, when the kind of average AI like friendly person goes to implement to coding agent, what they usually find is that it like works okay, but that's just because of the model. And there's actually like, if this is like the baseline performance of the model, even with just a shell tool, you can actually get like a lot more performance out of the model simply by providing it the right framing and environment for it to work in. So I guess that's like a summary of the benefits of of using slate or really just like the learnings that we've put into it. I think why you should use it? It's specifically designed to handle three types of issues in agents. So one is completeness, one is correctness, and one is just like general intelligence. And the tools that we've provided, the way that we built them, the way that we provide context, all of that stuff is designed in a way that it like makes it easy for the model to be as smart as it can be. A few things to unpack there. I totally see what you mean with people not liking the compact feature. Every guess I've had on who's mentioned it talks about how they just ignore it. They do their own compaction because that type of control tends to benefit from view. One part of my workflow is I'll make my own to do list. So where I just kind of set up a list if I need the AI to update it, I explicitly tell it to do so. And sometimes I rely on the built in to do list, but I find just having that level of control. Where do you see slate kind of alleviating the need for human intervention versus the other models? Because I'm telling people if you're using Cloud Code, slash new slash clear, very frequently referred to a markdown file. So there's a lot of human intervention, but with slate, if I'm understanding correctly, it's better for long running tasks. like the big work. Yeah. Could you elaborate on that a little bit? Slate generally won't get bogged down by context in the same way that CloudCoder codex will. Now you have a secondary trade-off here which is the way that we built it and the way that we have to like probably improve it is in its memory and it's like kind of ability to you know retain that high-level bull however what it's really good at is just it'll keep working. It'll continuously like work through your task. It doesn't really stop until it's like task list is done and this was super intentional. I think you if you keep things simple you kind of have this trade-off of like okay you can either make it so that it is very very agentic and it'll just do a lot of stuff or you can make it so that it retains a lot of context and then starts to degrade and we wanted to keep the performance very stable so Slate is a lot more like a function than I would say other tools and I think that's really important and that's probably what's going to allow us to scale up the task length reliably and scale up the like kind of build other other abilities of Slate because it is much more like a function and like than I think other agents are more like a function of what regards like it's more guaranteed to execute in a specific way. Pretty pretty exclusively it will like if as long as it's given like a complicated enough task you will get your to-do list it will work through the whole thing it won't it's not going to stop it's just going to keep going and eventually it'll declare victory which may or may not be correct but by that point it like has done the exploration it has gathered the context it has maintained it the way that we do like tool and environment interaction allows it to be more efficient and so it can actually get there faster and also likely more completely because it's like it's a little bit more prone to exploration I would say but then at the same time that gets you the added benefit of it having more context about like what you're trying to do when you want to do it and so for anybody who's going to go use it they I think and they'll end up feeling like it feels different and are it's going to be hard to articulate why it feels different but the actual reason is that it is just more like it's almost it feels almost more like autonomous or programmatic I think that's what I mean by function it's like it's more autonomous it's not this like back and forth thing that's like really kind of flimsy it's like very just it'll just go do the stuff that you want it to do and then obviously the intelligence is limited by like the underlying model but we're actually like also doing stuff to alleviate that right now would you say with the current state of things the bottleneck is the LLM's capability or the the framework the tooling around it like do you think we can squeeze more of the existing LLM's or do we need another step change to get to that next level of capability something that I like to say a lot is all the cards are already on the table and we just don't know how to use them and I'm like pretty confident in that statement as far as model intelligence goes because if you like look at the way that people design tools for language models they don't seem to be that effective I think you can get away with like much with with better framing of the environment and and better tools that provide the model more capabilities without actually increasing the combinations so what I mean by this is like there are people you know build like an MCP and it'll have however many tools right or like you'll have tons and tons of things individually that the model can do and that's probably not a great pattern because that's a lot of things for the model to keep track of what you actually want is you want more likely more powerful tools and so I think a great example of this is like providing back or like a shell tool shell tool super powerful you can do most of the things that you need with it right and because of that you know the models are trained to be aware of all the tools that are going to be available in the command line but because of that you end up with like really really powerful agents based off of just the shell tool and when somebody goes and designs a new tool for an agent they'll overload the number of like combinations that the agent has access to in terms of like the parameters and things like that and that usually degrades performance so instead like the goal is to design the context and actually specifically like design the message history in a way that the model can create a coherent understanding of the world that it's in and I think that's really important and I think that most of the time people don't do that and so another thing with Slate is that you'll see that it is actually probably more coherent and aware of what's going on in the environment than other systems and part of this is like downstream these principles that I'm kind of going over right now so an example is like Slate can you know use can use VIM it can use T-mux and go over SSH like spin up an SSH session into like a remote box and debug something on a remote box it can debug more complex systems and like multiple things in parallel and these are kind of more complex issues that most agents can't do and the reason is that I think they don't really do a good job of tracking like the context that the model is working in but I also don't think that it's about you know stuffing or stuffing the context or shuffling things around because let's say you dump all the docs for something into the context that's like that's actually not help at all and what you really want is you want the model to be able to make its own decisions about like essentially snipping pieces of the context so let's say you have this giant document you want the model to be able to extract just this small section move it into its own context and dump the rest of the document and so you should build tools that allow you to do that instead of building a tool that just gives it access to the whole document and sticks it in context yeah because I always thought it would steer it the wrong direction just giving too much and just dumping everything in right and I've been a big proponent of telling people use LLM's as little as possible like if you create a pipeline have deterministic code as much as possible and the LLM's just kind of like the fuzzy match or the bit in the middle what one thing that stood out to me just the ability to do all these other things whether it's like read the docs capture little parts or go into remote servers it feels like that's just creating a lot of data a lot of information that goes into the context is your approach to prune the context or do you create artifacts that it can reference to pull in like for example like creating the to do list a markdown file or maybe it creates a copy of the docs as a txt and then it can just read that and extract it or do you leave things in the context yeah right to right now actually we just leave things in the context I've done a lot of experiments with things like pruning and I am not can I'm convinced that there is a good way to do it I'm not convinced that there is an amazing way to do it and so if if you yeah if you look at like the actual performance of things models tend to degrade in performance when these surrounding context about why they got that information disappears so if they don't actually see a coherent narrative about like oh hey this piece of context ended up in my history because I did this thing the the kind of like I was saying earlier coherence of the system the agent the model decreases because it doesn't actually know where that thing came from it doesn't know why it's there and you really you really have to explain like why the the model is seeing the things that it's seeing or it has to understand that into intrinsically and so if you do these things like compression and pruning and whatnot you actually create a incoherent narrative if you do it in like a naive or not even naive even if you try to do it well you still create some amount of an incoherent narrative because the model is missing information that should be surrounding the information that it cares about so that is actually that is a big problem and this is actually surprising that I haven't heard this address too often because it makes perfect sense when you have an entity processing language. If the language is incomplete, it could scramble it and just go incorrectly. Can this be resolved by things like having XML tags as like section headers or is that just kind of a bandaid solution versus giving it proper reference to where the context came from? I think that's the bandaid to be honest. Obviously you can throw your own tokens in, like your own delimitors in, right? But without the correct history reconstruction, the model will just end up doing weird stuff or at least they used to. What I'm describing is some behavior that we saw a lot on Sonnet 35 and 37. Then we fixed our systems and everything started working. We haven't seen that problem since May. Prior to that, it was a big issue because the model would see, I was testing out some pruning stuff. The model would see the piece of information that it was supposed to see. It would be like, "Oh, this could be important." I think the correct way to think about it is if you do any of this stuff, you do summarization, you do pruning, you do whatever. It needs to just be treated as like background information and not actually information that is part of the current trajectory. Because without the cause and effect relationships that are being shown throughout the tool call history, you lose the context as to why that thing exists. The model will see it, but it doesn't really have a strong narrative around it. Something that's pretty important is that models like to roleplay a lot. If you give them a strong narrative, then they're really effective. If you give them a weak narrative, then they're not as effective. This is just an example of instantiation of that problem of like, "Okay, here's a bunch of information that seems completely irrelevant or tangentially relevant in comparison to all of these recent tool calls I just made. Well, I'm going to focus on these tool calls. I'm not going to focus on the thing that was just stuffed into my context." Even if you do it, there's a couple of approaches that you can take. You really have to think about how the context degrades essentially. If you're starting the context over here and you're ending the context here, the context and the attention to that context kind of goes like this. It tends really strong. The attention mechanism attends really strongly to the end and really strongly to the beginning. Anything else ends up being noise. You can, sure, you can introduce noise in the form of valuable context and it'll act as almost just a bias towards the behavior, but it won't actually really improve it. More so what you want to do probably is pull things, guide the model with it's like to do list to behave better right? Make it actually behave more correctly rather than trying to patch up these issues. A lot of the times when people have some type of chatter interface, they'll just keep appending to the conversation. Is there any difference in your experience from doing something like that where you have a clear system message, user assistant, user assistant back of earth versus taking all of that and creating a few shot example where you kind of compress everything into a single message and maybe remove the user aspect, the assistant aspect of it and just kind of have one message or conversation versus many messages of back and forth dialogue. Yeah, I think the models are trained on the chat format so use it to your advantage is the pretty straightforward answer there. It's definitely use that. You can use that to your advantage in terms of making it perform better. So do that. The other question I had is around that the attention curve where pays more attention to being in the end. Do you know why that is because my understanding transformers was it should look at everything equally. How come it's evolved in such a way or maybe a structure in a way just I'm misunderstanding why it gives more attention to the start in the end because I've heard that from multiple people. I just I don't know why. I have not trained a model out of frontier lab. So I can't say for certain and I actually I'm not sure if they can say for certain either to be honest. I think it's just generally related to the data. The like if you think about having a conversation with a person or about right the initial context is really important for the rest of the conversation usually unless it shifts into something completely different and then the most recent question or turn is also the most important it's like okay yeah we've had this conversation for 20 minutes whatever we started with is probably really important and whatever we're talking about right now is also probably really important. And if you look at your own chats like something that comes to mind for me right now is if I look at like chat history on something I usually only care about the last two responses personally. I don't really care about the response that happened five responses ago. And so the the best response from a model is going to be the one that actually focuses on the things that I care about and not the things that I don't care about. And so that's an example of like a recency bias that can be trained into the data but also the bias towards like that initial initial state. I also think that because everything is downstream of those initial tokens that there could be some form of impact but the models are probably large enough that the like kind of path dependence of the rest of the conversation washes out with enough data diversity. So that's probably not true but it could be. And I think that would be an interesting thing if somebody could prove that one way or another. So you can go check out slate at random labs dot AI. And you'll be able to during the payment flow you'll be able to get a month free of credits to just try it out. And if it works, let me let us know let me know let us know at team at random labs dot AI. And then you'll be all set. Hopefully it works out. Thank you for listening to a conversation with Kiran. I thought it was great learning about slate and what really makes a good coding agent. I also think it's important that we keep in mind that you shouldn't just download these tools using blindly but if we work on optimizing our environments to get the absolute most of them, we're going to be more productive, more efficient and be able to accomplish more. I want to give quick shadow to tool hive supporting the show so I can have conversations like this and I'll see you next week.