The Self-Improving Company | Kavak's AI Playbook

I'm investing more today in tokens than in knowledge workers. We could build superhuman agents. This means that by every dimension that matters, our agents would outperform the best human we had in our hire. The most ambitious companies listening to this will decide to follow suit, which is you decided to build an agent per customer. Yes. Every day between 100 and 200,000 agents get instantiated. Specifically for this customer with its own virtual machine. There's a lot of people worried about how the organizations of the future are going to look like and the role that humans are going to play. If you haven't faced fear before you haven't felt it, then you haven't tried AI. We launched a program inside Cadac. That's called the Jedi Academy. From the CEO to like AI engineers to mechanics, which are everyone, and after six weeks, they launch state-of-the-art agents to production. What advice do you have to future founders or first-time founders that might be listening? What we're trying to do now is most companies are asking how to add AI to the organization. Kavak asked a much more radical question. What would we build if we were starting the company from scratch with AI? Angela Strange and Gabriel Vasquez sit down with Kavak's Chief Product and AI Officer, Alejandro Ayala, to unpack what happened when the company began to take bet on rebuilding itself around agents. Today, hundreds of thousands of agents can be instantiated each day, handling everything from selling and financing cars to maintaining long-term customer relationships. They discuss why Kavak tore down an agent architecture that was already working to start again. How EVALs became the foundation for moving faster. And what happens when agents don't just work for humans, but humans sometimes work for agents. The ACC podcast today, we have Alemassa, the head of AI Kavak. We're going to discuss today the transformation that Ale led within Kavak to turn into an AI-nated company. Thank you, Alem, for being with us today. Thanks for having me. Before starting at Kavak, you were running a company called OP and Analytics. That's right. And you were very much into AI before ChatGPT. You want to tell us a little bit about that journey? Yes, yes, of course. Well, we called it machine learning back then. It was a different family of algorithms. And we found that the company with this very ambitious vision there, that new machine learning models would be so powerful that they could solve any complex problem. This was pre-transformers. This was 2013. So we started building the company that way. And I think we were like 10 years ahead of time. But we built a great company. We served 1,500 companies around risk algorithms, logistics, forecasting, marketing. But really the power of what transformers and then the ChatGPT moment when he arrived made things very clearly that we could now build a whole new company and way of building companies. And we joined Kavak to encourage us to build that. Amazing. So we're going to spend the bulk of this podcast talking about exactly how you've identified Kavak. But maybe just to start, what does Kavak do and what is your role there? Kavak started out as a used car marketplace. So we buy cars, we refurbish them, and then we sell them and finance them. But to do that, we also had to build a Fintech and logistics company and the car facts. And basically all the infrastructure for this to work didn't exist in Latin. So we had to build everything vertically. So we could serve our customers the right way. I'm going to start with the framing of what the architecture looks like. So a consumer comes in and says, I want to sell my car. Like how many agents do they touch? What's the harness look like? Groundless in how you design? Right. So we bet the company in transforming to a company run by agents. The questions we ask ourselves is how would we build Kavak in 2035 with fabled 10 or GPT-level intelligence? And actually that company looks very different than what we had built or what we had back then. So when a customer comes in right now, agent will get spawned specifically for this customer with its own virtual machine. It will remember years of interaction of this customer's with Kavak, what they visited in the webpage or call they had two years ago. Remember everything in its memory? Come up with a strategy and set a long-term goal to maximize the lifetime value of this customer. And do whatever it takes to make the customer happy and convert them into like all their different products across time. And they set completely new and groundbreaking architecture at scale, I think, because people are still building multi-agent systems with experts. And we realized to bet that long-running agents with hard goals, not just workflows, could maximize our customer's satisfaction. And obviously their lifetime value. OK, so we're going to jump to the new ones. That may be versus many companies that say, hey, we want to be a gen tick. And they try some workflows. Yes. You guys took the just rip. We had to make us work you to downsize dramatically. It didn't work for a year. Right. So you want to talk through. Obviously you had to tune a lot of things to make that work. Like describe the harness at that time and what models you were using and sort of specifically. Yeah. So there were like three main decisions that we had to make. The first-- and this is where I think many companies are stuck right now-- is the first instinct is, OK, let's adopt AI. And you basically leave your structure as it is and just give chat jippity or clawed to your team. And then there's no efficiencies. Your customers have the same problems. And nothing happens, right? And so you need to redesign your whole company around the agents and around the future capabilities. And this means really rebuilding most of your APIs, rebuilding your system so the agents can use them to perform. Then you need to start generating the data and the feedback to find your desegents. The only way to really make them work is if you teach them-- how do you teach them? You put them out in the open. You put them in front of customers. You get that data. You get those evils. And then you train your agents. And this is the second bet that we made that we could build superhuman agents. This means that by every dimension that matters-- conversion, life-to-value, customer experience, our agents would outperform the best human we had in our hire. And we put them in front of the hardest problems. And finally, you start to change how you measure the success of the company. Kavak was a transactional company. We used to measure how many cars were bought, how many cars we sold, how many brake pads we needed to buy. And we moved to a relational company. Where now I have 10 million customers in my database. And I have agents assigned to most of them with the task of maximizing their life-to-value. Now, we're selling cars and personal loans and very high ticket items. So just activating 1% of this customer base, it's hundreds of millions of dollars if we do it the right way. So it's a bet that made sense for us because of our industry, because of the ticket, and because at the end of the day, customers need to build trust with a company, because they're buying a used car. And the way to build trust is to know them and to plan and nurture a long-term relationship. Ale, I just wanted to double click on something. EWO is over agent demos. Yeah, you probably get pitch a lot of agents. And it's never been easier to build things like before. But one of the questions is, are you guys going to evaluate in this? Because not everybody tests them across 90% of their customer interactions to see they're really working. And you guys, I believe, is about 98% of their interactions, or something. Yes. Like that are now handled by agents. Yes, totally. So to give you a sense of the scale, like 96% of all interactions are handled by agents. So no humans there. 95% of all transactions are completely handled by agents. Obviously, you meet a human when you pick up your car, like there's someone physically there to give you their keys. But the rest of the experience of the journey is handled by an agent. Every day between 100 and 200,000 agents get instantiated in a date. They wake up. They work sometimes for three minutes, sometimes for eight hours, sometimes for three days. And they set on alarm clock for the next task and they go back to sleep. So the scale of this is just amazing. And it's working. Now, how do you get this to work at scale? And the answer you mentioned is EWO. I like to move extremely fast. But in order to move fast, you need to have breaks. Imagine a car. You'll hit on the gas just if you have the right breaks. And AI is super powerful. And I've seen many companies get this wrong because they try to go slow because they don't have the right breaks. So I thought about it the other way around. Like how fast can we go? Well, it depends on the quality of our emails. So a good rule of thumb here is we spend about the same amount of time, engineer time, tokens, and money on building the EWO, the building the agents. And this is how you get better and better and better. Not letting EWO says an afterthought. So why do we measure? First and foremost, the results for the business. Like if my customer's happy, they'll buy a car, they'll get their loan approved. They'll sell a car to us. And that's the first check. it convert. And that's where most things break. I see companies measuring number of calls or minutes during the call or some superficial KPIs that give you some information, but that doesn't really work. The important thing is this customer convert is bringing value to the customer and is the customer happy to reengage with us after a while. And once you get those evals connected, then it's just optimizing the right agent to carcotte texture and giving the agent skills to scale this and cater to millions of customers. It's really amazing. And you know, related to this is like, okay, so you create the right evals, noise working. You know, some people, some companies still feel a little bit risk of errors and putting them in front of the customers and being able to perform the highest leverage tasks, which in your case would be selling. Do your agents really sell to customers? Yes. So we never built customer support or customer service agents. We built like sales agents. It's extremely hard to sell a car in Latin America. So imagine someone wanting to buy a car, they can choose like amongst like 20,000 SKUs, then they need to pick like financing and go through the financing process, insurance and coverage. And then they're probably trading in their car. So we need to quote that car. So it's a process that if someone does it or the way they've acted it back in in 2020, 2021, you need to be extremely good at 15 different things and have 15 different experts in 15 different teams. And usually the person would go and speak with the expert in financing, the expert in car advisory, the expert in buying the expert and insurance and they'll build a package and buy a car. That's extremely hard to do. But like the first thing with data was, okay, can we get an agent to be better than the expert in each of this things and then put it together and have like a mega expert that's an expert in insurance, financing, etc. And that's who we put in front of the customer. So the experience for the customer is amazing. We tripled NPS and customer satisfaction score by putting the agent in front of the customer and it at first it converted like 50% more than our human team. And now it's converting over that like 2.1 X more. So it's a completely different company. Other agents are better sellers. Totally better. And you get this right because they're experts and they're infinitely patient and they know all your history and they can plan for the long term and they never get tired. So and if they make a mistake, they learn it and the next day, not just them, but the other 200,000 agents will have learned from that mistake. So that's a feedback loop that we engaged and that's showing in the growth and results and satisfaction of our customers. One of the two of the very cool things I think about Quebec is I think the world has gotten comfortable with AI can do customer service and still very hard to do well. But as Gabe said, there's still a view that well, customers aren't going to want to buy expensive things from AI and you are proving them wrong. The next layer on that is, well, you're not actually going to be able to do regulated financial services and end with AI. But if you walk through or you're doing you are underwriting a thin or no file customer pricing them correctly, doing servicing. So maybe talk through how did you how did you write the evals to get comfortable with that and then versus I don't know, go into a bank branch or even a fintech sort of how how is that experience? Yes. So so the first financial product that we launched was a car loan and usually in Mexico and in some emerging markets, it'll get like two months or or or more to get a car loan approved. We usually approve it in under three minutes, which is like pretty cool because we have all these data around the customer and the car. And if the customer can't pay for the car anymore, they'll just return it to us and we can give them a cheaper car and then they'll pay a smaller amount each month and they like get out of water, which is amazing about the vertical integration of the business. But then like when we started launching all their financial products, we realized that this is a very important decision for the customer, right? Like like they usually take three to four months to make their make up their mind and buying a car and and and and getting a loan or getting a personal loan like a large personal loan that that we also do. So if you get to know your customer throughout this process and make the process easy for them, then just your conversion and retention metrics start going through the roof. It's not just the transaction is understanding each customer personally and get them to to convert when they're ready with a very big personalization of the interest rate, the risk, the maximum amount of the loan, you know, way that makes sense for the portfolio as a whole, obviously. But that's optimized to the risk level and and probably the other offers that the customer is getting. And then maybe give us just to be, you know, evals are always a very hot topic. You kind of led with that. What is a like what's an example of maybe a hard to design area for evals or one where you had to spend extra amount of time. Let's just give in the fact that like there's real money PII at risk. Yeah. So we decided to to to to really sign the company around the I. Yes, the question, okay, is is AI going to be able to do this job like even the CEO job or jobs worth the relationship base? And the answer honestly is probably yes. Like in 2035 with a rate of improvement we will to do so we said, okay, let's try it now. Let's try and build an AICO. So we carved out a city in Mexico. It's it's Guarana Baca and we put like an agent in one of our harnesses as a CEO and it starts learning and it starts making decisions and evaluating on those decisions. And it's only been running for for six weeks now. The goal of the first month was to double the profits of Guarana Baca. It didn't reach it, but it was 1.5 X like 50% more profits just like managing the city, which is it's crazy, right? It's it's amazing. And it's it's a CEO like people were like that was the last job I was supposed to take and no, it isn't really and how did this happen and and it's like a very smart person like like fields metal levels more like going into every single number, every single customer making the perfect forecast and going to micromanage every single things that needs to be executed every day to reach a plan. So he'll literally send messages to all the physical workers in Guarana Baca with their plans for the day and ask them to send voice notes back to to know their progress. So customer satisfaction grew. We got a better inventory. We rotated better better financing penetration like every KPI started to improve. And so it's super cool. It's super exciting. Now what are the jobs where we think we're still like training and hiring humans. Those are related to the physical world. So when we talk about mechanics, Cadac has around I think in Mexico around 800 mechanics. There's lots of thexterity and and sense as that's super hard to substitute. So there we also build this agents with exacting harness that scaling and the the mechanics have the sidekick. I was telling you guys earlier, it's like the movie Ratatouille, like the mouse that's actually a chef collaborating with a with a human. It's kind of like that. So it's a sidekick. We call it a mic and it tells them how to inspect a car and give them tips and and shows them the way to to do it. And the quality of inspections again, when through the roof, we're inspecting faster. We're preparing faster. It's cheaper. But most importantly, we're delivering higher quality cars. Warranty's came down around like 20, 26% since we launched and customer satisfaction again went up. So it's about this. Like how would you design your organization from scratch with with abundant super intelligence that's cheap and just go build that. Now this is a good segue to a key topic right now. It's like a mallee where you know, there's a lot of people worried about how the organizations of the future are going to look like. And the role that humans are going to play. Yes. And I think you touched a little bit on that. So we love to hear how you guys are thinking about that. And the organizations. Totally. So we took that question very seriously. three years ago. And the truth is that everyone's job will change. So, and what we were doing a couple of years ago will probably be be performed better by an AI agent, right? So what does this mean? We need to train everyone. So, so we launched a program inside Kavag that's called the Jedi Academy where anyone from Kavag, like from the CEO to yeah, and it sounds like from the CEO to like AI engineers to mechanics, like going to the academy. It's super hard like I I like that you that myself you designed the program. I designed the program. But constantly because you need to be upgrading the program because everything's changing so fast. And there's just like you can't send these people like outside to Stanford to learn this because like it's new stuff, right? So, which train everyone and after six weeks they launch state-of-the-art agents agents to production. And it's mechanics and and finance guys and engineers like everyone can do it. And what this generated is maybe this person won't become an AI engineer. Some of them have, but they they know how to collaborate with this new technology, right? So that we looked about it was guys, there's no way back like this is a way Kavag is going. This is a way the company will look like. These are the changes for the engineering team, the finance team, the product team. Like this is what's going to change. You have the choice to like train and get the skills to perform in this new reality in this new world. Or maybe leave Kavag if this is not for you, but this is a way we're going. And it would great like like we strengthen the culture. It was super excited. People really know how to build this agentic systems. And then if you look at Kavag now, any process, it's really a collaboration of agents and humans and sometimes like agents are the bosses or of humans and sometimes humans are designing the agents. But we managed to really build this and change this. And it's through this idea that we need to be learning every day and things will continue to change. And the only way to continue being relevant is to upgrade your skills every month or every couple of months. But if you do have or did you have thousands of people now agents do most things. So what is the org structure of Kavag? Does the middle management concept even exist anymore? What does your look like? Right. So that way it looks like now is very flat teams, very senior teams, super empowered. If you look at a team, you'll have engineering, AI, operations, everything. And they're either building the agents, working for the agents or being in the physical world in front of the customer. Like most of organization looks like that. So it's really built around the idea of how organizations will look like in the future and around AI and really harnessing this new technology. Obviously, these required lots of retraining because in 2023 or 2022, no one was building agents. No one was helping agents or taking orders from agents. And the way you cater to the physical world or the customers was in a different way than if an agent's telling you what to do or helping you make your job better. So it's a completely different structure than we had just two years ago. Yeah. Explain. We talked about this before what working for the agents look like. I think the way you described it was the eutogenetic system. And then sometimes when it fails, it's like, oh, that's kicked out to kind of a human queue. Right. But then that's lost. And so how have you brought that together? So like the least human in the loops and most of these agentic systems in production right now, like large scale agentic systems, usually if an agent hits a wall or can't perform anymore, it'll send the these cases or this customer to a tier two support and forget about it. That doesn't really work because you don't close the loops. So you don't generate the data to train the agent to do this better. What works right now is we have an agent that's obsessed with each of the customers like millions of this. They have access to every single API, every single skill, like and we have agents building those humans building those skills for them. And then if an agent hits a wall or cancel something, it'll call this API saying, I need help. And on the other side, it's not an agent or software. It's a human helping the math. But if you map this out in an org chart, it's really human teams that have an agent. I'm getting better results. It's super clear. Like, it makes sense. That's actually a perfect segment. I know you get lots of of leaders at larger institutions inbounding to you. So maybe this will save you many phone calls. But I think rationally many leaders of companies intuitively understand this. It is very hard still to deploy AI through the organization. Like the models are good enough. You know that. It's an org problem. It's a psychology problem. Like what advice do you have or what have you seen? I think it's two things. The first is it has to be top down because of this. Like, yeah, if you just get adoption, it won't go anywhere because it's hard to generate this taste or strategy for people to bottom up decide what to build and what not and come up with something that works for the company. So the transformation has to be top down and leaders need to adopt and leaders have to have a very clear plan on what to build. I've seen so many companies just like, oh, like we're doing a hackathon. People are coming up with use cases. We're sponsoring some of these use cases. That doesn't work. It's like be very clear on what the company will look like in three or five years and then start building that and be like very vertical in guiding your troops towards that. Like an army doesn't really work if everyone comes up with ideas on the strategy and tactics and goes to the battlefield and like those whatever they want. They need a very clear strategy. And that's what we need now. It's a transformation stage. The second one is you need to measure what what really matters and it's evils, but it's also the right evils. So I see a lot of companies spending now huge amounts and they say, okay, I got adoption. I'm just spending like hundreds of millions of dollars in tokens now. What about that? Like there's quality in the tokens. So have a framework here that's also useful. Like tier three tokens that most valuable are this agents where you can get the ROI of each specific token. And I can do that now. That's great news for me because I'm growing and because I know the ROI of each token because it goes to agents that are performing the job of the organization, right? These are the best tokens. Two tokens are things that you can measure indirectly. Do I see deaths in the code base and I can evaluate the value of these tokens at least indirectly and then push those two productions. Tier one, when most companies are, is gorgeous using plug code or chat to pt or co-work or whatever. What happens with those? I have no idea. So it's not just about adoption. It's really about having a very clear vision and then measuring that each token you spend is bringing you those benefits and just iterate, iterate, iterate from from there. And then one, and we touch on this a little bit, but I think it's worth a dive as maybe the most ambitious companies listening to this will decide to follow suit, which is you decided to build an agent per customer versus per task and then discovered along the way that each one of those agents needs its own sort of micro virtual machine. So maybe kind of walk us through those decisions that are architecture. And I think we're seeing these results now, but it was a really risky bet because people usually go from workflows, like five koonan vice-everland, don't build agent to workflows to graphs or functions or objectives. And we built that. These are multi-agent systems that can perform a whole function for a complex goals. Like the ones I told you that to sell a car, you need to do financing purchasing, like recommendations, etc. And we had thousands, like tens of thousands of these agents working at scale, running the business back in December. But then opus 4.5 came out. And I realized like this isn't the right paradigm anymore. Like the intelligence now doesn't need like the graph and the multi-agent lattice work and harness because it will constrain this level of intelligence. So we decided to like destroy everything we had been building for two years. That was working. That brought us to profitability. That brought us amazing growth and started over with a harness that we thought would be robust and scalable and in leverage recursive self-approved or new models, more intelligent models coming out every month. So the way this looks like it's sub ritual machine with an agent, with access to memory in e-vals and the CLI where they can access every tool in every API in my company and the long-term goal. And I instantiate hundreds of thousands of these each day with long-term goals, like maximizing the life and value. This self-improving organization. Exactly. The self-improving organization. And I think people are super obsessed with with our aside now. And this will improve the models. But if you look at it this way, economic value in humanity for the past 4,000 years has been delivered by organization, not by individuals. So what you want to self-improve and to engage in that loop is the organization that can deliver more economic value. Right? So that's the loop that I think companies will start to focus on because if you get that loop working and it's an organization that is really self-improving and harnessing the newer models and the better intelligence that we're getting every couple of days now, then you hit the exponential, not just intelligence, but in the value that you can generate as a company. So that's really exciting. That's what we're working on. You mentioned that because of all the challenges in adopting the AI, the biggest opportunity on net new companies being formed, working in this new way and then disrupting markets. You want to talk a little bit about that? Yes. There's this concept in economics about creative destruction from Joseph Schumpeter. And what it says is that the way innovation hits the economy is sent by companies adopting the new technology, but by companies remaining the way they were and in convinced with a new technology, destroying the old companies. So this destroys value in the short term in the economy, but in the long term it's better for everyone because this new, more efficient, more effective companies will provide better products and services for the economies. And this has happened in the past industrial revolutions and this has always happened. It's a great opportunity for entrepreneurs and people today because it's hard to adopt AI deeply. It's really hard for a CEO today, especially of a large company or public company, to go and say, "Hey, I'm betting everything on AI. The company has to look this way. I'll destroy and rebuild everything I've been building for the past 40 years to become an AI native company." How many CEOs will do that in a company at scale? So while they adopt new companies can be formed that are built around the strengths of AI and take over and bring new products and services to the masses. And this has happened before. This happened with electricity. This is a story I always tell my team. The technologies for Ford's production line were developed in 1879 and 1881. Edison started commercializing electricity in New York and then London and he invented a dynamo that was extremely efficient. So you could have built Ford's factory for 40 years before Ford. The technology was there. Everything was there. But the way people adopted electricity and forced dynamo was, "Okay, I'm going to leave my factory like four floors, shafts and belts and just change my coal engine for an electric engine." And this will bring you benefits, yes, but like 6% efficiency. What needed to be done was to destroy that factory, build it in a flat surface, not in the center of New York, but in Connecticut or New Jersey. And really sign your whole factory around small dynamos and electricity. And then you get like the 3X improvement productivity that powered the US during the 20th century. And the same happened again with the computer and the same is happening again today. People want to adopt it, but they're not willing to really sign the whole company and they just adopted superficially. In the end, that'll give you a 6% or a 10% improvement, not a 10X improvement. And it's like the innovators dilemma at an industrial scale again. I think you've just made an amazing case for any future founders out there that it's time to build. It's time to go. And maybe a great place to end is you've built and scaled your own company. You've now turned Kavak fully agentic. Like what advice do you have to future founders or first time founders that might be listening? So this is a most exciting time in human history. I believe that. Like we're living in the most exciting time in human history. And it's the most exciting time to be a founder because it's the first time that anyone has access to the most powerful tools and intelligence in the world. Like for almost for free or for $20 a month. So literally, the democratization of the tools for people to build has never been this way in human history. And there's so much problems to be solved and a new reality to be built around this new paradigm. So say like just go for it, but go for it deep. Imagine what the future around AI will look like. It's just a it's not even an exponential. Just just map a trend. It's linear. If things keep getting like AI keeps getting better at a linear scale and just build for that. And you come up with wonderful ideas that will like bring a lot of value to the world. Amazing. I like thank you for joining us. Thanks for listening to this episode of the A60Z podcast. If you like this episode, be sure to like, comment, subscribe, leave us a rating or review and share it with your friends and family. For more episodes, go to YouTube, Apple Podcasts and Spotify. Follow us on next at A16Z and subscribe to our substack at a16z.substack.com. Thanks again for listening and I'll see you in the next episode. As a reminder, the content here is for informational purposes only. Should not be taken as legal business, tax or investment advice or be used to evaluate any investment or security and is not directed at any investors or potential investors in any A16Z fund. Please note that A16Z and its affiliates may also maintain investments in the companies discussed in this podcast. For more details, including a link to our investments, please see A16Z.com/disclosures.