Claude Engineering Lead: "Stop Using Claude Like It’s 2024"

This episode is brought to you by Indeed. You're ready to level up your business, but the pressure to find the right person to make that happen is intense. Indeed sponsor jobs can help carry the load. It boosts your job post to help it reach quality talent fast. Cut through the chaos with Indeed sponsor jobs. Get a $75 sponsor job credit at Indeed.com/podcast, terms and conditions apply. Hirey now, then this is a job for Indeed sponsor jobs. When you shop, Kroger delivery, you can expect the savings you love and fresh groceries delivered right to your door. Our refrigerated trucks keep your items fresh while our associates are committed to getting every detail right. Carefully selecting and packing your order with care because bringing you fresh quality groceries is what we do best. And right now, enjoy $20 off your first online order of $75 or more. Restrictions apply, see site for details. Kroger, fresh for everyone. If all we're doing is like summarise this email for me, you're not really benefiting from the new intelligence. This is Felix Reesberg, his team built clawed co-work in about a week and a half. Now he leads engineering for clawed at Anthropic and he doesn't write code anymore. I'm just a conveyor belt for clawed. Clawed needs to have access to all the crash data and then it's like figure out itself, what it needs to fix, what it doesn't need to fix. Is there still anything you would not trust an agent with? I really want the current session to judge itself that I think is still really iffy. - How do I protect myself from AI just confidently presenting the stakes to me? - It happens to a lot of us. I have like a very practical easy answer. - Felix, thank you so much. I am very excited to have you as a user of clawed like an avid user. We're using clawed for all the operations on this channel. Every team member has their own project for the things they're responsible for. And as someone who runs it, it's a pretty big team for a creator. I have 35 people and I want them to be as efficient as possible. I want to produce as much as possible. But also something that I've noticed this year. We were able to launch I think three new social media channels. So we launched a newsletter. We launched a LinkedIn a year and a half ago because it's text-based. So AI has enabled me in many, many ways. And the goal of this conversation is to make me even more advanced than all of the viewers and all the listeners. - I mean, from what I've seen you do, it seems like you're pretty advanced as it is. - But there is no borderline, right? To being as a business partner. - That's true, that's true. - Yeah, I think I'm discovering that about myself every single day. And I come sort of one of the most fun things of working on clawed by far is this notion of, I built this product and in comparison to my other products, with my other products, it was pretty clear to me what you're going to use this for, right? Like, the example I always go with is I worked on Slack and I kind of have a feeling of like, "What are you going to do with this?" The thing that's magical about working on clawed is that I see people really fly and do amazing things that I didn't consider possible. And it's like for a product builder that's such a treat to see people fly with this thing that you made. - Is there something you've seen that's super ambitious and you're like, "Oh my God, I want to copy this." - I'm not sure about copying, but two days ago, I think he's actually a product manager by trade, his name is Pavel. And he pointed clawed at a large amount of NASA data and asked clawed like, "Find him a new planet," and it did. So apparently, it is not confirmed yet and NASA is working on actually confirming it. I'm on a little shaky ground here because I'm not an astrophysicist of any kind. But apparently clawed may have found a new planet we didn't know of before. - This is fascinating. Okay. - And it's the kind of thing that I would have never considered. But if you hear the story of, I think for us who work maybe on more knowledge-working things, or the idea of point AI and a large amount of data, and you're gonna find interesting things, that in a nice relation seems pretty reasonable, like something we do every single day. But the idea of looking at all of the public data we have about space and considering a similar problem of go and see if you find anything in here we haven't noticed before, so beautiful. - Can you talk to me about your day? You have built clawed co-work, but you said you're not coding anymore. So what happens in the morning? What do you do? Is it like your agents are briefing you on what's coming? Or what is it? - That is true. My agents do brief me a little bit on what's coming for the day. I've really levelled up in sort of altitude on the problems that I'm thinking about, right? Like sometimes I feel like these days, I'm much more concerned with human emotion, empathy, trying to understand how people work in a way that I think I was less so as a classic engineering manager a couple of years ago. So I think for me, what I'm really discovering is that the amount of times I have to reach out to other professions to get their help has really dwindled down. I'm now this weird mix between like a product manager and engineering manager, just a traditional people manager, a data scientist, right? To answer your question very specifically, most days I'm busy with a lot of prototyping, right? We just try a lot of things. We sort of like think about stuff. We sleep over it for a couple of nights, try to figure out if it's a good idea or a bad idea. And then we just go tell Clot to like build all the various variations and ideas we had. And then we need to do critique, try to figure out which ones of these have good ideas, which ones are bad. But I think prototyping is really what I spent most of my day on. - How do you decide what is a good idea? Like what are the signals you're looking at? - This has been difficult for people for a long time now. And I'm not sure that I have all the answers to like what is a good idea, what is a bad idea? I think generally speaking, the way we look at good a bad idea is the calculus has changed, right? Like if you think about products we've built in the past for the last 10, 15 years, we always made implicit cost decisions. Like we sort of had to figure out, okay, especially for software. Can I build this? What is a maintenance cost? What is the ongoing cost? How much time do I invest in this? What is the opportunity cost? Only these decisions that engineering managers make to figure out do I go and build this, yes or no? And I think now we can fast forward to the end state much quicker. And we're like, all right, just build me a product, let me play with this, let's try it out. I think human attention is still rather limited. Let's say you for instance, you have a lot of taste in AI, right? Like I can probably show you maybe one thing a day. I can easily build a thousand. I can't show you a thousand. Even if you're willing to sit down with me for all day, like at some point your eyes are going to glaze over and you're like, okay, I'm kind of done with this. So what makes a good product? I think especially now that we live in this AI world, there's this all saying that like perfect is sort of when you can't take anything away anymore and less so about like building a product that is every single feature under the sun. Like we can make as many buttons as you want us to, but I think good products actually have fewer buttons. I kind of know whether or not a product really works if we put it in front of a few people and you see things click for them rather quickly and they can do things that they couldn't do before. That's really the signal I'm looking for. - And how do you think about the underlying problem that you're solving? So for example, if you came to my company now, how would you think about the tasks that you would fully automate with AI versus just leave it to humans versus maybe just automating parts of it? - The first thing I want to say is that I think about automation and augmentation in slightly different ways. Automation can be really empowering if you automate away tasks that sort of keep you from doing work that you enjoy or if it's like expensive and automating it away like saves you a lot of money or time. In those cases automation is useful and I think it's really where AI currently probably provides most of the economic value, right? Like when we evaluate is this method or not we usually look at automation. The area I find most interesting though and I think where people really do things that they weren't possible with AI before is augmentation. It's this idea of AI is helping you do things that you couldn't do by hand because maybe you're lacking skill or techniques or just the raw knowledge to do something. And when I go to company, the thing that is often most impressive to me is less so what the tool does and more so how you change to workflow. Does that make any sense? - Yeah. - AI is an interesting product because to really harness it, you kind of have to rethink how you do things. You have to think about your outcomes. You have to figure out what does good mean, what is bad mean. How do I evaluate whether or not this thing that I'm asking for whether the result is good. And the people who impress me the most, I usually the ones who are very good at coming up with very good at reflecting on their own workflows. - Yeah, but it's so hard. When I'm thinking about my work for example, I know people use AI for drafting texts. I think how do you define a good text? Is it still possible for knowledge work? 'Cause for coding, I guess it's possible, right? 'Cause you know what you want, you don't want any bugs. But when it comes to knowledge work and the result of human creativity, how do you even define what's good for the agent? - Yeah, it's such a tricky problem. The example I often go with is let's say, your job is to report on the weather, right? And you're like, "Agent, please figure out what the temperature is outside." There's two ways to look at that because you have to rethink what good means, right? If the model just guesses the temperature and is correct, that should not be a passing grade. That should be bad. We should rethink what we're doing here. But if the model is actually off by half a degree or something, but it really figured out the right location, figured out some reputable data sources and the way it got there was reputable and good, that should actually be better than a gased good result. That makes any sense. But for writing and all the other outputs where if you like a lot of taste is involved and it's a little hard to prove or verify whether or not the result is better than another, it is tricky, but I think I really enjoy talking to very good writers who are also a little AI friendly. - What do they say? - It is fascinating to watch them work. My friend, Adam Davidson, who was a staff writer at a number of pretty good publications, he has all of these various prompts for Claude, and he's like, "Okay, take my argument and put it to magnify it by 100, just mess with my text a little bit." What are the things you disagree with the most? What is the strongest counter-argument? It's interesting because I think to me, as someone who's not a writer, I think the naive approach would be given a text Claude, "Make this good." And I think what I'm discovering more and more is the for knowledge work. The secret sauce is like less than asking for a good output and sort of doing one step in between which is The best in the field. How do they work and can we automate those steps? This wouldn't kind of mimic their Workflow, but still the taste isn't there. That's why I think you still need a lot of yeah, I think that's true I think that's true But the interesting thing is that if you talk to Really good fighters most of them would disagree that the Texas really good just because they have like the medic touch They would often say no, no, this is the result of hard work. I'm just saying right like it's this idea of like most artists have a process and Many of them would actually probably find it I don't want to say insulting but like if you the more talk to artists and the more I explore this idea of like Is your stuff just good because it comes natural to you and it just like happens to be good? Or do you have a process that you like refined over the years a lot of them are quite proud of their process I'm like now this is very very hard work like there's a lot like a lot of like Manual individual steps going to making this good. It's not like I sit down and suddenly there's a book That's great and if you go to like vider studios right there's like a lot of white boards and like a lot of information And I'm noticing this across a number of different domains where Claude is quite receptive to being told Not I wonder result that looks like haha, but Claude I want you to take the same steps that a professional in this field would take This is like very high-minus. Let me up bring this down all but I think we're currently in the public scene this quite a bit with like motion video and 3d Where I think opus five five has made a pretty big jump But the big jump that the public has made is to just tell Claude to use blender, right? Just like Claude please use the same tools the professional animators use and now the results are quite good Yeah, the unlock really is here in the media and the technique and the tools and this less so and like Braun understanding of what makes a good animated movie this episode is brought to you by indeed you're ready to level up your Business but the pressure to find the right person to make that happen is intense indeed sponsor jobs can help carry the load It boosts your job post to help it reach quality talent fast cut through the chaos with indeed sponsor jobs Get a $75 sponsor job credit at indeed.com/podcast terms and conditions apply Hirey now then this is a job for indeed sponsor jobs When you shop croger delivery you can expect the savings you love and fresh groceries delivered right to your door Our refrigerated trucks keep your items fresh while our associates are committed to getting every detail right Carefully selecting and packing your order with care because bringing you fresh quality groceries is what we do best And right now enjoy $20 off your first online order of $75 or more restrictions apply seasite for details Croger fresh for everyone If you're listening on Spotify and this conversation was useful Please, please, please radar show. I see all you've used but I'm not getting the actual feedback This can really help our team to decide what to fill next and it can help someone who's never heard of us Decide whether to give us a listen. Thank you so much But I wonder for people like you mentioned this prompt act as if you're a professional in this field right and it's gonna go on the internet and research But a lot of information is just not there How do you trust the output like for example if I want to start another medium or like do Discover stars right and I have no idea about that But I just tell it act like an asset researcher How do I protect myself from AI just confidently presenting mistakes to me. Yeah, that's a good point I think it still happens to a lot of us. Um, I have I have like a very practical easy answer Which is adversarial reviews Um, which has proven quite effective in coding math and like various other areas where we can Give you numerical results on like how effective it is adversarial reviews basically ball down to the idea of you've given given one clotted task Spin off another 10 that try to like break this down their whole task is to attack this and prove to your why it's wrong It's essentially the same concept as rat teaming or short selling I using the same model for that or you want to use another model Kind of depends on the task. This is a little context dependent Um, but usually the adversarial review It is it is sometimes easier to find flaws than to create new things much like as it is with humans right It's much easier for me to say whether or not the new phone is good It's much easier for me to do that than to design a new phone from scratch So usually you can like take a step down into engines and still get very good results That's the very practical advice. How would you prompt that like if there is a process already what's the additional form that you add? So you have some kind of output right like let's say it's a slide deck or a document Or in my case, it's you talked about this blog post that I wrote this morning I just tell Claude very I'm using Claude projects Which is this new product that we have that allows you to do work in multiple threads with a little bit more ease And I basically told Claude please fire our five threads and task each of them with like proving that this This post is wrong that I'm thinking about it the wrong way that I have my facts wrong that I misunderstood something And then they go off with a given goal of like break this down You know proof that I'm wrong. How do you then deal with all the output because I do the same But I use I just use different models when I'm preparing for the interview like tell me if I'm digging too deep into this or that But then I'm giving so much different feedback and I'm like okay Now I have to decide because this model gave me this this gave me something else and gave this gave me something completely different Yeah, I think you're going to enjoy this because you can see where this is going the answer is once again Claude But we use an encoding context something we call a judge Where you get the output from the people who are like the the agents whose job it was to attack your output And then I here's our main points of criticisms And then you use yet another Claude and there a job is your your jobs to validate and verify and like opinionate whether or not this is valid feedback And then what you can sometimes do is that ultimately Depending on how you sort of your workflow The feedback that is valid to probably go back to the cloud that actually made the thing And that claw to that agent who can be discerning and whether or not to act on feedback right because sometimes Um, and we see this in a coding context fairly often I also see it in like the knowledge work output that I make sometimes the feedback is like maybe reasonable but Um has missed a point or is actually not that actionable or is not that useful and that's usually when I know that my output is pretty good That's one way that's when you are really quite uncertain whether or not the output is good yourself and you can't easily check But for a lot of a lot of things that I create with AI on the other hand I actually don't need to fully understand the entire flow because as I mentioned like to me as a person I do have opinions about almost anything I use every single day I've really made my website with all of these 3d assets I have no idea how to use blender. I've never used it But I can sort of judge whether or not this looks good feels good and that might be enough Right like you sort of put yourself much more in a client position for so many other things in our life We don't really have a good way of verifying whether or not we get a good output other than like Giving feedback to the person we're using right like for a contractor or lawyer or like For many people a doctor, right? So you sort of just put it in one of these two buckets like either you really have no idea what goods like good looks like There my answer is more AI just throw more tokens at it Or you do have an you do have an understanding what good looks like and then maybe that's enough And you don't really need to like have opinions about the process itself Yeah, I think it comes down to Making sure you're select only the top projects where you want to understand every single step instead of just want to control everything Yeah, yeah, it's like the old problem of like sort of being willing to let go right exactly Not micromanage. Yeah, it's really hard comes to the end now. We have this problem with agents Is there any knowledge for us is that you have nailed with agents like you completely trust them They just go. I don't know. Maybe it's yeah, I'm playing like editorial tracking I fully trust now the output I get 100% if I ask Claude to verify that all the facts that I'm stating in the slide deck and this presentation Um, it doesn't even need them any rounds, right? Like it's not like I have 50 agents doing that I really trust one agent I'm like I need you to find primary sources for every number I mentioned I want you to verify that that is a good primary source. I fully trust that 100% when you say agent Do you have a special agent for fact checking that you've trained or is just And one agent that you're using forever is just just cloud threat. I feel like this is going to be a spicy take Which I think you'll like I think we're kind of over rotating on this agents thing and I can back this up with a little bit like I think models Of really good generalists And we've seen this now time and time again like when we first started looking at math as an industry I think the the first company to really look at math like with with a very serious lens was google deep mind and they trained a custom model and a custom harness And they unleashed it on the math Olympiad and they got a silver medal which at the time was state of the art And then four five months later All the major generalist models come along and win a gold medal I understand why people use agents. I have empathy because it's easier for us humans to rationalize What a I can do if we sort of like put on a little hat like you're a data scientist, right? But that same data scientist has all the intelligence as like a field leading biologist or mathematician or what I have you And I think we're we're sometimes limiting ourselves and our use of AI If we sort of force ourselves to first cut like make up this cast of characters So for a lot of these things that I just met I didn't really have any agents who are being told who they are But does the data matter? Because for every agent you upload certain data set or just because your clot has access to all your tools It doesn't really matter what you feed to each particular age exactly Isn't it a lot of data for it to consider when answering each particular question? It is true, but it's quite discerning and what kind of data and what's your use, right? Like I think the right approach here is not to make these slices of data And then give every single agent the slice of data that at once because many of us if we want to be quite effective at our job We do actually need the full view on the entire business right? We're not quite effective if we only get a small slice of data Like the as an engineering manager if you only give me code access and probably going to be much less useful to you as opposed to if I also have customer data access to customer calls. Sometimes I do want to sit in like a customer call like when we're trying to sell something, I want to have deep usage information until you get business data about what is working, what isn't, and you want your agents to have the same range of data. Now, how do you deal with the fact that it's probably a lot of data? Tools. You give them tools and you document really well what kind of data is behind this tool so that they don't actually have to ingest the whole thing, right? You're just like if you do want business data, here's the tool to get business data. So when you're setting this up, because I'm thinking like when you mentioned this, should I move away from all my projects that have access to all the like specific data? For example, I don't know, there's a project for ranking a guest and we have the whole description of the process and then all the old scripts there so that we know how each guest performed. But now that you're saying this, does that mean that I should move away from that project? Tell my cloud, okay, we have this as a folder with all the scripts. If you want to look at them in motion, we have all the intelligence on how every podcast guest performed. And this is how we're in guests and just leave it there. And whenever I want to rank a guest, I just start a new chat without telling it anything and just tell it rank the guest. And it's going to yeah, I'm careful to be prescriptive in workflows. Like I always say, if it works for you, it works for you, I don't think you should change it. The problem is the bottom is getting better. And I want to get better. So this has been working for this year, but a year is a lot. Yeah, okay. Well, I mean, if if you're looking for the recommendation that I give for like, what do I do? Again, everyone should do whatever makes them happy and comfortable and what makes them powerful. But it is true that the models have gotten so much better at tool calling and discerning tool calling. And tool calling is this process of like the model knows when to use a tool, how to use a tool doesn't like get stuck in tool call loops doesn't accidentally say, oh, I found a data tool call. Let me call this tool until there's no more data left, right? Like the models have gotten particularly good at efficiently and effectively using tools. And for that reason, I don't really make this a priori decision anymore on what kind of data do you need? It also allows me to like have more threads going and more sessions going without having to cross as much easier. Do you still use skills? We do use skills quite a bit still, although our official andthropic guidance as well as my personal guidance is be really careful with skills. We often tell models things that were sort of relevant for like the last generation of models, which time is so compressed in AI, right? The half-life of a good skill is very short. And sometimes what we see is if you completely remove the skill, you actually get better output. Yeah, I think for his journey said that, you know, just forget about it. Just prompt. Just prompt and see what happens. Yeah, it is very effective. This is obviously most useful to people who have some kind of evil and have an idea of what good and bad looks like. But I'm pretty careful and very discerning when I use skills and when I don't. Okay. Do you know what a what a great way to think about this is? I like to consider the model experience as opposed to like user experience. And model experience is you pretend for a second that you're the model. You're a cloud, right? What kind of things would you want if you had to fulfill whatever task you've been given? And you're very quickly discovered that you actually do want access to the raw data, you want access to a computer, you probably want to be able to install some software. I wanted to make some decisions myself without somebody telling me what they think already. I just want to see exactly what you said raw data and make my own decisions. Like I think the upfront cost is higher. Like if you like even a year and a half ago, I was in this world of like telling cloud, I found another crash. Please go and fix the crash, right? But it very quickly dawned on me. I'm just a conveyor belt for cloud, right? Like I go look at our crashes, pick one up, move it over, drop it off in front of cloud, I'm like, please fix this. I'm like, okay, this is this is no, I'm just going to cloud needs to have access to all the crash data and then it's like figure out itself, what it needs to fix, what it doesn't need to fix. The upfront cost is really worth it, but it's also like important, I think to consider that there is upfront cost. But once you set this up, it opens up so many doors for you and that's where I find a lot of like real hard value coming out of AI. If you've been hearing about AI agents and wondering how do you actually set up one, I put together a guide that walks you through your first one in about 20 minutes. It helps you choose what to automate and get it working with no coding required. You can get it through my free newsletter, the link is below. For someone who just, who's listening to this conversation and who thought they would need like a major setup to start using AI correctly, would you say if somebody's starting just to date, they're using it for knowledge work, they don't need all these tone of voice things like my business strategy, you just give it access to your email, it learns from it, from your tone of voice, you give access to your newsletters or whatever. So you don't need any of that setup anymore. The best way to get started with cloud today, I think is step one, download the desktop app. It has a built-in browser where you can log cloud into some system you're using and forget about connectors, forget about like downloading all the data, forget about like all kinds of things, just let cloud use your computer a little bit, sort of give away the keyboard. That gets you so far, it nearly shows you what works and what doesn't work and then at some point you might discover, oh, cloud is really helping me with whatever tool you're using, whatever internal external tool you're using and once you see that a workflow really works for you, you might consider, okay, can we give cloud something better than like having to manually click in this dashboard around? Can we give it some kind of connector? But just starting with the browser, it gets you so far. Okay, and when it just to wrap up with the agents, when you say agents, that means you have just basically chats, right? And that's it, there's no like specification, what does research, no specific agent, what does your tax return say? No, no, I mean, I do have projects, so I do think about, I do think about the different work items, but I don't think about different roles, right? So I don't, I never tell a cloud, you're an excellent whatever. Yeah, I think this was this was very unvoked, sort of like 2024, we were like, cloud, you're an excellent engineer, please go fix whatever problems are here. But you still use it in prompting now, act like something that we started with, act like a top whatever. I specifically implore a cloud to like tell me what tools someone top of their field in this field would use. And then I'm like, I want to use the same fields. And a problem most models still have today is that if you simply tell it data sciences, maybe a good example or like accounting, a lot of the models still have the problem that if you tell them to act like a very good accountant, that they engage in a little bit of roleplay, and roleplay is not really what we're going for. We want the techniques, the tools, the knowledge that a professional would use, we don't really want just the vocabulary, right? Like I don't, I don't need cloud to sound like, I don't need to put on a costume. I actually wanted to talk to me like I'm five, but I wanted to use all the top tools in the field. It's a tricky question. We can just keep layering more validation on time. So I think the question for me is much more, it's less about, it's less about what do I not trust cloud with, and much more about what things do not trust one single session with. And I think the list of things where I don't trust one single session is actually much larger than the list of things I wouldn't trust cloud with. Like the current cloud session that is running wherever you have it in cloud code, in cloud core work and chat, and even then a project, the current session tends to be quite bad at being a judge of whether or not the work it did was quite good. So as much as I can, I try to find that outside signal. And the outside signal in a pinch can be another cloud session that has proven to be quite effective. But my preference is to find some hard outside evidence for what works or what doesn't work. And sometimes you need to like trust the process a little bit and give cloud a little bit of time. Like for instance, in your field, I'm kind of making this up, but I assume that maybe like the number of users or subscribers is like an important metric thing. No, the number of views, that's that's what we're whenever I give a task to cloud, I tell it like we need to hit this number of views. Yeah. Go and try. But that and that's that's something that we're still like struggling with is this feedback loop, right? Yeah. Because it's me giving feedback to my team. And then my team actually going and posting something else, not what cloud gave us. It's like maybe 70% the same, but then the 30% that I've added is actually my taste. Yeah. How do you close that loop? I think that has worked for us. There's probably some kind of place where this feedback usually happens. Is it usually in person or does it usually happen? It's for forever. There isn't a native cloud connection. And we have a block that scrapes every chat, but it's still very manual. I have to trigger cloud to look at that block, look at the feedback. Yeah. It's still not it's it's not there, even like looking at something we've sent, right? Yeah. Because we do we do obviously again in a coding context, we have a lot of engineers who you sort of know which engineer you want to review a certain thing that you're working on. And the kind of things that they look for, cloud is very good at figuring out from the feedback. It's pretty good to like given all of this feedback that I've read. Feedback on Slack, right? Yeah. On GitHub Slack, wherever else the feedback might happen. And you didn't tell it to look at feedback specifically just because you gave it access to Slack. It's looking for feedback. Yeah. That is much easier to do. Like if it has the tools and you don't have to like as human yourself figure out, okay? Let me assemble a list of all the feedback I've ever given. But if you do this like three step process of step one, assemble all the feedback you've ever given step two, tell a cloud to like condense that down into a list of rules or principles that you care about. Sometimes it's useful to like then again do a third round of like, okay, what are the principles that maybe you might believe, but you don't actually know you believe. And then take that list as like an input for your ongoing reviews. That is that has been quite successful in our attempts to I want to say like specialize the feedback that individual sessions gave. That has really worked. I think for for work that is a little different than raw coding or not very text-based, I think the problem is still bigger. But I think similar thing things can be done, right about like give me feedback on a customer call. How did that go? If you read the transcript or like maybe you're, I'm doing a lot of interviews for Anthropic, which is like a typical knowledge work problem, but also like quite interesting. When you shop Croger Delivery, you can expect the savings you love and fresh groceries delivered right to your door. Our refrigerated trucks keep your items fresh while our associates are committed to getting every detail right, carefully selecting and packing your order with care, because bringing you fresh, quality groceries is what we do best. Croger, fresh for everyone. But then it's teaching me. I wanted to teach itself, and I think a lot of people who started using I experienced this. You want to do more. You see how productive you are. I don't, there is no agent that I trust 100%. Every output goes through my team and then eventually through me. So I become the biggest bottleneck in the company. Honestly, I feel like our products would do that for you pretty well today. I'm actually quite confident. Like if Claude has the ability to watch you give this feedback for a while, like the first thing you would probably do is you would sort of run it side by side. This is an experiment I recommend to almost everyone. It's like really fun to do. If you currently have a trust problem, you will probably not be super excited to immediately give over the reins from one day to the next, but sort of run it side by side. You read something, you write down your feedback, and you do that for a few rounds, and then just to check what kind of feedback Claude would have given. Where do they differ? Where are they the same? That also gives you something where you can actually sharpen the workflow a little bit, because I think models will still be imperfect. There's still going to be things where the feedback is a little different, and I give them for most humans. If we take the same human and we ask them to review the same document on a Monday versus a Friday, the feedback will be different, but this side by side process is rather powerful because it shows you so much. How do I build this agent that talks to my team? Is it just something that, like, a website where they upload stuff and it gives feedback? I would recommend Claude Tag, which is Claude Insights Lack, and you give it access to my taste. Yeah, it's like, it's like currently our core multiplayer experience, right, where people can talk to Claude at the same time. Yeah, yeah. This is all getting very interesting when it comes to building things, right? Because you can build so many things easy. What becomes valuable when you're building software? And I'm talking about people who are beginner entrepreneurs. I'm thinking about myself. What is it? To 2015, we came here with a booking platform for study broad experiences. Amazing. A mode was building this whole platform because we hired a lot of engineers, that hundreds of thousands of dollars, we automated stuff. Now I can do this in two days by myself. Yeah. What is my entrepreneurial mode when you think about building software? Is there any? Spies your opinion. I think it actually hasn't changed that much. I think the cost of investment, like some of the upfront capital cost, was much higher that is true. I think we're now entering this interesting area where like single-person teams or like two-person teams can build billion dollar companies. But I think the secret to building a good product is actually not changed. But I believe it's empathy, empathy for the user, empathy for the use case, empathy for like trying to figure out if this is actually valuable for the person who's ultimately going to pay for it. I think that is, again, and I think this is maybe a spies your opinion, but I think this has always been the key ingredient in any entrepreneurial team, right? And I think especially in startups, like before we started taping, you and I talked a little bit about five-front startups. I think this is even like sort of common knowledge with NBC's that whatever you show up with doesn't really matter. It's probably going to change anyway. But you as the founder, you're the key ingredient. And we can go one level deeper. What does that mean? Is it like you're telling your skills? I think empathy and like an ability to really build something of value is probably the core thing that matters in a founder, and I think that has not changed. It has just gotten much easier to tap into that empathy and like to execute on it. I totally agree with that, but also sometimes I'm thinking like I have this idea, but then I see something on X where we'll say, "Oh, this new model just killed this." And we see people vibe-coding their smaller tools. Do you think it's a trend that we're going to vibe-code a lot of tools versus paying for them? Like what would us make us pay for a certain tool if we can spend an hour just building something similar? I think software that is hopefully going to go away is bad software. Like the software that doesn't really serve your use case, and you're only using it because you don't have anything better. I think all of us have some tools that come to mind where you're like begrudgingly, "Okay, I guess I'm going to use this because I have to." And I don't have the time to like look for something else or use something else. I think that will come to an end. However, at the same time, we talked about how hard it is to build good products. I think there's still going to be tremendous value in like figuring out how to best deliver value and how to like to have empathy in a way where most of the products I use that I'm like a big fan of, I wouldn't have come up with myself. Right? Like I'm on record as like having written blog posts about how the iPhone is stupid because obviously you need a keyboard. I have. Of course. I think for most of the products that I really enjoy, someone else came up with an idea executed on it and then gave it to me. And I think the main reason I use those products is really because it was very expensive to build or because it took them a long time or because they needed a big team. The reason I use the product is because it solves the need I have really, really well, probably in a way that was better than I thought of. The gap that I think we're going to completely fill is this garbage software and I only have to use it because I have no other choice. I think that will hopefully go away and I'm very excited about that future. Before we keep going, I'd love to get to know you a little better. I share a lot about my work here and I'm curious about yours. What do you do and what you'd like more help with? We've put together a little survey. Your answers will help me decide what to cover in future conversations and what resources to put together for you. I'll leave the link in the description. Please, please, please take a couple of minutes and tell me about you, about your business, about your ambition so I can make better content for you. I really, really appreciate you. Thank you so much for doing this. You mentioned something about Windows computers that quietly run work and that they are a big business opportunity. Can you talk to me about that? I built AI products. I work on cloud. When I go out into the world and look at what are the computers and the machines that are really running the economy, I think it's really easy, especially for like Austin. So I can value to consider your typical startup with a bunch of latest generation MacBooks and that's all that exists. And then you go to an airport and you peek around the corner and you see that they have not actually yet moved to graphical operating systems. It's still like the black and white. Oh, yeah. The transportation authorities, yeah. And I'm doing this for fun now sometimes that I just go to all of these places that run some of the economy. Like the last place I did this was a hotel. Like I love doing this at a check-in where I'm like, "Hi, I work in software, I build tools." Can you just show me real quick what you do on your little machine right there to check me in? Heroing, like all of these devices have software that is so clunky, so bad, truly awful. And I think it is probably an untapped opportunity for many businesses to go in there and try to clean that up. Well, before I became a thing in 2010, I joke that if I have wanted to retire, I would build really good software for dentist offices. Now you can build it in a day, I guess. Is the relationship part? Yeah, exactly. I think that's becoming, well, that's the hardest, right, not just the software. Any other businesses you wish existed? I still think we'd be early getting started on software that feels a little bit more personal and has a little bit of a touch of AI that is not just, he's a sidebar with a chat partner. I think we've sort of permeated the entire industry with low intelligence, cheap models that operate sort of at the level of, "I have read the FAQ page, and if you give me a question, I'm sort of lying, I could spit out the answer." We haven't really figured out what can we do with top-line intelligence across many of our applications. Businesses that I specifically wish existed, I'm just going to double down on this empathy thing. The industry moves slowly, right? Like, even we see this move kind of slowly, and it's really hard to adjust to a new world. And I don't think we've properly adjusted to execution as cheap. We've always said ideas are cheap, that was true. And we said execution kind of expensive, but it's gotten cheaper over time. And now execution has become extremely cheap. I think we need to really double down on empathy, connections, who's capable of going into a business and really understanding what the needs are and how can you actually optimize things here. So this might actually be a nice opportunity, like the bigger eventually, the MBAs. Right? I think MBAs can now make a bunch of software startups that might be quite successful. I was just talking to a Nobel Prize Laureate in economics, and he was telling me that we have such powerful models, but we really lack the application layer. We agree with him that there is this huge gap between what models are capable of and what average user he's using them for. Oh, yeah, for sure. Yeah, we definitely have like a capability overhang, right? Like the idea of you have an amazing model in front of you, and if you're essentially using it to ask, "Hey, what's the capital of France again?" You're probably not really tapping into what is possible there. Our founder in CEO Darryl talks about this concept of like a data center of geniuses. My job relates to figure out, "Okay, let's assume we have a data center of geniuses. How do I empower you, the user, to actually make use of that?" And it requires creativity, it requires like a rethinking of like your needs and like what you think is possible, it's a really hard problem that I don't actually think is new. Right? notion of people have always been held back by what they thought is possible rather than what it's actually possible. Yeah, I do agree that we have like, we've just scratched the surface of what's possible. How can one close that on an individual level? On an individual level? I thought about this all weekend. I think on an individual level, the thing that you see on acts a lot is this rallying cry of be more ambitious. We ought to be more ambitious. Like what being more ambitious really means is I think so context dependent, but I would like people. To shut all the shackles of like what they believe is possible. And rethink why are they doing things, right? Like what like I'm doing a certain task in my business, what I what I'm hoping for is going to happen next. And you just keep giving the model that task, like whatever outcome you're hoping for, give the model that task, you can't play this forever. You will eventually find the limits of the current top intelligence, like a good counter argument here is okay. Like the purpose for most businesses is to make money. And if I just talk a lot, go make money. It's probably going to like not make it very far. So we haven't reached like the top level of altitude yet. It's somewhere in the middle. But once you bump against that ceiling, you do know that you're at the forefront, you're at the frontier and you're going to really benefit, you're going to benefit from the next model generations, because if all we're doing is like summarize this email for me or like make this email sound as angry, you're not really like benefiting from the new intelligence. We'll be your advice to someone who's just starting out now as a software engineer in this ever evolving. In this ever evolving like landscape of like how to do software engineering, I think it's, I think it really depends on what you want as a software engineer, right? Like what is it? What does it mean to be software engineer? I think we're like as engineers are currently in this like moment of crisis where we're like, what does it mean to be software engineer? Does it mean that I've completely memorized the entire API service and react? Is that what I'm valuable for? Is that what I do? And I think that's probably less the case now. To me, this is a pretty welcome change. I've always been of the opinion that a good engineer versus a bad engineer, I measure, I've always measured my engineers on their impact and the impact is like, does the user like this enough to buy the software? There's always for me been relatively little value in like really elegant, beautiful mastery solutions where we can be really proud of the engineering craft here, but the user doesn't feel any benefit. How do you develop that? What do you double down on now in order to be relevant in three years if like every position is only a senior position or be your advice there? The way I got my start in this career is by like building a strong portfolio. I think that would be the same advice today. I think building a portfolio that is not just like a mock is much easier. So my recommendation for anyone young would be go and build some real things, prove that you can do it, prove that you have the empathy, prove that you have the chops, the ability to talk to customers, like figure out what is necessary versus not necessary. I do think that's like a slightly new definition of what software engineering is. I personally think that's good. I do see the young people adapt to this new style of thinking much quicker than some of the older engineers who maybe enjoyed the luxury of primarily working on on tickets without having to ask why too many times. I've seen so many people be quite successful in that. The things that I would maybe to contrast not recommend anymore. It is true that I think like completely memorizing individual API services, I think has become less valuable. What is still valuable is being able to combine different systems in interesting ways. I think the value of that is amplified now because you can so quickly combine things, right? You can just like point AI like three different systems. I mean, like I want these three, like I want these five ingredients mashed together, go and do it. Is that something that you look for when you hire people? Yeah. Disability to, is there something else that you particularly looking at? I don't know. You hire. Do you still do you care a lot about the resume? I do want to cause when you mentioned relationships, they're very important for my industry. I am looking for someone who knows people. I am looking for someone who knows how other business operate because this is not public information. Like how do I learn how to, what, what do I charge as a podcast or right? What are your charge brands? I want someone who knows exactly what's going on in the market. So we do care about relationships. But when it comes to the creative part, actually with creative part, we still look at how many views their videos have produced because exactly, they know the art. I don't look at their degree, of course. Yeah, yeah, exactly. Exactly. And I think this is, I don't know if this is still controversial. I guess we both work like in a, you know, like tech adjacent but I've never cared too much about which degree do you have or which school you went to. It was always much easier for me to look at what are the things you made are those things I can touch, feel, try and I can feel like, oh, you either were the winning player on a team or you at least worked on a winning team that is sometimes useful for me to know. I think there's really valuable information. And the thing is if I didn't have that I might go take the shortcut and look for schools degrees, I don't really have to. I will always pick someone who's proven themselves to build something amazing over someone who hasn't built something amazing, but hasn't amazing degree. What would you tell your kid when she grows up and says like, "Dad, I wanted a coding, what would be your advice?" I mean, she's nine months old right now. So I think it seems full change a little bit. I think you have to be in line for your daycare already. Just saying. And all happens quicker than you think. But when I tell my daughter, I think I would tell my daughter, to very early on, try to understand how businesses operate, how people collaborate, how to really find value. I've not gone to the University of Waterloo but I'm such a fan of the University of Waterloo's process. All of the amazing minds. All the amazing minds. But I think the University of Waterloo has always had this approach which was quite unique within computer science engineering, which was this is an outside perspective. They might disagree with me on this. But my outside perspective was always coming from Germany, where computer science is very much computer science. It's informatics, it's a lot of math. And very little, like only a sliver of how do we actually build a good product, let alone computer engineering. JavaScript was in Germany for a long time, universally considered to be not what real computer scientists use. And the University of Waterloo's approach has always fascinated me so much because I would get these interns, a junior engineer who worked at five different companies before in their internships. It was pretty common, especially in San Francisco, that you get an intern and it's their fourth internship. And they're previously interned at Tesla, Google, and Apple. And they have so many interesting things they learned about. How companies operate, what are the upsides and downsides? How do you actually advocate for an idea? How do you get resources? How do you convince legal to get something done? I think those are the skills I'd really like for me made University of Waterloo's engineers almost immediately effective. It's their first job, maybe with me. But they're in a completely different category than someone who has maybe even gotten a PhD in computer science but has never worked in a company. They might be very smart what they do. But as we all know, building a good product and getting it out there is about much more than just knowing what code to write. You need to be able to talk to people. You need to be able to work together with other functions. You need to have a little bit of a sense for how does this company run. So I would tell my daughter to probably really invest in that. Love it. What do you tell your kids? What do you think your kids should do in the ever-evolving economy? I tell that. You're ready to level up your business but the pressure to find the right person to make that happen is intense. Indeed, sponsor jobs can help carry the load. Cut through the chaos with Indeed Sponsored Jobs. Get a $75 sponsor job credit at Indeed.com/podcast Terms and Conditions Apply. Hire it now. Then this is a job for Indeed Sponsored Jobs. When you shop, Kruger Delivery, you can expect the savings you love and fresh groceries delivered right to your door. Our refrigerated trucks keep your items fresh while our associates are committed to getting every detail right, carefully selecting and packing your order with care. Because bringing you fresh, quality groceries is what we do best. Restriction Supply. See site for details. Kruger, fresh for everyone. I'm to try all the things and I tell them you can be anything because sometimes they're like, "Oh, robots, you can be the queen of. " If I tell her you can build a robot, she's not very excited. I tell her, "You can be the queen of robots." And she's like, "Oh, I like queen of robots." And then she says, "We bought this self-driving car that you can program yourself." And she was building it with her dad. And just telling them they can be anything and there is no saying that they cannot touch because it's, I don't know, out of this or whatever. Because I came here with a notion that everyone here is from another planet. If you went to Stanford for me, that was like something that I couldn't even fat them. But seeing that these are all normal people, just very passionate, hardworking, and giving this idea to my daughters that anything is possible, is just hard work and passion. Oh, man, I love that. I think the reason I love that so much is because in AI circles, we constantly talk about this idea of ambition and the reason people are not ambitious enough is because we're sort of programmed as humans to only consider certain things possible and if we don't consider them possible, we don't even try. So what we believe is possible is the real limiter. Like a much bigger limiter than what the models can do or what we can do. Maybe that's another reason I really believe in the new generation is because they will never have experienced all these limitations that we have. Okay, I think we covered so much. I just wanted your final thought on our future. How do you think our life is going to be different in five years? That is such a big question. I'm actually of the right person for like, I know that going on any podcast is like there's an occupational hazard of having to make predictions. The biggest thing that I've noticed so far is that we're all so wrong about the intelligence exponential coming out of AI. It's really hard to like wrap your finger around. What does it really mean for the exponential to exist? And I keep going back to math, right? Like when we won the silver medal at the Math Olympiad when I say we, I mean Google, as an industry, I was well aware of scanning loss, I was well aware of like how much internet is progressing and I think if you'd ask me at what point will AI solve what we would otherwise consider unsolvable math problems. I don't think I would have given you like the middle of 2026, I would have been like that's that's in a year and a half what are you talking about. So I think instinctively having a good feeling for how well what will the future look like is really really tricky that's the first part and I think the second part that is rather tricky for me is the one thing I can tell you with a decent amount of confidence is that I think the products that we use today will all look completely different. I think I think we're currently we've hit this point where AI is coming so mainstream that's becoming so popular that it's like easy to imagine or we're maybe like this this is sort of the end state but I'm quite convinced that all the AI products we use today are roughly in the same face as all the funny phones that we had right before the smartphone came along so I think we're still in the pre-i phone era which makes it very exciting for me to work in. Wow this is really fascinating it's already like from in our pre-AI era there are so many things that are changing and you made me rethink my whole process that won't even show you what it what it looks like now because it's fine if it's very outdated because we give it all the data and tell it exactly what to look at but I totally understand how giving it access to everything is just so much more powerful. It's such a tricky thing right like all the advice I've given you today in about 10 weeks we're probably going to have a new revision on all of it. That is also true so being an always on agent when it comes to new models I think that's the that's the thing in trying new things thank you so much Felix if this conversation with Felix made you want to try Claude in your work watch my interview with Ali Miller next alley works with teams on using AI in in our conversation she showed me her real workflows and built a Claude morning brief alive. It's a practical place to start if you're wondering what to do first.