Andrej Karpathy’s vision of Software 1.0 → 2.0 → 3.0

Software eras If you've been watching the software world, you might think, well, things have been pretty stable for, what, 70 years? You write code, computers run it pretty straightforward. Well, get ready, because that whole picture is changing fundamentally, and not just once, but twice, like right on top of each other. It's a really, really unique time to be involved with tech, and today we're going to dive deep into exactly why. We're drawing heavily on some fantastic insights from Andres Carpathy. You probably know him, former director of AI at Tesla. He's got this great framework for making sense of this huge evolution. Speaker 2 Yeah, Carpathy's framework really breaks it down nicely into what he calls the three eras of software. So first up is software. One point O this is basically what most people think of as software. It's traditional code written by humans. Think C++ at like the stuff that power the early Tesla Autopilot systems. You know, GitHub, that whole universe of code repositories. That's pretty much the map of software. One point O. Speaker 1 OK, so that's the code we all kind of grew up with, written explicitly by people. How fast did things start changing from that? Speaker 2 Actually, pretty quickly that led us into Carpathy software two point O and this is where neural networks really take centre stage. So here you're not writing code line by line in the same way. Instead it's about gathering huge amounts of data, running optimizers, and basically training the network. The output is the network's weights. Speaker 1 So the code is the learned way, it's not lines of text. Speaker 2 Exactly. And think of something like Hugging Face that's kind of become the GitHub for these software two point O models. Alexnet for image recognition is a classic example here. Although until pretty recently these were mostly like fixed function. Great for one specific thing, like image classification, but you couldn't really programme them broadly. Speaker 1 Right. And that brings us to the really wild part software 3 point O This is the newest thing driven by large language models, LLMS. The Big Deal And the huge deal here is that these models can be programmed using just natural language, often just plain English. I mean, telling a computer what to do in English, that's just wow, how big a deal is that really? Speaker 2 It's massive. It basically creates an entirely new kind of computer. Your prompts, your sentences are the programmes now. They directly instruct the LLM. It fundamentally changes that whole interface between US and the machines. Speaker 1 OK, so for you listening, what's our goal here today? The Big Opportunities Well, our mission in this deep dive is really to help you get your head around this new way of programming. We want to explore what these LLMS actually are as this new kind of computer, figure out how to work with their unique psychology, and importantly, spot the big opportunities opening up. And as we go, you'll hear Karpathy use some really cool analogies. He compares LLMS to things like utilities, factories, and maybe the most powerful 1 operating systems from way back in the 1960s. Speaker 2 Right. And we definitely need to unpack that psychology because these LLM's have these incredible superpowers, but also some frankly bizarre cognitive deficits you really need to know about if you're going to build with them. Speaker 1 Yeah, so whether you're just getting into tech or maybe thinking about how your current job might change, or even if you're just curious about how all this stuff is reshaping daily life, understanding these shifts is pretty critical right now. You know, Carpathy mentioned his time at Tesla working on Autopilot, and that really sheds light on the transition, doesn't it? Speaker 2 It does. He literally saw the software two point O stack the neural net starting to like eat the software one point O stack. Features originally written in C++ were getting replaced by train networks. Speaker 1 And his take now is that Software 3 Point O is doing the same thing again. Speaker 2 Yeah, he thinks we're seeing that exact pattern. Repeat software 3 point O is starting to take over tasks that used to belong to one point O or two point O, which means you know we suddenly have these three totally different ways to programme existing side by side. Being fluent in all three is fast becoming essential. Speaker 1 So developers now need to decide, OK, should this part be classic code, should it be a trained neural net, or should I just prompt an LLM? Speaker 2 Exactly. Making those architectural choices moving smoothly between these paradigms, that's a brand new core skill. Speaker 1 So let's dig into this new LLM computer. What is an LLM computer? What is it actually like? Carpathy's analogies sound really helpful here. Speaker 2 OK, first analogy. LLMS act a bit like utilities. Think about open AI, Anthropic, Google. These labs spend enormous amounts of money, huge capital expenditure to train these models. That's like building the power grid. Then there's the ongoing cost. The operational side to actually serve that intelligence through AP is that's where we pay, you know, per token used. Speaker 1 Like paying your electricity. Speaker 2 Bill, pretty much. And just like electricity, we expect to be fast, always on reliable quality and you can even switch providers like using open router to route your request to different LLMS, kind of like a transfer switch for power. It's interesting because it's software, so they don't physically compete for space, which allows for many global providers. Speaker 1 And we've actually seen this play out right When a major LLM goes down, people talk about an intelligence brownout. Speaker 2 Totally. The more we rely on them, the bigger the impact when they have issues. It affects productivity information access globally. Speaker 1 So they're shifting from just being tools to becoming actual critical infrastructure. Reliability becomes a huge deal, like the power grid. Speaker 2 Exactly right. And that brings a whole new set of responsibilities and vulnerabilities too, OK. Beyond utilities, Carpathy also sees parallels with fabs, those massive semiconductor factories. Parallels with chip fabs How so? Speaker 2 Well, again, the huge upfront cost, the capital expenditure to build an LLM is enormous, similar to building a chip fab. Plus there's this rapidly growing tech tree of R&D secrets building up inside these LLM labs, much like the proprietary processes in chip manufacturing. Now the analogy isn't perfect, because software is just inherently more changeable, less physically defensible than hardware. But you see the parallels, right? Using NVIDIA GPU's without making your own chips is like being a fabulous semiconductor company. Google training on their own TPU's? That's more like Intel having its own fads. Speaker 1 Interesting comparison, but you mentioned the analogy Carpathy likes best is operating systems from the 1960s. Operating systems from the 1960s That sounds intriguing. Why that comparison? Speaker 2 Yeah, he finds this one the most compelling because LLM's aren't simple commodities like electricity. They were doing incredibly complex software ecosystems. Yeah, think about it. You've got a few big closed source O layers like Windows and Mac O, right? And then you have Linux as the open source alternative, right? We're seeing a very similar structure emerge with LLMS. You have the big close models from Open AI, Google Anthropic, and then you have the Llama ecosystem sort of shaping up as an early version of Linux. Speaker 1 For LLM, OK, I can see that. Speaker 2 And we're still really early. These things are getting way more complex. They're starting to use tools, handle multiple modalities like images and sound. The LLM itself is kind of like the CPU. The context window acts like RAM or memory and the whole system orchestrates these resources to solve problems. Speaker 1 It really does feel like a fundamental rethinking of how computers work. When I use something like ChatGPT 4, it feels like there's this core intelligence running the show. Speaker 2 And another key point, the actual LLM compute is still super expensive. So it's forces centralization mostly in the cloud. We're all basically using thin clients interacting over the network, kind of time sharing on these big central brains, which is exactly how computing worked back in the 60s before personal computers. Speaker 1 So the PC revolution hasn't quite hit LMS yet. Speaker 2 Not fully, no, though people are trying apparently. Mac minis are surprisingly decent for running smaller models locally for certain tasks. Speaker 1 And Carpathy's point about interacting with ChatGPT like it's a 1960s terminal. What's the significance there? Speaker 2 It really highlights that we haven't invented the proper graphical user interface. The GUI for LLMS yet beyond just chat bubbles are we're still mostly typing commands, essentially operating in command line mode. Speaker 1 That's a great point. And there's another unique thing about LLMS, how they spread. Usually big tech like electricity or the Internet started with governments. Corporations then trickled down to us. Speaker 2 Right, because of cost and complexity. Speaker 1 But with LLMS, it's been the opposite. We, the consumers jumped on them first for like asking how to boil an egg or write an e-mail. Corporations are actually playing catch up. Speaker 2 Yeah, that's a fascinating inversion, and it tells you something about where the first really practical apps are likely to emerge. Speaker 1 OK, this picture of LLMS as these complex early stage operating systems distributed like utilities, accessible to billions almost overnight. What makes LLMs tick That's a massive change. And that accessibility points us towards the opportunities. But first we need to understand what makes these LLM computers tick. Carpathy calls them stochastic simulations of people. What does that mean? Speaker 2 Yeah, that's fundamental. An autoregressive transformer, which is the underlying tech is trained on basically the Internet, all that human text. And because it learns from our data, it develops this kind of emergent, almost human like psychology. Speaker 1 Which leads to both good and bad things. Speaker 2 Exactly. It gives them incredible superpowers, but also some really weird cognitive deficits that are absolutely crucial to understand. Speaker 1 OK, superpowers first. What are they? Speaker 2 Well, encyclopaedic knowledge is 1. Their memory recall is just vast, far beyond any single human. Carpathy compares it to Rain Man, you know, instantly recalling phone books or obscure facts. They definitely have superhuman aspects. Speaker 1 But then the downsides, the deficits. Speaker 2 And these are serious. First, they hallucinate a lot. They just make stuff up, sometimes very confidently. Their internal model of what they actually know is still imperfect. Let's say it is improving. Speaker 1 Right the confident wrong answers are tricky. Speaker 2 Then there's what he calls jagged intelligence. They can be brilliant at some really complex tasks like advanced reasoning or coding, but then turn around and make incredibly basic mistakes that no human would like. Insisting 9.1 is a bigger than 9.9 or spelling strawberry with two Rs. These are like little traps you can easily fall into. OK. Speaker 1 Jagged Intelligence. Got it. What else? Speaker 2 Maybe the biggest one for practical use is andrograde amnesia. Unlike a human colleague who learns, remembers past interactions, builds expertise over time, LLMS don't inherently do that. Their context window is like short term working memory. Every time you start a new chat or query, it's basically wiped clean unless you explicitly reminded of past context that. Speaker 1 Explains why just expecting them to get smarter on their own doesn't work. Or remember that thing you discussed last week? Speaker 2 Exactly. He mentions movies like Memento or 51st dates characters with knowledge but their memory resets constantly. That's kind of the default state for an LLM. It makes building ongoing relationships or complex projects tricky. Speaker 1 Wow, that's a powerful analogy. Speaker 2 And finally, there are security issues. They can be quite gullible, prone to prompt injection attacks where someone tricks them into doing something bad, or they might accidentally leak sensitive data if you're not careful. Speaker 1 So wrapping that up, the big challenge and also the opportunity is figuring out how to programme these superhuman, amnesiac, sometimes hallucinating entities to leverage their strengths while working around their weaknesses and security flaws. Speaker 2 That's the core task right now. Speaker 1 OK, so knowing all that, how do we actually start using these things effectively? Opportunities What are the big opportunities Carpathy sees for building useful stuff with LLMS? Speaker 2 One of the absolute biggest, he thinks, is in what he calls partial autonomy apps. Speaker 1 Partial autonomy meaning not fully automated. Speaker 2 Right, instead of just going to a general chat interface like ChatGPT and manually copying and pasting code or bug reports back and forth, it makes much more sense to use specialised applications designed for specific tasks. He points to Cursor, the coding assistant as a prime example that's gaining real traction. Speaker 1 OK, I've heard of cursor. How does it exemplify partial autonomy well? Speaker 2 It keeps a familiar interface. You can still code manually, but it seamlessly integrates the LLM to let you work in bigger chunks. You're not just asking questions as part of the coding workflow. Speaker 1 And these successful apps like Cursor, what do they have in common? Speaker 2 Carpathy highlights a few key things. One, the app manages a lot of the context for you. You don't have to manually feed the LLM all the relevant code files or project details every single time. Speaker 1 That sounds way more efficient 2. Speaker 2 They orchestrate multiple LLM calls behind the scenes. Cursor for example, might use one model for understanding code structure, another for generating suggestions, maybe another for applying changes. As a DIF. It handles all that complexity for you. Speaker 1 OK, context management and orchestration. What else? Speaker 2 A really crucial and maybe underrated feature is the application specific GUI, the graphical user interface. Speaker 1 Why is the GUI so important? Speaker 2 Because just getting raw text back from the LLM operating system isn't always helpful, especially for complex stuff like code changes. A good GUI like cursor showing code changes with red and green highlights let's a human review and audit the AI's work much much faster. Our visual processing is incredibly high bandwidth. Speaker 1 So you can quickly scan and accept or reject the changes with the click instead of reading dense text and typing instructions. Speaker 2 Exactly. It dramatically speeds up the human supervision part, which is critical because these systems still make mistakes. Speaker 1 Makes sense. And there was another feature you mentioned. Speaker 2 Yes, the autonomy slider. This is key in cursor. You can choose different levels of AI help like simple tap completion is low autonomy. Then maybe command K to change a selected chunk of code that's more autonomy. Command L might refactor a whole file command. I could let it try and generate something across your entire project, full autonomy for that task. Speaker 1 So you, the user get to decide how much control to hand over based on the task, or how much you trust the AI at that moment. Speaker 2 Precisely Perplexity. The Research Assistant app is another good example. It packages info, orchestrates LLMS, has a GY with source links for auditing, and offers different levels of search depth that it's autonomy slider. Speaker 1 OK, so this whole idea of partial autonomy apps raises a big question for anyone building software now, right? How are you going to make your product partially autonomous? Can an LLMC everything a user sees in your app? Can it take all the same actions? Speaker 2 And critically, can the human stay in the loop effectively to supervise? Given that the AI isn't perfect, it really implies that software design just for humans now needs to evolve. It needs to become more understandable and controllable by LLMS too. Speaker 1 That leads nicely into the generation verification loop idea. Generation verification loop Can you unpack that? Speaker 2 Sure, the AI generates something, code, text and analysis. The human verifies it cheques if it's correct, safe, useful. Our goal as builders should be to make that loop incredibly fast and efficient. Speaker 1 And Guis help speed up the verification part. Speaker 2 Massively, because vision is such a fast way for us to process information, much faster than reading and parsing text. Speaker 1 It also means we need to keep the AIS generated output manageable right? Getting a 10,000 line code change isn't helpful if it takes me days to review it. Speaker 2 Exactly. You're the bottleneck, so working in smaller incremental chunks is key. Generate a bit, Verify quickly, generate the next bit. Keep that loop spinning fast, iterate. Speaker 1 And being specific with prompts helps too. Speaker 2 Hugely important. Vague prompts often lead to output that fails verification. You waste time, end a little extra effort making your request recise, and you dramatically increase the chance the AI gives you something useful the first time, like an education. Just asking an LLM. Teach me Physics is too broad. It'll likely ramble. Better to structure it, create specific course modules, give it a syllabus, keep it focused on auditable steps. Speaker 1 Carpathy's Autopilot experience seems relevant again. Here he mentioned the GUI showing the AI's vision and the user having that autonomy control. Speaker 2 Yeah, and he talked about that impressive 30 minute self driving demo back in 2013. It made him think full autonomy was just around the corner. Speaker 1 But here we are, over a decade later. Progress is huge, but fully driverless still often needs remote human oversight teleoperation for tricky spots. Speaker 2 Right, which makes you pause when people declare, say, 2025 the year of agents. Carpathy leans more towards the decade of agents. It takes time, especially for complex safety critical tasks. You need humans carefully involved. It's still software, and software have bugs and edge cases. Speaker 1 Which brings us to the Iron Man suit analogy. The Iron Man suit analogy I like this one. Speaker 2 Me too. The suit augments Tony Stark, letting him do incredible things, but he's still driving it. It can also act autonomously sometimes. Carpathy's point is right now, given LLM fallibility, we should focus more on building the Iron Man suits, powerful tools that augment humans. Speaker 1 Rather than jumping straight to fully autonomous robots that might look flashy and demos but aren't reliable enough yet. Speaker 2 Exactly, build products with great G wise smart UI UX that speeds up that human generation. Verification loop are for the autonomy slider sure, but focus on making humans faster and more capable with the AI. Speaker 1 OK, that makes a lot of sense for building practical tools, but there's another huge piece to this. Vibe coding LLMS are programmed in English. Speaker 2 Yeah, natural interface. Which means, kind of amazingly, suddenly almost everyone is a programmer. Speaker 1 That sounds incredibly disruptive and exciting. Used to take years to get good at coding, Not necessarily anymore. Speaker 2 Seems like it. Have you heard people talking about Vibe coding? Speaker 1 Yes, it's become a huge meme, right? That feeling of just describing what you want and conjuring software with natural language. Speaker 2 Exactly. Karpathy mentioned Tom Wolfe from Hugging Face showing kids vibe coding. He called it a gateway drug to software development. Karpathy seems pretty optimistic about what this generation could do with these tools. Speaker 1 Did Karpathy try it himself? Speaker 2 He did. He actually built a basic iOS app without knowing Swift, just by prompting an LLM and another little web app called Menugenin, which generates images for restaurant menu items. Speaker 1 He just vibe coded. Speaker 2 Them pretty much, yeah. He got the basic iOS app running in a day menu. Ginen, the image generator part was also quick. Speaker 1 But was there a catch? Speaker 2 Yes. What was fascinating with Menu Ginen was that the code generation was the easy bit for the LLM. The hard part was everything else needed to make it real. Like what? Like setting up user authentication, integrating payments, registering the domain name, deploying the app, all the dev OPS stuff. Speaker 1 Stuff that often involves clicking around websites following instructions. Speaker 2 Exactly. Human instructions like go to this URL, click this button, select from this drop down. That part took him another week. It wasn't code the LLM struggled with, it was interacting with systems designed for humans, clicking through Guis. It was like the computer was telling him to act like a robot. Speaker 1 OK, so that raises a huge question. Can we build things for the agents themselves? Can the LLM's eventually do that manual clicking work? Speaker 2 This is where we start thinking about agents as a new kind of digital information consumer. They're computers, sure, but maybe think of them as these people spirits, as Karpati puts it, the need to navigate our existing human centric digital world. Speaker 1 So how do we help them? Speaker 2 We can start building specifically for them. Just like websites have a robots dot TXT file to tell web crawlers like Googlebot how to behave, we could have LM dot TXT files, simple markdown text files describing a website or service in a way an LM can easily understand, much more reliably than trying to parse complex HTML meant for human eyes. Speaker 1 OK, like a simplified instruction manual for the AI? Speaker 2 Exactly. And think about documentation. So much of it is written for humans. Bullet points, bold text, screenshots. Services like for Cell and Stripe are already creating docs specifically for LLMS, often a markdown. They're even replacing instructions like click this button with the equivalent curl command and agent could execute directly. Speaker 1 That makes total sense. Carpathy tried this too, right? With an animation library. Speaker 2 Yeah Madam. He fed its extensive human written documentation into an LLM, describe the animation he wanted and boom, the LLM generated the script. It basically vibe coded the animation by understanding the docs. Speaker 1 So making existing documentation LLM readable could unlock tonnes of value from software that already exists. Speaker 2 Absolutely huge potential there, and tools are popping up to help. Like there's a service where you change github.com in a repo URL to get dash ingest.com and it pulls all the code into one big text file formatted for an LLM. Or even more advanced things like Deep Wiki, which uses an agent like Devon to analyse a whole GitHub repo and automatically generate documentation for an LLM. Speaker 1 Wow, so while LLMS might eventually learn to navigate sites themselves, it's slow and expensive right now. Speaker 2 Very. So meeting them halfway by making information easy for them to ingest is incredibly valuable, esecially for all the existing software out there. The long tail that won't get a fancy new agent friendly interface anytime soon. Speaker 1 OK, let's try and bring this all together. Key Takeaways What are the key takeaways for someone listening? Speaker 2 Well, first, we're definitely in this profound shift. Software isn't just software one point O anymore. We have two point O neural Nets and now 3 point O LLMS programmed in natural language. These paradigms are coexisting. Second, understand what LLMS are. These complex, almost 1960s style operating systems delivered like utilities but with his unique human like psychology, amazing powers coupled with serious flaws like hallucination and amnesia. Speaker 1 Right, and understanding that opens up real opportunities. Those partial autonomy apps seem key, using good G wise context management. And that autonomy slider. Plus the whole vibe coding phenomenon is democratising creation in a way we haven't seen before. But it also highlights the need to keep humans on the leash, supervising effectively. Speaker 2 And the value of deliberately designing things documentation, AP is even simple text files specifically for agents to consume easily and that. Speaker 1 Iron Man suit metaphor really sticks. Build tools that augment humans. Speed up that generation verification loop. Speaker 2 Rather than chasing fully autonomous demos right out of the gate, focus on making humans better with AI first. So yeah, the decade of Agents, as Carpathy puts it, seems like it's really here. But it's not about just flipping a switch to full autonomy. Instead, for you, the listener entering or working in this field, the challenge and opportunity is learning how to thoughtfully, carefully slide that autonomy slider, moving it from left to right, little by little. How do you design and build the next wave of software that truly leverages a IS power, but does it by intelligently integrating human oversight, human verification, human collaboration? That's the puzzle we all get to work on now, shaping how humans and AI work together in the future.