Stanford Professor Erik Brynjolfsson: Why Success Starts With Looking Like Failure

I'm sitting on the campus of Stanford, California right now. And I have to admit that there is something really special about this place. You can feel the weight of the volume of ideas and conversations that has really shaped how we think about technology and organization and our future. And in about 30 minutes, I'm heading into one of those conversations that I really feel will matter. I'm speaking with Eric Brinielson. He spent three decades by this point studying a question that I feel that almost every leader that I talk to right now is really struggling with. So when transformative technologies come along, why do some organizations really thrive through this and others stall? And he's research on both AI and productivity has become really essential reading for anyone that's trying to navigate what's happening right now. But there's something that I've been thinking about in the taxi over to the campus that I feel that is kind of the core tension of what I want to explore together with Eric today. It's easy to think about AI as technology and it's all system two in economists way of thinking. It's very logical. We approach it as how can we save costs or save fts and make things more efficient. But then on the other hand, we have organizations really struggling with the adoption side of things. Impressionally, I think it's because we're lacking kind of the vision of who do we want to become? What's the future that we actually want to build with this new technology? How can we make humans become truly exceptional and not just automated way? I look for that in the conversation and I hope, really hope that this will be one of those that I will remember for a long time. Cheers. Eric Brinnellson. That's so nice to meet you. Well, it's a pleasure meeting you and thank you for pronouncing my name correctly that doesn't happen as much in America. I'm a Swedish guy. It's probably a bit easier. Exactly. It's exciting for me to be here today. We're at Stanford campus, California. Here we are. Hard to see the Silicon Valley. I came here about five years ago and I've been just loving it as you can see the weather is almost always beautiful. The atmosphere of all the entrepreneurship and tech is just very invigorating for me. Before that, you spent almost 20 years at MIT. Over 20 years at MIT, I was a PhD student. I was a professor there. I love MIT. I love Cambridge and Harvard and all the amazingly smart people there. I like to go back when I can, but there's just no place in the world like Silicon Valley. That's really unique. So for those who don't know your work, what is it that you do and why does that feel important? Well, I'm a professor here at Stanford and I've been focused on what are the economic implications of information technology and especially artificial intelligence. I actually first started teaching a course on artificial intelligence. Just when I graduated from college in the late 1980s, built expert systems. If anybody remembers what they are, they do rule-based systems. Then we started a company, Todd Liffborough and I started a company called Foundation Technology. So I've been doing the built expert systems for companies. So we've been doing this for a long, long time. When I decided to get my PhD, I tried to do both AI and economics at the same time. There was nobody who could really help me with both those things. And so I had to choose, I chose economics, which was fun for me. I wanted to think about how it's changing the world. So that's what I've been doing ever since, thinking about the economic implications of AI. Of course, there's never been a time in history like the past few years where AI's just really taking off. I also have a company called Warkeelix. I've started a number of different companies and Warkeelix is the one I'm most excited about ever. And what Warkeelix does is it looks at the opportunities where AI can be applied in companies and helps identify a roadmap for doing that and then measure the benefits. Well, that's interesting. I'm sure the benefit is on top of everyone's mind. It feels like we're in an interesting intersection right now where Gen AI and Gen TK AI is kind of forcing almost like a moral question within leadership teams. What are we building towards? Are we building an automation play? We were trying to automate humans as much as we possibly can and it kind of forces a question of, "Okay, then what do we do?" Or are we trying to augment humans towards being more and more exceptional? I would imagine that you spend quite some time thinking about this. I do. That is such an important question. It's the heart of a lot of my research. A lot of people like I was inspired by the "Turring Test" when I was a kid, when I first heard about this idea, "Could you make AI that perfectly imitated a human so closely you couldn't tell which was which?" And I thought that was an amazing idea. Now I think more about it. I think it's a cool philosophical concept but it's exactly the wrong strategy from the perspective of economics or from business. In fact, there's a lot more value that can be created by augmenting humans, doing things that humans can't do and allowing humans to do new things they never could have done before. Then there is by imitating them and simply matching humans. I mean on one hand, with all due respect to my fellow humans, it's too low a ceiling to just match humans. And imagine if Henry Ford, when he was building a car, had said, "Oh, we're going to make a vehicle that can walk or run as fast as a human." It would have been kind of an unambitious goal. So we want to do new things that are better than have ever been done before and raised the ceiling. The other thing is, and his little subtler, is that if you simply imitate humans, then that tends to lower the value of human labor. Machines become a substitute for humans and that drives down wages and at least more of a concentration of wealth and power among the people who control the technology and the capital. And most of us would like to have a world not just of prosperity, but of widely shared prosperity. So for both those reasons, I've been pushing for people to think more about how we can use machines to complement or augment humans. I wrote a paper called "The Turing Trap" that lays out this argument in more detail. The basic y'all used is that the "Turing Test Mentality of Imitating Humans" can become a trap and we should really, whether you're a business person or a policy maker or just a technologist working with the technology, you should be looking to extend capabilities through augmenting humans. And what's the trap? Well, the trap is that if you make machines that imitate humans, then that tends to drive down the economic wages and economic stuff. Exactly. And with the loss of economic power, it comes a loss of political power. And then you're in this very bad equilibrium where people aren't happy with their outcome, but they have no economic or political power to change things. You see, as long as humans are valuable, then they have some bargaining power. They can say, "Hey, we don't want to go that way." And throughout all of history, there's always been this tension between workers and capital and citizens and central power. And one of the leverage points that citizens had is they could say, "Hey, we don't like this. We're going to step back." And that gave them some leverage, but if people have no economic power or leverage, then they could be trapped in a bad equilibrium forever. And who gets to make that choice? Because it's interesting. It's happening so fast. Also, it's happening across every organization right now. And there's, it seems like a smoother or at least a faster path that works better in a quarter-economics logic of going to automation path, where the augmentation, it's so much about human change, company culture, seems also more difficult to prove in the short term, I would imagine. But it definitely is harder to measure the benefits. It requires more creativity. Ultimately, I think it's a lot more rewarding. So I talk to a lot of managers, and I think that, you know, especially CFOs, or just looking at dollars and numbers, there's a temptation, a gravitational pull towards cost-cutting. How many labor hours can we save? And I have to say, that can be very profitable at times. But it doesn't give you much sustainable advantage. The real benefit comes from doing new things and raising the ceiling, having better customer service, new products and services, higher quality. That requires a little bit more creativity as you're suggesting, but ultimately it's more sustainable. So I do encourage the CEOs and managers that I meet with to try to resist the easy temptation of just cutting costs and be more creative. I think that with all new technologies, the real value almost always comes from doing these new things, from reinventing the organization, with electricity, there was a reinvention of factories, with steam engineers, reinvention of the whole economy. And with AI, you can get some wins by cost-cutting. But if a manager is creative and creates new products, services, ways of working, they're going to get more benefit. That said, I do understand that it's more natural to just look at replacing tasks. It doesn't take a lot of creativity to look at the things you're already doing. I think, "Okay, how could a machine do that or that?" It takes more creativity to imagine entirely new things. But when I tell people this story, I gave a talk at the I clear one of the big AI conferences. And I was encouraging that the AI researchers there to think more broadly about augmenting and complementarity, not just imitation. And so many of them came up to me after my talk and they said, the delivery said, "Oh, we didn't realize all the benefits of augmentation. Thank you for telling us. We've been focusing on the wrong things. But now that you gave us a new target, we can do that. This is something we can do." And I think they're right that they're just been remarkably good, that when someone puts a target out there, that they focus on that and they succeed. You see all these benchmarks being saturated very quickly. And what we need is new benchmarks that are more focused on augmenting not just automating work. - Yeah, it's interesting. And we also have in kind of the private sphere, we struggle to implement the already existing technology in a sense. There's a lot of, I don't know, fear perhaps. Are we automating ourselves away? And that's creating a kind of silent pushback in terms of engagement and so forth. And I'm feeling I'm lacking a lot of the kind of positive story about what do my work looks like as an augment in the video. - That's right. Yeah, I think that another benefit of the augmentation approach is it's a lot easier to get by and from the work. - Yeah, of course. - If they see this as a tool that's going to replace them, of course they're going to resist it. But if they see it as a tool that allows them to do their jobs better and allows them to do new things, that's awesome. I did this paper with DE Yang, EGIA and several other researchers. And what we did was we interviewed thousands of workers. We actually had a voice agent interview them that helped us with the interviews and asked them what kinds of tasks in their job would they like help with, where would they have automated. And it was very striking how the paper's called the Future of Work with AI agents, by the way if anybody wants to look at it. It was striking how workers, they do want agents to help them with a lot of their tasks. And there's others that they would like to have agents just automate and replace. But the current technology doesn't match up very well with their desires. If you actually take the time to talk to workers, I think you're much more likely to get acceptance and actually people welcoming it. And if you want, if you're a CEO, you want your workforce to be buying into this. And so I encourage them to be more conscious about taking the worker's voice into account. When I did the research going into this interview, I stumbled across your J curve. Can you explain what that is? - Yeah, well, like most people, I'm blown away by the core AI capabilities. At the same time, I'm disappointed that we're not seeing real business value. The official productivity numbers aren't really picking up. And this is a pattern actually see over and over with earlier technologies. I mentioned electricity earlier. It took about 20 to 30 years before you saw a big gain in productivity from electricity. And when I studied this as a graduate student, what we found was that the first factories electrified, they simply pasted electricity onto the old ways of working. They didn't really rethink things. And it didn't lead to much of a productivity gain or really any at all, according to the official records. It was only after they kind of reinvented the factory instead of having all clustered around a central power source. They had each piece of a machinery had its own separate electric motor, which allowed them to lay it out over an acre or more and have the equipment laid out based on the flow of materials as opposed to who needed power. When they did that new layout, they had a doubling of productivity, even a tripling of productivity in some of the factories. So the lesson there is not that electricity was a dud or it was overhyped, the lesson is that electricity, I itself, didn't give you the big benefit. Now coming to your question about that J curve, during that period when they were trying to figure out better ways of using it, they were inventing new business processes, they were re-skilling the workforce. There wasn't much of an increase in output, but there's a big effort and time of management, even consulting work. That showed up as more inputs, no increase in output, in other words, lower productivity. Later, once they had it figured out, then that productivity took off. If you map it, it looks like a J. At first, it's kind of flat or down even, and then it takes off. We saw this pattern over and over with different technologies. In this paper, the productivity J curve, we mapped out formally and mathematically why you would expect this pattern. To bring it back to AI, right now, I think we're in the early stages or actually the turning point of that J curve. I think we're just beginning to have some take off now. After a little bit of a low. How do you know if you're on that path versus that you're kind of just floundering around doing bad experimentation, bad implementation? Yeah. Well, in the practical world, both things are happening. I've certainly seen some companies that are floundering around, not using it effectively. I've seen others that are doing genuine reinvention. I do feel like we're doing things a lot better and faster today in 2025 than happened with electricity or with the steam engine, some of the earlier technologies. Part of it is the technology itself is just easier to implement. It doesn't require as much special equipment. You can run the chat GPT or cloud or Gemini on an iPhone and you can roll it out very quickly. You don't need a lot of special training. So that helps a lot. The other thing is just managers are much more scientific, I think, and careful about implementing things. They've got consulting companies that help them out. They've got business schools. They're just being much more disciplined about it back in the late 1700s or the late 1700s. There wasn't as much of a science of management. So both those reasons were speeding things up. Yeah. How would you measure the actual quality that you get out of AI? If we assume that just having the CFO numbers game is so easy to gravitate down towards, we eliminate hours. But that says as you were talking about earlier. Yeah. I think in Genie, you want to have a dashboard of suite of different KPIs. To be concrete, one of the studies we did that just came out earlier this year in the quarterly Journal of Economics was called Genitive AI at work. And we did a really in-depth study of the rollout of Gen AI and a call center, a contact center. And we measured about a dozen different KPIs. We measured an average handle time of the call. We measured customer satisfaction at promoter scores. We measured customer sentiment. That is if you looked at all of the transcripts of the discussions, how many happy words, and how many angry words we looked at, employee turnover, et cetera. And you could see these different metrics. Some of them improved a lot. Some of them improved a little. Actually, I was surprised. I'm almost all of them improved significantly. I've done a lot of these studies since I was a student. And I hadn't seen any case where you had so many metrics improved so rapidly across the board. Just within four or five months, you had double digit productivity gains on almost all these metrics. And also these different groups, the productivity, the stockholders were doing better in terms of more efficient company. The customers were happier. It wasn't coming in the expense of customer satisfaction. Actually, they had higher customer sentiment. And even the employees were happier. It wasn't like this was an electronic sweat shot that was just squeezing the workers. Employee turnover actually went down. So this was a case where there's a real win-win, but you need to do these multiple metrics. The other thing that's important is you need to do causal estimates, not just correlations. So in economics, there's been over the past 10 years or so, there's been something that's called the credibility revolution. And the idea is to take the, we've all had the saying, correlation is not causality. But most of economics for the previous hundred years was just looking at correlations. Most of business was looking at correlations. The credibility revolution helps show when you can actually get causal estimates out of it. A controlled experiment, a randomized controlled treatment study, that can give you causal estimates. That's kind of the goal standard. Unfortunately, business, it hasn't really caught on business. And one of my missions is to bring the credibility revolution to business. My company, Workheelux, helps companies do that kind of careful causal estimate mission. I can give you an example of a company that didn't, I think it means just to make it a little more concrete. We visited one company and they were very happy that using an AI tool, and they found that the workers using the AI tool were about 41% more productive on several different measures than the people not using the AI tool. And they thought this was a big win. But then we looked at the data a little bit more closely, and we saw that the workers who were using the AI tool were actually more productive even before they got the tool. So they had already been more productive. This was not like a random sample of people, it was what we call selection buyers. You're most forward looking workers, were the ones who first adopted the technology. So when we went back and did things more carefully, we controlled for it. Luckily, it was still productive, but it was only 11%, not 41%. So most of the effect was due to the selection, but some of it was due to the tool in fairness. But that's the kind of thing you have to do carefully to really understand where the benefits are. And by the way, theoretically, in some cases, it's even possible for that number to be reversed. You could actually have it, the tool have a negative effect in reality. [BLANK_AUDIO] It looked like it's positive if the selection bias is high enough. There's all sorts of errors like that that you have to be careful about. I've always been interested in how come we're seeing in relative terms so low productivity gains, if you're talking about 10, 12, 15%, it seems like the potential in the already existing technology is quite massively larger than 11%. Well, it is sort of a mindset difference for economists if we do one or two percent. We get very excited. I have a bet with Bob Gordon. The Congressional Budget Office in the United States has been predicting about 1.4% productivity growth per year for the rest of the decade. I made a bet with Bob Gordon. There's going to be 1.8%. He thought that was like, "Oh my God, that's so high. I'll take you to the side of the bet." That's 4.10% of a percent. I think actually we're already on track. It's going to be more like double, more like 3%. It is a matter of calibrating what you think is a big deal. There are certainly some particular implementations that have 30, 40, 50% or 100% productivity gains in their own things. When you go across the entire organization, getting 15% productivity is actually pretty good. Of course, hopefully you can do that year after year and continue to compound. I'm compared to most economists. I'm fairly bullish saying productivity could double to 3% per year, but a lot of economists think it's much less. I do notice the difference when I talk to technologists versus economists or when I talk to people in the Bay Area in Silicon Valley versus Peace on the East Coast or in Europe. If you've been listening to this podcast for some time, you know what I think. I think about AI. I'm deeply excited about it, but I also see companies struggle with it. You know that I think it's not primarily due to technical reasons. Compared to previous revolutions, let's call it the internet wave. Part of why we struggled initially was that the infrastructure wasn't built out. We didn't have online credit card payments and whatnot. But the difference with AI is that the infrastructure is already here. The technology is so much more capable than we actually use it right now. The problem for most C-suite's I talk to, CEOs and senior managers is that they actually know about the importance of AI. But they can't seem to get to that position where they truly understand how is AI making me fundamentally more competitive. To stand there, they can say costs hours here and there increased some productivity number. But how do I distinguish myself on my market? The ones that truly survived the digitalization era wasn't a bookstore that got a website. It was the Amazon's of the world. So we've truly find the transformation of what we do today and how can we unlock things, our customers that we never could do innovations that weren't ever before practical, that type of innovation. The primary problem that I see in all honesty is that the C-suite, you don't really understand the technology and not understanding the technology for the sake of understanding the technology, but for the sake of having a strategic discussion. So that's why I started Braille. I stand right smack dab in the middle, understand the technology, I understand strategy and execution. Bringing these two together is what we do at Braille. So if you understand that this is something that needs to happen, then we should talk. Thing is slots for this spring is already filling up. Braille has been in the works for some time, so I've already had engagement for the spring. So what I'm looking for is that one CEO in each market that truly wants to make a dent in the universe. So if you're that guy or that girl, please head over to Grail.works and let's talk. Are you optimistic for the kind of future of jobs? Is it a show, sort of kind of bet on where we actually will end up? Yeah. Well, I'm very optimistic about productivity, at least compared to most economists. I'm optimistic about a lot of wealth creation. I'm optimistic about the potential of AI to really transform the economy. Worried about jobs. I think that there's scenarios where it becomes very troubling. I did a paper recently where we found some significant negative effects on jobs. I would say that a lot of this is going to depend on our choices that we can design systems that are more likely to augment and lead to job growth. We can also design systems that lead to a lot of job loss and falling wages. So this is something I'm pretty worried about and there's no guarantee that the next five or 10 years are going to be good for most workers. It's something we're going to have to work on. What are the most important of those choices? Well, one of them is augmenting versus automating. We already talked about it. And let me just show you how the data turns out. So we did this paper, we called it Canaries in the Coal Mine, the early warning. We looked at what was happening to different job categories. What we found was that overall the labor force was noisy and having ups and downs. But if you zoomed in on certain categories, we found falling employment for early career workers, especially in highly exposed occupations like coding and call centers. We found rising employment in the occupations that were not exposed like home health aids. But most interestingly, if you divided the workers by how they were using AI, where they're using it to automate tasks versus to augment or complement tasks, you got very different trajectories. The people who are using it to automate tasks saw falling employment. The people who were using it to augment work saw growing employment and growing productivity. So that's an example of a choice augment versus automate that leads to different outcomes. It's interesting. We got a couple of brain studies coming out of over this summer as well. I think it's kind of a similar thing happening there. Are you choosing to completely delegate your thinking to AI or are you using it as a central part of the kind of creative process? It's the same choice, really. It is the same choice. And I think the outcomes are a lot better when you're human stays involved. The other thing I have to say is that to some extent, although I love, you know, agente AI and the power of AI, in some ways, it's an oversold as being able to do everything. And what we found was that in many cases, AI is able to do a lot, but there are some things that really are best left to humans. Is that a question of AI in the fall of 2025? Meaning that, well, the technology is kind of so rapidly evolved, so that's just a one-year statement or is it always going to be true to you think? Well, I wouldn't say either that it's a one-year statement or that it's always going to be true. Somewhere in between. I do think that over time, AI will be becoming more powerful and be able to do more and more tasks. It's going to take a lot more than one year for it to be able to do the full set of tasks. The humans are going to be able to do. I don't know whether it's a decade or more. At the same time, it is the reality that right now when companies roll these systems out, most of the times they're going to have more robustness by keeping humans in the loop or humans on the loop. AI is great, especially machine learning when you have lots of training data. And so lots of examples. But if it's something that hasn't appeared in the training data before, it's an exception, machines have a lot of trouble. I have to say humans aren't so great either, but we're better than machines. Improvising is one of our superpowers. We're able to do things that just figure things out on the flag. And in the real world, you end up seeing a lot of these long tail, one-off, exceptional tasks. And that's where humans can be stronger. So there's kind of a division of labor there. Do you see an in-eumental model of kind of the augmentation, human plus machine equals what we really want to get to? Do you see that there are, besides handling the exceptions? Are there other things that humans should focus on? If we almost give career advice to people listening, what are the skills that I currently possess that I should kind of double down on because they're future proof as a consequence of the technology? Yeah. I can't promise anything that it's completely future proof. And I wrote a book with Andy McAfee called The Second Machine Age. And we described some of the skills. And I think, mostly I feel like it held up reasonably well, but at the same time, things have changed since we wrote that book. And there's this constant evolution. So I don't think anyone can write down a list that will be true forever. But I can tell you, circa 2025, some of the things that I'm seeing. One way to think about it is if you divide, most tasks can be divided into three parts. So if you're defining the problem, asking the right questions, there's executing on that to answer that question or address that need. And then there's verifying, checking is this what you really want? How do we need to modify it? Humans can do all three of those parts. And we've been doing them since we've been humans. But machines are getting very good at the middle one. Yeah, like well defined, they can execute it. And so they're blowing away some of these tasks. And that suggests that humans are going to disproportionately have a comparative advantage in the first. and the third. So we need to be better at asking the right questions to find the problem. You know, speaking of second machine age, it was a quote in that book that I like a lot. It was from Pablo Picasso and he was shown an early computer and he looked at it for a while and then he said, "Huh, you know, that's not very interesting. All it does is it gives you answers." And you know, there's some truth to that that really the most important thing is asking the right questions and that's becoming even more important now. It's not just prompt engineering, but it's just more, you know, in the grander scale, what is it that we want to aim this incredibly powerful technology at? And then on the other side, we all know that machines, you know, they still hallucinate and they sometimes you misalign, they aren't doing exactly what you meant. Maybe they're doing what you literally asked for, but then you realize that's not really what I wanted. And so there's, it becomes kind of an iterative cycle where after you get the output, you verify it, you iterate. I do this all the time when I use work with, you know, chat GPT or cloud or Gemini. I will, you know, have it answer a question. I'll raise that. It's not quite it. I'll go back. Sometimes I'll notice that it's made a mistake, you know, that I know some of the economic literature. I was recently talking to it and gave me a citation that was exactly what I was looking for. And then it was from Eric Pernielson. I said, "Are you sure that Eric Pernielson wrote that?" Well, okay, no. It didn't, Eric didn't write it because I knew I hadn't written it. That's the kind of thing that he would have written. I was like, yeah, it is the kind of thing I wouldn't written, but I haven't written it yet. So it was good though. So I'll have to next go. I know. I know. But you have to be familiar with the topic area to be able to push back sometimes. But that kind of give and take, I find that you can get a lot further than you could by just throwing it over the wall and saying, "Okay, AI, you do it." And many, many problems are like that. And I think these are skills that can be learned. We can have students, kids, adults learn how to get better at asking the right questions, how to be creative, how to be careful about verifying things, and working in teams. Those are some of the things. There are other areas just to throw in a few other skills. I think a lot of interpersonal interactions, a lot of people prefer to interact with other humans. I think the whole entertainment economy where we're watching sports or chess with other real humans is something that people will have a preference for. So there are aspects of the economy that I think we'll see growing over time. They really require the human touch. But I guess my last piece of advice on that is we just have to be very nimble and keep updating because it's constantly changing. The era where you could learn some skills when you're 18 years old and then just do the same thing for the next 40 years, that ear is gone and you don't have to be much more nimble. I think it's so fascinating how fundamentally a question of technology is at least for the people that I speak to, that I respect and know they know their things around AI. Dom has always comes down to a question of the essence of humanity. And I think it's fascinating that kind of a technological question drives that discussion. So it is really interesting. Philosophers for a long time have been debating what is the essence of humanity, what is a good life, what is our ultimate goal. And it used to be just sort of a philosophical discussion, no insult intended, but something that philosophers would talk to each other. Now it's really becoming a very practical question that you go to the AI labs and they have people working on alignment research and it becomes really relevant. What is the goal that we're trying to get these machines to do if we had something very capable of carrying out executing our desires. We better be clear about what those desires are, what our values are. Yeah, I think it went around on LinkedIn a year ago, but there was something along the lines of I expected when the robot center to not having to do the dishes and clean up, but to be fully focused on my creative side. But what ended up happening is that my creative side got automated, but I still do the dishes. Exactly. That is, I did see that. It is ironic that sometimes it's not mash up. And that paper we wrote, "The future of work with AI agents highlights that." We had kind of a two by two there. What are the things that AI is good at? And what are the things that people want them to do? Yeah, exactly. And you could have high low on each dimension. And so there's sort of four quadrants. And the sad thing was that the data was almost evenly distributed through all four quadrants. There were certainly examples of things that AI was good as what we wanted it to do, but there are also examples of AI doing things that we didn't want it to do. And people wanting AI to do things that it wasn't yet good at. So we're hoping that that research will help align people as they develop AI to focus it more on the things we really care about, but not the things we don't. I would imagine that you have plenty of discussions with the really high impact people, people with power. How do they feel? Do they reflect a lot around the responsibility for this question that we're currently in? Do we go the automated or augment path? And what's my role in building a future that we all want to inhabit? I think they do. When you're talking about them informally, a lot of them do care about these things. At the same time, I think there's some economic pressures that drive them in certain directions. And sometimes I feel like their actions don't match up with their words or maybe with their own desires. I think I've got the sense none of them have said this to me explicitly, but that they feel a little bit trapped in a race to the bottom of prisoners dilemma where they would like it if everybody kind of agreed to work on some of the higher purposes, but they're being pulled in another direction. I mean, to give you actually one person did say something. You didn't say it this way, but I'll try and quote him accurately. Demis Hassabis at the Paris Action Summit. Everyone had a little five minute chance to give a five minute mini speech about what they thought was important. And it was striking to me that Demis, the head of Google DeepMind, Nobel Prize winner, one of the best researchers in the field, took his five minutes and used the whole time to advocate for a certain entity, the particle physics laboratory where everyone cooperated. And he was hoping that just as the particle physicists work together from all the different countries, all the different labs to try to understand the nature of the universe and the nature of particle physics, that something similar could be created for AI, that the frontier labs, the governments, the researchers could come together and coordinate on research. It seems like we're pretty far from that actually really happening, but it wasn't just Demis who said that when I talked to a lot of other researchers informally, I won't say their names, they expressed a similar desire and sentiment. So what do you think that it's unlikely to happen? Is it because a lot of the actual research is not done by state funded schools? It's more done by like open AI or stuff like that or is it? I don't think it's impossible. I mean, so I don't want to imply that there's never going to happen because a lot of these leading folks would like to see it happen. At the same time, there's a really strong economic incentive to literally hundreds of billions, maybe trillions of dollars at stake, if you can have a more powerfully AI system and you don't share it with other people, there's a lot of economic value in that. There's a lot of literal power in that, power to an individual, power to a government. The reality is that these technologies are generally dual use, meaning they have amazing civilian uses. They can make it wealthier, healthier, happier, cleaner environment. They also have military uses. Right now, I'm told that the majority of the deaths on the battlefield in Ukraine are from drones. Of course, their cyber technology. I'm very worried about biological viruses and other pathogens. So as you get more powerful technologies for these good uses, it's often not very hard to also apply them to some of the military dangerous uses and that leads to a geopolitical tension. And the modern bad guy is also a knowledge worker in a sense. Yeah. Yeah, exactly. With nuclear weapons, it's pretty hard to make a nuclear bomb. You have to have a lot of specialized materials, a very expensive equipment. With AI, we may be giving the capability for really vast levels of destruction to somebody with a laptop computer to make a new virus or something like that. That is a scary possibility. Yeah. How do you see your role in this? What do you want to be the force for? Well, my comparative advantage, I think, is in studying the economic side of it. I care a lot about some of the more catastrophic risks. And I support the people who are working on that. Yashu Abengio, I think he's now the most cited scientist in all of history, has devoted a lot of his life to that. And I just had dinner with him the other night and support what he's doing as are a lot of other top researchers. But for me, I'm focusing on the economic implications. Right now, what I see is our capabilities is just taking off. You've never seen such rapid progress to have really just immense power. And that's in part because we have so many resources going into it. Literally hundreds of billions of dollars. And I don't know how many IQ points. A lot of small smart people working on it, gravitating. At the same time, look at the economic side. It's barely moving. There's very few people, certainly not billions of dollars. So my mission is for me, myself, to study that and bring the people here at Stanford Digital Economy Lab to study it. We've got some amazing researchers who are working on it. I'm also trying to change the field two weeks ago, along with Ajay Agrawal and Anton Koroneck. I organized the first ever NBER workshop on the economics of Transformative AI. NBER is the National Bureau of Economic Research, it's kind of the premier gathering of economists on different topics. And it was a big milestone when they were willing to support us having a workshop on the economics of Transformative AI. Not just regular AI, you can call it Transformative AI. And we define that as AI that's powerful enough to transform the economy the way that the agro-cultural revolution became the industrial revolution. Or to be more concrete, we borrowed Dario Amode's definition of powerful AI. Dario is the founder of Anthropic and CEO. And he wrote a terrific paper last year called Machines of Loving Grace. And I gave him a little bit of advice as he wrote that. But he had a definition of powerful AI, which was it was like creating a country of geniuses on a data center. So you have all these geniuses, Einstein level, Nobel PhD level geniuses in physics, in chemistry, and management, marketing, logistics, economics, all these different topics. And you make millions of instances of each of them. And you have them each running 10 or 100 times faster than a human thing. Just imagine what that does for the economy. Dario thinks it's only a couple of years away. I'm not sure how long it will take, but I'm already seeing glimmers of it. And so our mission, our, sorry, our, I would call it our charge to all of the economists who were gathered here at Stanford. We had about 16 presentations was to imagine that we had this country of geniuses on a data center. What would that mean for whatever topic area you're working on? And then the following week we came out with a paper called the economics of transformative AI or research agenda, where we laid out a set of questions for nine big areas like economic growth and productivity, inequality, concentration of power, how you measure and what the meaning of well, you know, well being and welfare is catastrophic risk. And now people are working on all of those topics. I think it's fascinating how we as humans, we're kind of really bad at understanding those kind of exponential curves and it must be kind of thrilling to be part of the thought experiments of what could we like what are the problems that we could tackle. That's right. Yeah, we need to get start doing it in advance. We can't wait until we're there. Yeah. You know, Ernest Hemingway once described how people go bankrupt. He said slowly at first and then suddenly. Yeah. And that's the nature of the exponential curve. And we have to be careful that we don't fall into that trap where we things happen slowly at first and then suddenly before we're ready, there's complete transformation, big changes in unemployment, wealth, economic power. And that's why we want the researchers to work on it today. And we're sounding the alarm that they have to start paying attention to it. That gap between the capabilities and our understanding is just getting bigger and bigger. And we need to speed up the economics of it. So my mission is to help transform the economics profession, lead that charge towards understanding the economics of transformative AI. That's cool. That's a cool mission. What do you see in terms of one of the things that I worry about is the rate of change will probably mean that that sort of level of professionals will be quite all of a sudden out of jobs. And it's different types of people than are normally out of jobs, if you'd say. It's knowledge, workers, it's went to college and stuff like that. And what is the actual impact on our societies and what time scaling, what's going to happen after that? How do you see this playing out from like the macro perspective? Well, current AI certainly has a very different set of capabilities than the technology of 20 years ago or 50 years ago. And it's affecting different tasks and different skills. And you could see that, for instance, one of my early papers I did with Tom Mitchell was what can machine learning do and we kind of ranked all the tasks, 18,000 tasks in ONet as to what I think could be Daniel Rock also on that. And then later, Daniel Rock and a team at OpenAI did a similar analysis for LLMs in particular. And what you could see is exactly what you're saying that a lot of the effects were in more high paid jobs, professionals. So there's kind of an inversion where what we may see is that the doctors are increasing replacement, the nurses are still needed. And the lawyers and the accountants find that they have trouble adding value, but the carpenter and the plumber becomes more and more valuable. And that's partly because the cognitive tasks are having much more AI, having much more success at doing cognitive tasks than physical tasks. Now robotics may be coming along as well a few years later and then it's a broader effect. And so the initial effect will be some disruption of the labor market. Some jobs becoming more valuable, others jobs becoming less valuable. Over time it could spread across the entire labor market. And then we need to think about a new economic system that isn't so based on labor income, but based on other things, maybe universal basic income or Nicholas Bergruin has an idea of universal basic capital. These are some alternative approaches. That's. What do you see yourself doing in like 10, 15 years? Well I hope I can help us manage this transition because I mean the good news is we have amazing technology that allows us to change the world in ways we've never seen before. And because of these exponentials a lot of it is going to be hard to predict. So we're trying to have a dashboard that measures the changes in close to real time. And then I hope we can help guide it towards some of the good outcomes. If we do it right, I think the next 10 years could be the best decade in all of human history. But if we do it wrong, it could be literally the worst. It could be just terrible economically and on other dimensions. So I don't think we've ever seen such a sharp divide between the good outcomes and the bad outcomes that we have right now. In a way it really relates back to this fact that we have very powerful technologies and the more powerful these tools are, the more we can change the world. I mean almost by definition. And that means there's only so much you can do with a spear or a shovel. You can do more with a bulldozer or nuclear weapon. And you can do even more with artificial intelligence. And if we guide them correctly, we're going to have a lot of good outcomes. My mission, as I mentioned, is to understand economic implications of this. It starts with basically science. So I'm not as focused on the policy right now. I'm focused on the understanding the economic implications and what the leverage points are. And then from that should follow some policy implications. Do you have a kids? Yes, I've got four kids. How old are they? They're all in their 20s and early 30s. I'm thinking, so my kids are much younger. So happily, I don't have to answer this question just yet. What is the recommendation I would give a 15 year old in terms of? It's tough. It's really tough. I mean, look, the thing I think you should always tell your kids is, do things you really enjoy, be passionate about it. And I think it's even more true now that if you're just clocking in, doing something outside about, you're not going to be that good at it and you're not going to really make a difference. In order to make a difference, you have to be passionate about something. And in a world of AI, I think a lot of the returns are going to look more like a power law. That is, there's going to be a long tail of average or low performers. And there's going to be a few that really make a difference. And to be in that top of that tail, you have to pick some air in. There's many different areas you can do that in. You have to be something that you're passionate about and you can make a difference. I think, and this is not the deepest thought, but using AI is a great way to help explore ourselves. Yeah. People who work with AI are able to just accomplish so much more. I got an email from a friend of mine recently. And he said he had just the past two or three days. He had written 50,000 lines of code with a cloud code. And he could never have done that before. And he said, you know, I'm a true believer now. I see the potential of it. And it's not just writing code. There's so many things where you can be tremendously leveraged. So I think it's something we really need to encourage my kids to dive in and use the tools because it helps them fulfill their other interests. Yeah. It's interesting. At a period during this spring where I found that working with AI was kind of hypercharging my ability to go deep into different subjects quite quickly. But I was also context switching so much faster because I started deep research. I started to research agent and while that was doing something, I had to jump down into something else. And I realized it was quite. exhausting work in that way because my brain works best in in like longer periods of deep focus. I had to really think about while I charge fully ahead diving deep into these tools, I also need to really guard like the biological side of me and creating like biological. I agree. I think that these we're not used to working with these tools and they have some really awesome benefits but they also can have some pathologies and you have to be careful. It's like being in a candy store and there's sugar and fat and protein and whatever and if you're not careful you end up getting sucked into the things that are unhealthy or our brains aren't necessarily wired for a world where AI and social media and tug-ar attention lots of different directions. So you have to develop discipline. I mean you know, Danny Coniman talks about system one versus system two. I find that a very useful framework. Your system one is your quick instinct. You grab for the cookie. You go click on social media. You see something fun and chat GPT. It's satisfying in the moment but it doesn't really lead to deeper thinking. System two is more deliberative. Sometimes it's a little bit painful but ultimately it's more satisfying and I think that if you use AI correctly it can help you explore those system two opportunities. I had a fun conversation recently with banked homestrom. He's a Nobel Prize winner in economics and he's at MIT, friend of mine there, but he's coming to visit Stanford and we had to go walk around the campus and he told me that every morning before he even gets out of bed he sits down and he had a long conversation with chat GPT. I thought that was pretty fun. I thought maybe he was joking but then he showed me all of the conversations and he was asking chat GPT about how do quantum computers work and where are the implications for financial markets and all these exploring all these different thoughts going back and forth. If you're someone like banked homestrom this is just an amazing technology. He's already a Nobel Prize winner but I could see from his conversations he was diving deeper into all sorts of other topics. It allows people to live their best lives and become, have a better understanding of the world. Obviously it is being used also in lots of negative ways. People are using it to hijack attention, redirect to do social manipulation of other people in ways that aren't very healthy. It's an interesting time to be a curious individual and that's such a competitive advantage probably in the coming years. Absolutely. One of the things that we're seeing both for individuals and for organizations is that rather than being a great leveler it's often something that exaggerates differences. That's a good point. When it comes to companies we're seeing the top 10% of companies pulling away from everybody else, these superstar companies and using the technology more and more effectively taking over market share and industries. I think for individuals as well my son really loves playing with these tools. He used to play with Khan Academy and I remember one time when he was a little younger I said go and go once you go spend half an hour in Khan Academy and learn algebra here in high school. He did that and I came back to him two or three hours later and he was there studying about the Civil War and biology and how the cells work. He was just sucked in and absorbing all this information. At the same time there's other people who you know they just don't like learning and then if you don't have the structure of a classroom they can fall even further behind. Yeah an interesting conversation with a Swedish entrepreneur who's building a based Nick Bloom's 1984 studied 2 Sigma problem which was about private two drinks effect on student outcomes. Right. So two standard deviations improvements based on private two drinks. He's trying to build an AI company. Absolutely. That's a great example. I mean when people are in the classroom everyone has to kind of learn the same pace. If you have an individual tutor you can like you said be two Sigma better. It's not cost effective and for most people to hire an individual tutor not each person but with AI you can do that and Tom Mitchell and I have been playing around with that Tom Mitchell's a head of machine learning at Carnegie Mellon or it was and he's just doing some great work on these AI tutors. He showed me one you know of course it knows the material very well and it can instead of if somebody gets the answer wrong it figures out how they got it wrong and instead of you know saying oh that's wrong here's the right answer it will ask them a new question. That's a secretive method. This is a credit method to help them kind of see for themselves which of course is much more likely to last but the other thing you know this was an eye over for me was he could tune it to be incredibly entertaining. Okay. And so he had it you know talking in the style of a pirate you know whatever the person you know resonates best with the person and I was just laughing on the floor because it was so funny what he had with it what the AI was doing and I could see this being incredibly engaging. We've all had these teachers who are both good at conveying the materials but also just very fun and entertaining and so the hope is that everybody will have one of those customized to exactly their interest. Yeah that's so true and I think most people who end up being kind of enthralled by knowledge and I want to live in that world and have that experience at some part of their life there was somebody who was probably a teacher or a father or mother or something like that who really shaped their way of thinking. And I think actually if done right almost everyone can be reached in that way you know you've got kids and you know that when they're when they're two or three years old if you take a pile of blocks you put them in front of them on the floor what are they going to do yeah they're immediately going to start building something you don't have to say oh you have to build something I'll give you no a dollar if you build something you know it's just we're wired to enjoy building or you know a paper and pen they'll start drawing yeah people love creating things and then they go to school and they're told oh no you can't draw you can sit quietly do this and it's almost like stamping out creativity but back to what we're saying earlier about these three different stages defining the question executing evaluating it you know we can nurture that creativity it's already there and if we can nurture it more everybody can think more about being creative and then the a i's can can execute and they can evaluate oh that's what I wanted that's not what I wanted yeah you know the other day I wrote a poem for somebody they were retiring and I had a lot of like stories and I was kind of struggling to structure them all and so I gave my different stories and things into you know catchy BT yeah and it came up you know I had to iterate a few times to get it right sometimes it didn't totally understand what I had asked for but it was it was actually kind of fun and and the end product was was really funny yeah and it's really useful um but it was it was a good kind of partnership of taking a goal I had and some creativity and then executing it and iterating yeah I agree and it's so fun too if you split up like the content of what you're trying to say from the tone or what you're trying to say so first I focus on on getting the right content I write all my stories and whatever and then you quickly iterate between different tonality styles and what if you write a rhyme on it or what if this is a exact century poem or something like that I tried a few different yeah I can't do that so that's not quite right you know that's try to be a little funny or whatever and you know the process itself was kind of enjoyable for me too so that that was that was part of it as well now you have to run soon but what is the the kind of best possible outcome as the last question what the what does in 20 years time what do we what does humanity spend their time on well I do think we are in range of essentially eliminating extreme poverty yeah I mean with AI technology there should be no reason that anyone starving or goes without food clothing or other basic needs now of course there's always going to be extreme things you know I want to fly to Pluto or something you know you can't you can't satisfy everything but at least our basic needs the core of maslow's hierarchy I think you can get taken care of and it'll be interesting to see how people of all in terms of what they care about I suspect that status and status hierarchies will continue to be important I was talking to Reid Hoffman recently and he said that's you know he thinks that's an important part of how we're we're wired you know from from caveman days so be it hopefully we can channel that status into something productive you know how we help other people or you just have fun you know video games and not into something that's a negative some or destructive status game and so we'll have to think about if you call it an economy where we we channel that in in a productive way but we could have a much lighter footprint on the environment on the on the world we could have much better health probably much longer health spans maybe a lot of or most diseases could be cured have significant extension of life expectancy I think that's realistic in the 20-hour time frame like I have that type of massive breakthroughs in that you know I'm not sure I know that you know I talked to Demis his office about he's pretty optimistic about it you know maybe there's some selection bias there that he's chosen but he he did you know he had a postdoc at MIT and in cognitive science you know there's a lot of biology you got a Nobel Prize so I wouldn't completely discount it he may be on the optimistic end and even if we don't have you know massive increases I think that there could be a lot of things that we improve you know alpha fold and understanding protein folding is giving us some of the tools I don't think it's translated into a lot of practical benefits yet but but that will come I think so there's just some really promising outcomes, what's going to be tougher is sort of the geopolitics in this. psychology. These tools can also be used, you know, as I mentioned early in very destructive ways militarily, they can be used in ways for social manipulation. I think it was Sam Altman who said that super persuasive super intelligence and persuasion could come before super intelligence and sort of rationality. And that obviously there's people who would abuse that, you know, for marketing or for political or demagogue reasons. So this is going to be tough, tough to navigate. The reason I'm doing what I'm doing is I want to at least help on the economic side and set up an economic system. We will, I think, ultimately have to have a new economic system that is much less dependent on labor income and find other ways for people to benefit. I think it has to start with the premise that humans are ends, not means. And through most of history, this is what Kant said that, you know, you don't want to think of humans as just a means to an end, but an end in themselves. And so we need to think about, okay, we want, you know, to just have humans benefit just for the sake of being humans and not just because there are tools, some other goal. And that's a different philosophy than we have had through most of history. It can only work. And it's everything. That's super interesting. What are you really optimizing for? And we've been kind of optimizing for GDP growth, I would assume. Yeah. Yeah. And I don't think we need to optimize for GDP growth if we have machines that can create at least basic abundance. And some people, you know, one of my favorite books is Sitartha, and I read it, you know, some people made a side that, you know, enough is enough. I feel like I have most of the wealth that I need, you know, I'm lucky. And I, then most of my time pursuing, you know, knowledge and I find that more, more satisfying. I'm sure there will be other people who want to pursue other goals. So we'll have to, we'll have to come up with a society and economic system where people are comfortable with having, hopefully, a broad base of economic shared prosperity and then pursue other goals in ways that are constructive rather than destructive. But it's going to require like a real fundamental reinvention. And it's, to be fair, it's not the first time. And when we, the agricultural revolution changed us from a hunter-gatherer society, industrial revolution, 1.90% of people were basically subsistence farmers and, and now we do a lot better than that. So this will be a transition comparable to that. Fantastic. Thank you so much for taking the time. It's been an honor to get to visit you. Oh, such a pleasure. I really enjoyed the conversation with you. Those are great questions. And these are just very interesting topics, aren't they? Yeah, they are. And I wish you the best of luck. And I wish us collectively the best of luck. There are truly important questions right now that we haven't figured out. And that's partly super interesting, partly super frightening. Yeah. It's an amazing time to be alive. And it's fun to be one of the people in the middle of all this. Yeah, I can imagine. It's a good note to end on. And there, we're at the end of the episode. I think it's so amazing that almost 70% of you actually listen to the end of the episodes, especially for a long format. I'm so thankful for that. Like the engagement numbers is fantastic. Before we leave each other two things, first off, this episode was sponsored by Grail. And I'll make no secret about Grail is the company that I'm starting. So it's the future of AI. And how can we unlock value that's not just automation, but actually building towards a future that we want to live in like a future where I want to work augmentation. So head over to grail.works to know more. Secondly, I'd appreciate so much that you listen to this podcast. But even though 70% listen to the end, almost 70% are not subscribed either. So if you can give a like button, a subscribe to all my channels, we're on LinkedIn, we're on Instagram, we're on TikTok apparently. We have an awesome web page and there's so much more content. Like yes, the reels and so forth. But what I really would like to push for is that every week I publish a long format article, which is the pinnacle of my thinking. So it's the most interesting thing that I come across that week, either in podcasts or in other endeavors. And I try to write about it towards you, a senior leader. Like what is the crew shall take away from the most important part of my week, let's say. I published that on LinkedIn and on my web page, thinkroompodcast.com. Thank you so much and see you next time.