Math Vs Machine

I have to apologize for like dragging you away from Twitter or I guess I should call it you out there, have you been. It's been really interesting. My addiction is coming back. It's probably like a smoker who stops and then I shouldn't say that to you. Are you still smoking? I have my dalliance. It's like you, I guess. Sometimes something happens in the world and it draws me back in. So that is Cornell mathematician and longtime friend of the show Steve Stroghats. And the reason I had to drag him away from Twitter was because just a couple days before we talked. It's really been an interesting few days. OpenAI announced that they had solved a long-standing big deal math problem. From honest, I'm not even sure I knew exactly what that meant. But all the mathematicians I was seeing talking online, including Steve, there's almost like they were having an existential crisis. It's a big deal. It was a big deal. But why? Like why? Because we're on a trajectory somewhere and it's unclear yet whether it's a happy trajectory or a very sad trajectory. But we're going somewhere. I feel like you have a sad, I don't know what I think in my heart. Well, I feel like you have a sad, or sure, yes. I'm terrified. Now, Steve wasn't terrified about a lot of things we've all been talking about. I don't know, like a big hack or crashed economy, computers taking over the world or even really losing his job. Instead, as we talked, it became clear that Steve was reckoning with questions about his own sense of purpose in the world in a way that tried, as it might sound, touched on one of the most fundamental things about being human. So I think we should just talk about it. Yeah, I mean, however you want to start. Okay. So I would start the story in November of 2022. Okay. That's when Chatchy PT made itself known to the world. Were you used? Did you start using it? I did. I used it right away for gags, you know, write this. At first it was all about writing, write a poem in the style of Malcolm Gladwell about some crazy subject. Okay. So whatever, then naturally being a math person, I want to see is it any good at math? Yeah, that's going to say it. It's not trained to do math. It's just supposed to do language, but you could ask it little math problems and it was pretty pathetic. It couldn't do elementary school arithmetic. It would make mistakes. It was easy to fool it, but over the years, it went from being inept at arithmetic to then like it could do algebra kind of clumsily. And then at some point it was getting good scores on the international math Olympiad, which is the test for high school whiz kids. Is this all just number crunching like what a calculator could do? It could actually solve problems, not just number crunching. No, you could give it logic puzzles. You could give it what looks like a hard math problem for a high school student that involved reasoning. I see. This is the part that's mysterious. And I'm not even sure it's very well understood by anybody. Somehow by having read all the books in the Library of Congress and everything on the internet. Everything on the internet, including log posts, news articles. Somehow the language itself encodes enough logic and reasoning to do math. They hadn't built it to do math. And this is what we would call an emergent property. It emerged without being designed. And you were saying it's starting to get better at math slowly like a child? Did you say it like sort of noticing? It progressed through all the developmental stages. It got good at high school math. Then it got good. It's been longer and trigonometry than it expected to because one of the more painful subjects. No, actually you touch on a really interesting point, Soren, because what they're not good at and have not been good at until recently is anything that's visual. But the bigger point is that it keeps getting better. Like as of a few months ago, we were saying it can do research level math. But it can't that seems crazy fast. Like a year ago, you were worried about as worried as you would be about me taking your job. Zero. No, I want to get the chronology right. High school was like 2024. Okay. 2025. It's as good as a really smart college student. Okay. And then in 2026 in the spring, it's as good as a first rate mathematician, but not the world's best. That was in March or whatever. But by May, it was actually solving problems that the world's best mathematicians had not been able to solve. By May of 2026. What problem did it was at one particular problem? So the first the first one that really got everybody's attention in my world was called the unit distance problem. I mean, I could describe the problem if you want, but it gets pretty hard to keep in your head, but the gist is that like, well, imagine a bunch of pegs on a flat board. And for some number of pegs on the board, the question is like, what's the maximum number of pairs of those pegs you can have that are just one unit apart? That's the idea. Could if I if my job was to make as many pegs, pairs of pegs as possible. You can see how like if you're doing a making a computer chip, that seems like a useful question. Maybe it's not motivated by any practical thing. Oh, it was just a pure question about you could call it discrete geometry. Okay. So there was a guess about what the best possible arrangement was for putting down the pegs if your goal was to have as many as possible unit distance apart from each other. Now, for many decades, mathematicians had assumed the the best you could do was basically kind of like a simple grid that you couldn't really do better than that. But no one was sure. So what everybody was trying to do was one of two things. You could prove it proved that the grid was the best solution. You have to come up with a logical argument that shows it's why it's correct. Or you have to come up with a counter example, which in one stroke shows the conjecture is wrong, that there's a better solution or a better arrangement of pegs in this case. And in May, open AI, using one of its best models at the time, they found a better solution. And what was very interesting, and this is the part that was really made all of us mathematicians sit up straight like I'm doing right now in my chair. It was very creative. It was not just a number crunching solution. Everyone who looked at it, all the experts said this a really beautiful argument. I mean, they use words like that beautiful elegant. And not only was it very ingenious, they'd brought together two subjects that we didn't realize had a bridge connecting them. I mean, it sort of took from way over here and then took something you didn't think was connected at all. Like a different part, a different branch of mathematics like a different branch of math. Okay. We know of because these AI models are now using what they refer to as chain of thought. So a chain of thought is a way for us to see what it's thinking. I see. It doesn't just compute. It actually says, oh, I'm going to try this now. This seems promising. And it literally has output that writes in natural language. That's really right. You can read it like a dot. It writes it like that, including expletives. Like it will say, oh, shit, this looks really, oh, this is going to work. I mean, it uses exclamation points. It swears. It will say any, it's it acts. Well, you're half given it the whole internet. I mean, so that's the point. It's it's sort of knows what people would say in situations of great discovery or an epiphany, but we're not even we're not even all that we you don't care about the F. I'm not interested in that topic. All I can tell you, I mean, yes, I am, but let's not go there. Okay. But what I'm saying is that it taught us something. And it's sort of like when this guy named Dick Fosbury figured out a good way to do the high jump. The flop. You know the Fosbury flop. Yeah. Fosbury flop. Right. Just for those who don't in 1968, Dick Fosbury broke the high jump record. Now, until then people would run up to the high bar facing forward and go up like over their stomachs. But Dick Fosbury ran to the bar. It sort of turned with his back to it and sort of jumped arching over his back. He showed if you go over with your back going over, it looks dangerous. It looks idiot. It looks ridiculous. Nobody had tried it. But my point was that that's that taught other jumpers and you don't see anybody doing it the old school style. Now everybody does the Fosbury flop. Right. So in the same way, kind of having seen the machine's brilliant idea on this unit distance problem, human mathematicians were able to borrow and improve it and apply it to other problems. I see. If a person had come up with this, we would have called that person brilliant and we would publish that result in the best journal in math. So May of 2026 is when it became clear that there was now a world caliber mathematician that was not human. And that's I think that's unforgettable. But of course, we've had many months now since May. So throughout the summer from May until I'm speaking to you in September, one by one harder and harder and more central problems have been given. knocked off or solved or mowed down, or whatever you want to call it, one by one, all summer long. And it's having a traumatizing effect on a big subset of the math community. And that brings us to September 26, just two days before Steve and I were talking. When OpenAI announced that it had solved a very important problem in math, called the Navier Stokes problem, that is one of the seven Millennium problems. So the Millennium problems are a group of seven problems that some mathematicians got together in the year 2000 and sort of said we should set our sights on these problems. For us to aspire to solve in the 21st century and not just to solve, but to make the trek. These are very cherished. Mountain tops that we aspire to reach. And this one just got solved. OpenAI announced it two days ago as we speak. And can you tell me just a little bit about what the problem is? So Navier and Stokes were two scientists from the 1800s. They had come up with this enormously complex set of equations to describe the motion of fluids. How does water flow or how does air flow? How does any more generally anything that's fluid like honey, molasses, lava, blood, there's a lot of things in the world that flow. And the Navier Stokes equations are this simplified description of how things flow. And over the last 150 or so years, these equations have been very useful for us. So we use it for weather forecasting. We use it for thinking about ventilation systems in a room. But mathematicians are interested in these equations, kind of for their own sake. And the big question, the big problem was, could the Navier Stokes equations, as a purely mathematical device, could they predict something that we know is wrong? Like, are they fundamentally unsound in a certain way? Could you find a situation where they'll spit out something nonsense? Yes. Is there something fundamentally flawed about the equations? Can they generate nonsense? And this is what OpenAI claimed it did. It found a condition under which these equations start spitting out a bunch of infinities. The machine came up with a really tricky vortex. So it's like a tornado that's getting stronger and stronger at the same time that it's getting smaller and smaller. And it creates a kind of singularity where the equations break down at that point. So they'd answered the question and it involved, at least according to the early reports, 10,000 agents. So these agents, this is a whole new thing in artificial intelligence. 10,000. So they spawned 10,000 little individualish, little genius mathematicians. All right. And they're independent to some extent. They try different tricks on the problem. But they also can collude and collaborate. They can make other computers do stuff for them. Anyway, they spent, it's estimated somewhere between A, OpenAI, somewhere between 15 and 20 million dollars to solve a problem that had a reward on its head of one million dollars. Which you could see the economics is not so good. They weren't doing it for the money. They were doing it to say our AI can solve a problem that no human being can solve. And also anthropic didn't solve it, right? I mean, there's a whole big competition between OpenAI and Anthropic. But they built on ideas of two Spanish mathematicians who came very close to solving the problem themselves. Diego Cordoba and his student, Luis Martinez-Zeroa. So these guys figured out a brilliant strategy. And they were following it, but at human speed. They were making progress. They might have gotten there. They might have gotten there in five years. They might have taken five or ten years. And as possible, they would never have gotten there because the calculations are so monstrous that were required. Apparently it took OpenAI about four days to solve this problem. But if you take into account that there was 10,000 little agents working on it, that turns into roughly almost exactly a hundred years of working hours around the clock day in, day out. And also, I mean, there's a whole controversy and more of a story here because it looks like there should just quickly say, there aren't some questions about how much OpenAI used, maybe without admitting the work of Tristan Buckmaster and Levin Alpaca. Which I don't have any special visibility into any of that. But what I wanted to know from Steve was how he felt about the Navier Stokes solution. Okay. Was it like the unit direction where you also like, oh, how cool, how clever, how amazing, how delightful, how. Well, not yet. That has not been the community's reaction so far. Why not? Unlike the unit distance problem where the, the solution is, I would say transparent to an expert. This does not look like that. No one yet knows how the AI solved this problem. We have 160 page write up. But the math and the calculations are so incredibly complex and intricate. It's that hard to figure out that even the experts aren't sure what it did. It's this, it's this inhuman computation that is very opaque and very honestly to me, very off-putting. Steve says when it comes to like these, as he describes them, seven mountaintops of math, just solving them isn't necessarily the point. I mean, of course they want to solve them, but. Even if we can't get there, we're made better by the quest, you know, by trying and struggling and learning. So if someone or something does solve the problem for Steve, I want to know why. And he feels like this rapid progress in AI solving all these math problems. It's pointing in a certain direction. The key question and what is so disturbing to think about unsettling is let's imagine a time where the machines are so good at math that we can't learn from them because they're too advanced. They're so far beyond us that we can't understand what they're doing anymore. And like eventually you'll, like all you'll get is sort of yes, no, yes, no. Right, exactly. We're going to start dealing with oracles that are infallible, but opaque. They'll tell us the truth, but we won't know why. We're going to take a quick break, but when we come back, may I talk about Regina Barsalai? No, I'm always going to say yes. Steve makes the argument that this is not just an issue in math. And we talk about whether if AI starts solving our problems or answering our questions, does it even matter if we understand how? [Music] May I talk about Regina Barsalai now? Sure. I'm always going to say yes when you ask me questions like this, like Kenna. Yeah, go. All right, well, this is Ray Elab. I'm Sarn Wheeler talking to Steve Stroghets about what happens when AI starts answering questions or solving problems in ways that we aren't able to understand. And it so happens that Steve has been working on a book. It's called Big Math that explores some of these ideas. And for that book, he talked to a woman named Regina Barsalai. Well, Regina is currently a computer scientist at MIT. But when she was 43, she was diagnosed with breast cancer. And this came as a big shock to her. She had no symptoms. She had no risk factors that she knew of. Nobody in her family had ever had cancer, not breast cancer or any other cancer. But she had already had three mammograms. And on the third, at the one at age 43, the radiologist saw something that looked concerning. And it turned out it was bad. And she had to have two lumpectomies, chemotherapy and radiation. Her hair fell out. She had a seven-year-old. She was recently divorced. And she had immigrated. So she's an immigrant to the US. And she had no family. Nobody to support her, really. It was pretty bad. And she said, "If something going to happen to me, what's going to happen to my little son?" Okay. So fortunately, she did survive. And she's alive today. This all happened in 2014. But when she was going through these awful therapies, she talked to the doctors. She asked them specifically, "How do people like me tend to do at your hospital? How do women with my specific diagnosis, and mammograms that look like mine? What are my odds?" And they said, "We don't know." Because that's a complicated medical study that they hadn't done about- Because there's too many different- confounding factors about your age or your genetics or your diet or your. Right, we're just doing our best, like we don't know, we're just trying to help you. So she thought, well, look, I already had three mammograms. Maybe there was a clue back in my first mammogram that no radiologists could see. Maybe it was missed. And is it possible that the clues are there, but human radiologists can't see them. It's too difficult for the human perceptual system. I see. She ended up collecting about a quarter of a million mammograms from records at this hospital where the women had been followed up for years afterwards. So they knew who survived and who didn't. And if they died, how many years till they died? I mean, they had all the data, but it just had never been analyzed. And so she was able to train, again, this is around 2014. To use machine learning, she and her team, to build a prediction machine for assessing risk of breast cancer from one mammogram. And it's now the state of the art. It certainly was better than any method that was known at the time. And it has been tested in countries all around the world, hospitals all over. And it really works. But here's the uncanny part. If you ask her, what is the machine seeing in the mammogram that enables it to make such a good prediction, she doesn't know, then the machine doesn't tell her. So you can't, based on what she did, say to doctors, keep an eye out for this. No. You have to feed it to the machine. Right now, yes. When I heard this, I asked her, does it bother you that you can't understand how the system is doing this? And she said something that has stuck with me, I think, about it every day. She said, why would we limit the machine to what humans can understand? We have to save lives here. Why would we limit the machine to what humans can understand? I found that very haunting because, yes, if it's a matter of saving lives, I get your point. Of course, it would be morally perverse to limit yourself to what. You know, the fact that a radiologist doesn't understand it, we don't have time to wait for that. And medicine is often like that. But I come from this math culture where understanding is everything. If we don't have understanding, we feel like we haven't solved the problem. And this is a point I guess I want to emphasize. It's not just that understanding gives us a pleasurable aha feeling. Yes, we like that. But I'm actually an applied mathematician, not a pure mathematician, meaning I care about how math meets reality in bringing an astronaut to the moon and back safely in ultrasound for a pregnant mother. That's math. That's a lot of math in designing a good ultrasound machine to interpret all those waves bouncing around. We could go on and on about this. This is the story of the past 400 years. Math in combination with science has made the world modern. And it's just had innumerable benefits for our health, for our society, for everything. Steve's point is that what you learn in one field or one area, even if it's abstract mathematics, what you learn by trying and struggling or from seeing how someone else solved the problem, that can pay off in another domain. That's it. What we have learned over hundreds of years is that understanding is the key to power. And I guess the hope, and you feel like you're running into this sort of like answers without understanding problem in mathematics, you're running into that more and more. Yeah, we are, you know, like if we solve things too quickly, because here's where we are today. The AIs can solve essentially everything that we consider worth solving. They haven't solved our hardest problems yet, but it's a matter of time. I think we're on the edge of it. And that's why Steve and a lot of the mathematics community right now, they're worried and not necessarily just about their jobs, but about the whole field of mathematics. A three or four thousand year old discipline and culture. I mean, we will still have the ability to do, we can still do math for fun. It can become a game like people do Sudoku for fun, even if a computer could do every Sudoku. Is that what's in stake in store for us in math, that we're going to just play math for fun, but we'll never make any discovery that matters. You know, what about young people who love it? Can they be mathematicians in the future? We don't know. Maybe we'll under. Well, it's unclear. We might be. I mean, I would want to argue against myself just for a minute. We might get a golden age of understanding. It's possible that the computers will teach us a lot. They'll be the best. not only the best mathematicians, but the best teachers. And so for those of us who love the subject, we might be entering the happiest time ever in history. Like our problems are getting solved. There's magnificent methods. Just like listening to Pavarotti singing the opera, like nobody has done math better than these fucking machines. And they're singing better. You're undercutting yourself with the F-bomb. Well, you see, I'm all conflicted. I'm very conflicted. So I've actually talked to Steve a couple different times since that first conversation. In the days that followed a week later, and he really did go back and forth a lot about what this is going to mean for math. But it was clear that he couldn't totally shake the feeling that he himself was losing something and being sidelined in this sort of a way. And if we're going to end up like the people in Wally, in the movie Wally, just sitting on our shays lounge plugged in with our big gulp and our credit card, like what is there. It does feel like being a human being in the world involves some level of currently I'm trying to understand my children. And for me, of course, you know, it's fine if math remains inscrutable. And I mean, there's so many things in the world that I don't understand anyway. And also, not just for you. I think most of us live our life not understanding a lot of stuff. I don't understand how my toaster works, but I don't need to, and I don't really care. I just want my bagel. Right. So. But I guess I just. I don't think that we should let go of understanding lightly. Yeah. Intuitively, it seems like it's one of the things that makes us human, that being able to understand is helpful. It feels fundamental. Yeah. It does feel very fundamental. I really, unfortunately, I think that we're getting now a little bit outside of our scope to the question of what's the purpose of a human life. That doesn't ever outside of our scope. Right. This is radio now. Actually, the only thing we're ever talking about. That's the only question. I kind of wonder, you know, I feel like that's what we're talking about. What we're talking about here is what are we doing? What do we want out of our life? We want to love our kids. We want to be, you know, have a meaningful time. We get to be a little blip of atoms, you know, organized into consciousness for maybe 80 years or 100 years, if we're lucky, and then we're dust. So what are we going to do with ourself? And for me, part of the answer is I love being curious. I love learning, and I'm not going to get to do it. In the same way, if this future comes while I'm still alive. But look, I don't want to dismiss the possibility that these will always be tools. If we can make it so that we're in charge, we can tell them what to do. We can unplug them as needed. They're like brain prostheses. So this year, 2026 will be remembered, either as a miracle year, an anus merabolus or a horrible year, anus, horribleus for the history of mathematics. And I don't know yet which one it's going to be. All right, I'll let you end on that. Okay, God. This episode was reported and produced by me, and also my producer, Maria Paz Gutierrez. Big shout out to her for putting me onto this whole thing, talking to so many different mathematicians, knowing what was going on. It just wouldn't have happened without her. It was also edited by Pat Walters, Jeremy Bloom did our dialogue, mixing, and our fact checker for this one was Diane Kelly. Also Steve's new book, which hits on many of these issues, rather presciently, is called Big Math. He wrote it together with his colleague, another mathematician at Cornell, Alex Townsend, is set to be released on November 10th, but you can pre-order it wherever you do your pre-ordering. Now if you happen to actually like hearing a little bit of math, or me and Steve talking about math, or whatever, we actually had an episode from a couple of years ago called The Middle of Everything Everywhere, in which Steve and I try to figure out in real-time what the average sized thing in the universe is. It's a fun one, go check it out if you like. Until then, this is Ray Alab, thanks for listening, and we'll see you next week. Hi, I'm Maya and I'm from London and here are the staff credits. Radio Lab is hosted by Lulu Miller and Latvnasa. Soran Wheeler is our executive editor. Sarah Sandbach is our executive director. Our managing editor is Pat Walters. Dylan Keefe is our director of sound design. Our staff includes Jeremy Bloom, W. Harry Fortuna, David Gable, Maria Paz Gutierrez, Sindu Nynas Ambandan, Mack Keelty, Mona Mordgaulka, Alex Neeson, Sarah Curry, Natalia Ramirez, Joanna Stroghats, Anissa Vietzer, Ariane Wack, Molly Webster and Jessica Young. With help from Maya Applebee Malamid, Laura Carfus and Olivia Rose Greco. Our fact checkers are Nora Belblidia, Hilary Elkins, Rachel Gross, Diane Kelly, Emily Krieger, Natalie Middleton and Sophie Samay. Hey, Radio Lab, Michael Komo Washington, leadership support for Radio Lab Science programming is provided by the Simon's Foundation and the John Templeton Foundation. Foundational support for Radio Lab was provided by the Alfred P Sloan Foundation.