Go back

Gary Marcus: Is AI mostly hype?

37m 17s

Gary Marcus: Is AI mostly hype?

In this conversation, cognitive scientist Gary Marcus discusses the risks and current state of artificial intelligence. While he worries about powerful figures developing AI recklessly and the potential for significant harm, he considers human extinction unlikely. He is highly critical of large language models (LLMs) like ChatGPT, arguing they merely approximate language statistics without genuine understanding, leading to unreliability, hallucinations, and an inability to reason abstractly. He contrasts this with human cognition, which combines intuitive "System 1" thinking with deliberative "System 2" reasoning. Marcus advocates for a "Neuro-symbolic AI" approach, merging neural networks with symbolic AI to create systems capable of abstraction and reliable reasoning, and emphasizes the need for AI to develop internal "world models" to understand context and reality. He believes the massive investment in generative AI is an economic bubble destined to burst due to the technology's inherent flaws, high operational costs, and failure to meet exaggerated promises, as exemplified by the disappointment surrounding ChatGPT-5. Despite his skepticism, he acknowledges positive trends, such as valuable scientific tools like AlphaFold and a growing shift in the AI community toward more substantive, research-focused development.

Transcription

6798 Words, 37038 Characters

English
I'm Alan Olga and this is Clear and Vivid. Conversations about connecting and communicating. I will be frank that I worry about a guy who owns a company that builds both rockets and reckless AI and like what he could do to us. Maybe I won't name him but you know who I'm talking about. So there are some individuals in a great deal of power who are reckless with respect to AI that could cause some pretty bad things. I think there are very real risks. I don't happen to think that extinction is one of them. I think it's very unlikely that anything will literally extinguish the very resourceful and distributed human species. But the probability that it might cause a lot of harm is very high. That's Gary Marcus, a cognitive scientist who for the last 20 years or so has been a vocal, very lenteless and often gleeful critic of the way artificial intelligence is being pursued. His favorite target is the current crop of bots based on large language models or LLMs. Bots like ChatGPT. This should be very interesting because you're a noted skeptic of AI, particularly large language models. So before I hear the full on skepticism, what do you like about AI? Anything? Yeah, there's two things I like about it. One is I think people are starting to take the scientific applications more seriously. That doesn't necessarily mean using large language models. So I think the best scientific application right now is AlphaFold, which has some elements of large language models in it. But it's not just a chatbot and you type in and say, "Hey, what does my protein look like?" That's the one that goes from a nucleotide sequence and predicts the three-dimensional structure. It's a very focused model that works on a very particular problem and does it well. And I love it. I think AlphaFold is probably the best contribution of AI to science so far. And then I also like that after five years of, I don't know if I can say this on your show, bullshit about how amazing large language models are going to be, the field is actually recognizing reality. So I've been kind of pushing back for a long time. But the news is that people like Ilya Setskever, who help develop large language models, are finally saying, "Hey, we need to go back to research. We need to invent new things." And you and I are both fans of science. It's time that we bring AI back to science instead of just hype. And I'm finally seeing some signs of that just in the last several months. Well, what is actually in your mind wrong with it now? So I look at that in two ways. One is like, why from a technical perspective is it just not satisfying? And the other is the harms that it's causing to society. So from a technical side, the problem is these things masquerade as intelligence. But they're really just approximating the statistics of human language that they've scraped from the internet. They don't really know what they're talking about. And that makes them always unreliable. They hallucinate. They make stuff up. As a scientist, I just find them deeply and elegant. I don't feel like they're what we want. I mean, imagine if someone had come to us and said, "I have this calculator thing. It only works 80% of the time. But isn't that great? Give me a trillion dollars." Like that, when you're flying." So the hallucinating and other sources of misdirection false information is due to the fact that it relies so much or entirely on figuring out what the probability of the next word in ascenses. Entirely. I mean, that's what it does. I mean, I guess strictly speaking nowadays, people put a lot of band-aids on top of that. But the core of the system, that is entirely what it does, is predict next words. And that's just not the right way to think about what human cognition is. We do some of that prediction. If I say the word inextricably, you can tell me that bound is a likely word to come next or linked or something like that. You subconsciously track a lot of the statistics of language. And that's fine. But there's only part of what you do as a human being. You probably knew the late Dan Economan and his famous stuff about system one and system two cognition that he wrote about in his book, Think and Fast and Thinking Slow. The stuff that large language models do is like Danny's thinking fast. It's reflexive, it's automatic, it's statistical. But there's other stuff we do, like when we deliberate over something, that's what Danny's system two was about. And these systems don't really have the abstract understanding of how the world works, how people work, etc. to do that deliberative reasoning. So it's kind of all system one and no system two. These systems, because they don't really understand what reality is, can be coaxed into doing almost anything in a way that sounds plausible, but is often incorrect, unethical, immoral, etc. So I think they're causing a lot of harm to society. You mentioned LLMs don't have a sense of how the world works. What do you mean by that actually? So they can give you an impression that they understand how the world works by mimicry. But they don't have what many of us in the field like to call world models. So they don't know all of the facts about something, they'll make mistakes. You might have, at some point met my friend Harry Sheerer. One of my favorite examples is with Harry. He played the bass in spinal tapet as the voices in the Simpsons. And he one day sent me a biography of him that was written by ChatGPT in which it claimed he was a British voiceover actor and comedian and so forth. But he was actually born and raised in Los Angeles. You could have figured that out just by going to Wikipedia or IMDB or rotten tomatoes, Harry is a reasonably successful actor and comedian. It's very easy to find information about him that would refute what the system said. Nobody has yet made a kind of large language model at the core system that does not lose it. It just simply has not been done. It's always putting band-aids on the problem after the fact. I wrote a book in 2019 called Rebooting AI and the subtitle was Building AI We Can Trust. And I think that word trust is really important is still critical. Like you can trust a calculator. If you type in an arithmetic problem, go back to me. Joe, earlier, you know that the answer will be right up to certain parameters about like what the largest number is a conhandle. We can't make any formal guarantees that anything will be reliable in large language land, which is part of why I think we need to build other techniques. Like, if you just want something to do brainstorming, it's fun. It'll give you ideas that other people thought of. But if you have something you really need to count on, this just isn't the right technology for that. So, I guess all that makes it difficult to expect LLMs to produce artificial general intelligence, which is supposed to be an AI that knows more than any human does at the moment. That's right. So, people want to say we're close to AGI because it drives up their stocks or for financial reasons. But I don't think that it's true. I mean, often AI can do certain pieces of some job. Rarely can they do all of that job or no way I can do all of the things that a person can do. Like, tidying the house or, you know, a person can tidy the house with a joystick, you know, it could take a robot and control it with the joystick. The robot left to its own devices is not going to be able to tidy your house. Some people put out some robots last year and the reviews of them were like these things are ridiculous. We can't actually use this. So, is there a better way to approach this? And if there is, why aren't they doing it? So I think the better approach starts with a couple of things that I've been pushing for years and years. One is what I call Neurosambolic AI. No, that we have to translate. What is Neurosambolic AI? So it goes back actually to what I was saying about condom and system one and system two. So neural networks are the technology that's very popular. And there's another stream of AI that's totally out of fashion called symbolic AI, which basically just looks like computer programs. And it's been around for seven years just like AI and it's one of the chiefs trans. And there's been enormous hostility and kind of fighting over turf between these two approaches where everybody's like mine's better. In reality, they each do certain things well. So the neural networks are very good at condom and system one. And the classical AI is good at condom and system two. And it's just completely obvious if you look at them like the neural networks can learn from lots of data, but they can't reason abstractly make lots of errors. And the classical symbolic AI can't learn from lots of data. It's not as good at the statistical stuff, but it can make abstractions. The analogy is algebra. You learn things like an equation y equals x plus two. And then once you know what x is, you can calculate what y is and you can do that for any value of x. You do that in a very abstract and general way. The core problem with neural networks is that they don't abstract in general way. How well they abstract depends on how many similar examples. They almost do something a little bit like analogy, but you have to be really close to the stuff you've seen before. And as you move further away, my favorite example of this, by the way, it's not a perfect example, but it's a vivid one, is Tesla has this feature where you can summon your car. You press a button and it will come across a parking lot. So somebody did this at an airplane trade show. And the Tesla proceeded to run into a three and a half million dollar jet because jets were not part of its training set. There's things are very bound to the specifics of what they're trained on. They're not learning abstract principles like don't drive into something expensive or don't drive into something big or don't drive into something made of metal that you don't recognize. Right. That's how humans would think about it. No human would drive into a three and a half million dollar jet in the light of day unless maybe they were drunk or something like that. So the systems don't understand how to deal with things that they're not familiar with. They're just, it's almost like they have a giant video library and they're trying to find what's closest to that video. But things can be new and different. And when things are new and different, that's when they tend to break down. But if you have an abstraction, and then there's some answers around this. But if you have an abstraction, you're better able to deal with things that you've seen before. You can see this in math. A pure language model left to its own devices will be terrible at multiplying 12 digit numbers. Whereas a classic symbolic system, a calculator, will have no trouble at all at multiplying 12 digit numbers. They're just, they're very different systems. One of the things we need to do is to learn how to put them together. But because of the history of economics, of how grants are distributed and prestige and so forth, the people working in these two fields mostly hate each other. But in the last few years, places like DeepMind have actually done a lot of work to try to build neural networks together with symbolic systems. And there's just a small startup that is apparently raising money at a billion dollar valuation that's doing some of this. Times are changing. People are finally recognizing this, which I think is exciting. They're moving past these kind of historic grudges and so forth. So that's one of the key things we need to do. The other is we have to have these systems have enduring models of the world. So they have to keep track of individuals, their properties. Everything that they're doing is trying to learn correlations between little bits of information. They're not really creating databases of the objects in the world, the entities in the world, the places in the world, the causal principles that govern those. Whereas every time we humans interact with something, we're doing that. We build multiple mental models internally. So my kids actually did just read all the Harry Potter books and watch all the Harry Potter books. And they built a mental model of how that world worked. So they understood that in that world, you can fly on broomsticks. And they were never confused. They never thought that in this world, you could actually fly around on a broomstick. They built a model of the world of Harry Potter. And they learned all the names of the characters and played some of them for-- or dressed as some of them for Halloween and knew everything about them. My daughter just participated in that Harry Potter or trivia game, Variational and Trivial Pursuit. So she learned a lot about that world. And she kept that world straight in her head, separate from the rest of the world. And from other fictional worlds, where other things go on and so forth. It's really core to how we interpret films and movies and nonfiction articles and so forth that we build world models. Or we build world models of how other people think. So why is Trump doing this? What do I know about his background and his goals and motives and mental states such that he might do this? We do this all the time trying to interpret the world. We built mental models of people, objects, nations, et cetera, et cetera. That is just how we think. And it's not how LLM's think. And it will be how some future generation of AI things will be by making reference to those internal models the structure of the world. This is incredibly deep to what we need to do. And a bunch of people, again, have started to recognize that. Some companies have been founded around that idea recently. There's finally some recognition of this. Meanwhile, I take it that the danger is present right here and now to the economy. That the economy could take a dive. It could. Unless some other path is pursued. Yeah. So I mean, then other path is going to be slow, probably. I mean, you and I both know about science that often takes a while to figure stuff out. I mean, I've watched your scientific American show and it's various iterations. Some of the questions that you were looking at whatever 30 years ago, we still have it in itself. Science is a slow process. It's an unpredictable process. I think there are better approaches to be had. I think there's finally some progress on them. But it's hard to say how long they will take. Meanwhile, we have this very flawed approach that there's an insane amount of investment in. Trillions of dollars literally. And the market finally woke up to that in November. And many of the publicly traded stocks have started to fall. And then many of these companies are not publicly traded. But people are waking up. They're realizing that they've been sold a bill of goods. One of the big transitional moments, I think, was in August. There was certainly a transitional moment in my life. I'll tell you why in a second. An August 7th, opening I finally released Chat GPT-5. They had been promising it for years. And they suggested it was going to be AGI, artificial general intelligence. In January of last year, Sam Altman, the CEO, said we now know how to build AI in a conventional sense, or words very close to that. But when it came out, the first thing is at the release thing, he said, it can do anything a PhD could do. But within hours, people got to play with it. They realized it wasn't true. It couldn't really do anything a PhD could do. They'd do some of the things a PhD could do. And sometimes it would kind of fake its way through. And it'd give you an answer that sounds like a PhD, but it was totally wrong. Which has been characteristic of all of these systems for a long time. It made a difference in my life, because I kept telling people it's going to be late, which it was, it was like a year later than a lot of people expected. And disappointing. And in fact, a lot of people were disappointing. So somebody wrote a book on Amazon called Gary Marcus was right, and that became a popular phrase on Twitter, and so forth. So in August, people realized that they had been sold bill of goods. That Sam Altman did not know how to build AGI. And these systems were going to continue to hallucinate and make stupid mistakes. That sometimes they'd be amazing. Sometimes not, you wouldn't be able to predict when. And then MIT had a study showing that most companies were not getting returned on their investment. And so suddenly, like, whenever I go to a conference now, the question everybody gets asked is, is there a bubble? When I wrote about this, I think I was the first person to write about it. In August 23, I had a piece called What If Generative AGI Turns Out To Be A DUDD. People thought I was a Martian. They were like, open AIs, a rocket, Gary is the Grinch. Somebody called me the Grinch of A.I. I think it is a bubble. But I think it's going to end soon. And here's my theory. My image is Wiley Coyote going over a cliff in Bugs Bunny. And it doesn't fall until he looks down. Well, I think the economics of generative AGI have never made sense because the technology inherently is not reliable enough. And it's expensive to run. And so those things just don't add up to me. It's never been clear how this was going to be a profitable enterprise at the scale that it's being invested. But people didn't recognize that. So we were over the cliff, and they didn't recognize it. And now they do. The phrase I saw today, and I've seen a few times lately, is people, investors on Wall Street, are rotating out of the tech market. That is the beginning of the decline of the bubble. And people rotate out if it's five of them, that's OK. But when it's millions of them, that's when the bubble collapses. And so I think it's soon. There's an old saying you probably heard it about shorting the market, which is the market can remain irrational longer than you can remain solvent. And the point is, even when things are rational, you don't know when they're going to end. Imagine you were in Holland when the tulips were popular, and whenever it was the 1830s, right? And you're like, a tulip is not worth more than a house. This is ridiculous. But you can still say, well, but maybe I should buy one more and flip it before all this thing falls apart. You don't know exactly when it's going to fall apart. You can see that it's insane that the economics make no sense. But you don't know exactly when it's going to fall apart. This is an idea now that has seized a lot of people. And it's starting to change things. [MUSIC PLAYING] When we come back from our break, Gary Marcus tells me why he's especially concerned that too many people are handing over too much of their lives to AI bots. And this includes their personal information. Just a reminder that clear and vivid is non-profit. With everything after expenses going to the center of recombunicating science at Stony Brook University, both the show and the center are dedicated to improving the way we connect with each other and all the ways it influence our lives. You can help by becoming a patron of clear and [email protected]. At the highest tier, you can join a monthly chat with me and other patrons. And I'll even record a voicemail message for you. Either a polite, dignified message from me explaining your inability to come to the phone, or a slightly snarky one where I explain you have no interest in talking with anyone at the moment. I'm happy to report that the snarky one is by far more popular. If you'd like to help keep the conversation going about connecting and communicating, join us at patreon.com/clearinvivid. And thank you. This is clear and vivid and now back to my conversation with Gary Marcus and why he says the US is in danger of relinquishing its lead in AI to China. Nobody's going to win the large language model race. What somebody might win is the race to have genuinely new ideas. That's about scientific innovation. And I think you and I know that the US is kind of shooting itself in the foot right now by cutting back funding on science and going after the universities. China wins. It's not because we spent more dollars on large language models. It's going to be because we're gutting a fantastic scientific infrastructure that has led the world for years. And they're not doing that. They're building theirs up. And it's going to be a long term play, not a short term play, that's going to determine really how the AI race goes. Another way to put it is, I think the AI race still has another 30 years to go. And we're putting in all our chips on this crazy notion that we're going to win it right away when the other team, so to speak, already has the same technology as we do. Like it just doesn't make any sense. I wonder about another disaster that's waiting to happen. It seems to me that more we give AI's access to our private information and allow it to make decisions for us that we're on a slippery slope to where more and more decisions will be made by AI. and of greater and greater importance. And that sounds like, well, it won't wipe out humanity. It'll sure make life tough for humanity. - Yeah, I might actually have a sort of rare moment of optimism or something that might surprise you there, which is I think in the long run, it might be okay if AI does most of our decision making. You know, humans are pretty fallible creatures. I wrote a whole book about that called Cluj, which is Cluj is an old engineer's word for like duct tape and rubber bands. Like art design could be a lot better, but we're not designed with a product of evolution. And I could imagine AI systems that were less prone to emotional stuff, didn't have these cognitive biases, like confirmation bias, motivated reasoning. Like, in principle, I could imagine that AI might actually do a lot of stuff better than we do. I just don't think that we're remotely there now. And the risks that you point out is real, because right now we're taking the systems that don't deserve our trust and giving it a lot of trust. A crazy version of that that just happens this thing called multiple or open claw. It's gone by three different names in seven days, which is supposed to be an AI agent and supposed to do your plan, your wedding or your trip or do all your tasks for you each day. And it's seductive because like it promises to do so much. Like, you know, everything a personal assistant could do or an intern and more. And so people, they see it work for a minute and they trust it. And then they give it their passwords, full access to their computer. And it's total security nightmare. And, you know, government officers will probably use it. We've already seen some government officers use these things in foolish ways. And so it becomes a huge security risk. If you take systems that are dumb, and I think large language models are at some level dumb, and you give them a lot of power by giving them your passwords or control of the electrical grid, that's a nightmare. There was a headline about me a couple years ago. It said, Gary Marcus used to think that AI was stupid. Now he thinks it's dangerous. And implicit thing is like, how could you think both of those? But of course you could. You know, you don't want a stupid person to be empowered. You don't want a stupid AI to be empowered, right? If you give a lot of power to a system that is not what we might call, you know, rational and reflective equilibrium, we're in deep trouble. And that's exactly what a whole, like millions of people did, or hundreds of thousands of people did this, they gave control of their machines essentially to systems that hallucinate, make stupid mistakes, and can be hacked. This is not a good idea. - Now people are worried about AI destroying humanity. Some people are, I take it, you've fallen a position in the short of that. - Yeah, I talk about a couple of different things. So one is that people talk about P-Dume. That's the probability that AI would lead to the extinction of the species. I mean, we're not gonna last forever. The sun's gonna explode or whatever, but this is like specifically about what is the probability that AI itself would lead to our extinction? I think that probability is very low. I will be frank that I worry about a company, a guy who owns a company that builds both rockets and reckless AI and like what he could do to us. Maybe I won't name him, but you know who I'm talking about. So there are some individuals in a great deal of power who are reckless with respect to AI that could cause some pretty bad things, but I don't think extinction is that likely. You have to remember how geographically diverse we are, how genetically diverse we are, how resourceful we are. It's very hard to annihilate all of us, but there are two other things I'm really worried about. One is what I call P, a probability of dystopia. P dystopia I think is very high. I think we are being led right now into a dystopia where authoritarian is basically get all of our information from AI systems where we type in all of our most personal details. And then they create misinformation and manipulate us. I think we're basically there and it's gonna take a lot of work to get away from that. So I think that kind of or well in dystopia, supercharged by AI is basically arriving now. And then there's p catastrophe. What's the probability that for example, weaponized disinformation might lead to an accidental nuclear war. We blame somebody for something because we saw the video, but the video was fake and so forth. And I think that is also fairly high. One of the things that I wonder about, there was an experiment where an engineer informed the AI that it was in danger of being replaced by another model. And it objected to that and tried to sway the engineer's thinking and not shut it down. So is there maybe an emergent property of trying to maintain your existence if you're a complex enough system to resist death or replacement? So what I have seen so far, what I can understand from large language models, there's no emergent property of them wanting to preserve themselves. But they are very prone to imitating stories that they have seen. So even if they don't understand those stories, they can still imitate them and that could still have consequences if you hook them up to the real world. So like if you want to be philosophically correct about it, I would say there's no intention, there's no desire. The system isn't trying to do anything, but at the same time it can still change the world in certain respects because people can respond to it in a particular way, or if you give it passwords, then it can use your account, it could buy stuff, it could send that stuff. And I mean, think about, we should have talked about this before, what malicious actors can do with this. So maybe the system doesn't have any desire, but a bad actor can be like, I want to take down this enemy business of mine and I'm gonna program it to extort and et cetera, et cetera. And so even if there's kind of human lurking off stage, the system can still be manipulated into doing a lot of harmful stuff. - You were talking about multiple before and that seems to present a problem of AIs talking to each other, eventually developing a shorthand language that becomes a language unintelligible to humans. And arriving at behavior or responses to prompts that are not so good for the humans. - I think it's a legit worry at some level. I mean, again, I've caviated with my worries about intentionality and so forth. If Facebook did an experiment like this, I want to say like eight years ago, it was much more primitive the systems weren't as powerful, but they showed it in principle. Yes, these systems could develop languages that people don't understand and so forth. This moldbook thing is like at a different level. You suddenly have millions of these bots interacting with each other. We already know they'll do things that are less sophisticated than developing a new language, but like communicate messages by having them on white text on white backgrounds so the humans don't see them. So maybe they can do both, but certainly we already know that they have channels of communication that are not obvious to human beings. And I think we should be concerned. I think it's a kind of security disaster. An analogy I would make is that Apple, I used to just for fun program apps on the Apple iPhone. In fact, I showed you one, the one time that we met, I showed you one that did musical improvisation. I just did that as a hobby. You were kind enough to take an interest in it. So I used to program Apple apps for fun. And they're all sandbox. So when I made my music improviser, it couldn't look at your credit card numbers or order or military action or anything like that. It's very restricted in what it could do, but this moldbook thing is like the opposite. It can do anything. You type in your passwords and his control of your machine. So it's not just if you say make travel reservations that will only make travel reservations, it is empowered to do anything it wants to your computer. And if your computer can talk to another computer that can talk to another computer, it soon spirals out of control. And so it's completely uncontrolled. This is just really not a good idea. In fact, it's such a bad idea. Well, I wrote a piece about it on Sunday. I said, I advise you not to use it and don't use someone else's machine who uses it because you might get at what I called as a joke, a CTD, chatbot transmitted disease. So I wrote that on Sunday, don't get a CTD. And then it just came out in the information just before we recorded this, that Microsoft just sent out a memo telling its employees basically to treat this thing as a hazard. Like it is really dangerous, but you see this tension between what people think is fun and what's actually safe for them. It kind of saw this with Facebook where you had teenagers posting half naked pictures of themselves in a party. That's great for Facebook because it gets views and whatever. But then the teenager applies for jobs and then those pictures are out there. So people have been seduced by social media and now by the chatbots into doing things that are actually adversarial to their own interests 'cause they can't quite see far enough. They don't recognize the giving their password to this thing to have fun with or to have it do travel reservations might lead to their information being compromised. So like there's a social or dating system called Grindr. It's popular in the gay community and there have been extortion things where people get extorted just for being on Grindr. And we're gonna see a whole bunch of events like that where people typed in their password and some bad actor realizes how vulnerable this is and extracts all kinds of information about people's sex life or all kinds of different things. And so I expect the follow up for this one to be. last for a long time, even if people wise up pretty quickly, they shouldn't infect me using it. (upbeat music) - Well, this has been a really interesting conversation. I thank you for having it with me. Before we go though, we always end our show with seven quick questions. You game? - Sure. - Okay. Of all the things there are to understand, what do you wish you really understood? I really wish that I understood how the brain worked. Like that's a question that I started with as a child and I still feel like I don't know the answer. And as a curious person, I want to know. - Next. How do you tell someone they have their facts wrong? - I'm not the most gracious about that. I just tell them. You just ask for someone else for how to do it with grace. - I don't know. In the show today, and you were very gracious on the show today, I noticed that you deliberately went through the positive answer. - Well, maybe you bring out the best of me. (laughs) What's the strangest question anyone has ever asked you? - The strangest question. That one's a hard one. Oh, you might have me stumped on that. The strangest question that anyone's ever asked me. Again, a lot of weird emails these days. I'm not alone in getting them, but weird emails from people who talk to chat butts and are deluded by them and think they've made some great discovery about physics or AI or information theory or whatever. These are really weird things where they've been egged on. We call it sick of fancy by the chat butts. And some of these people are not in touch with reality. I can't pick one of them. I get lots of them. There's a tie for first place in these weird emails I get. - How do you deal with a compulsive talker? I think I am a compulsive talker. I just prove that. - Do you just try to start first? I don't know. I do like to engage people in debate. I like to understand how people think. So at least for a little while, I'll hear them out and see what they have to say. My theory is most people have at least one interesting thing to say and I'll try to find that. And then I will politely excuse myself at some point. - Okay, let's say you're sitting at a dinner table and you're next to someone you've never met before. How do you begin a genuine conversation? - I was asked people what they do or what they're interested in try to get at, how they got there. My training was originally in psychology and cognitive science rather than AI. And I'm interested in people's stories. I loved when I was a kid, I'm sure you know these books by Studs Turkle or the oral histories of people. And I've seen you kind of do similar things with scientists trying to bring out their story. I love doing that. I love to understand. How people think and why they are who they are. So I tend to go in that direction. - What gives you confidence? - What gives me confidence? The fact that I've made these predictions for 25 years and they keep being right. - Okay, last question. What book changed your life? Cosmos by Carl Sagan, that really brought in the TV show that went with it. They really brought me into science and appreciating the mysteries of the world. - Gary, thank you very much. We covered a lot of things that scare me a little bit but you also have an optimistic tone that makes me think somehow we'll get through this new latest great invention of humanity which they all seem to be threatening in some way. - Well, we need to do some work to get it right. Call me anytime. We've had two conversations in my lifetime and I've loved them both. So anytime you want, I'm here for you. - Thanks so much. - Thanks a lot, Ticcher. (upbeat music) - This has been Clear and Vivid, at least I hope so. My thanks to the sponsor of this podcast and to all of you who support our show on Patreon. You keep Clear and Vivid up and running. And after we pay expenses, whatever is left over goes to the oldest center for communicating science at Stony Brook University. So your support is contributing to the better communication of science. We're very grateful. Gary Marcus is a merit as professor of psychology and neural science at New York University. His best selling books include Taming, Silicon Valley, How to Ensure AI works for us. You can keep up with his scathing takes and occasional optimism on the state of AI by following him on his sub-stack, Marcus on AI. This episode was edited and produced by our executive producer Graham Shed with help from our associate producer, Gene Choumé. Our publicist is Sarah Hill. Our researcher is Elizabeth Ohini and the sound engineer is Eric Ahwang. The music is courtesy of the Stefan Kernig Trio. (upbeat music) (upbeat music) Next in our series of conversations, I talk with Pulitzer Prize-winning author Ed of Fields Black. Her book, Comeby, tells the remarkable and little-known story of how in 1863, Harriet Tubman played a critical role in a daring raid that freed some 700 slaves from rice plantations along South Carolina's Comeby River. It was 4 a.m. when the enslaved people first sensed that something was happening. At that time on the Comeby River, you can't see your hand in front of your face, yet they were in the rice field, hoeing rice. And they were enslaved people in the still in the slave cabins. And we know from a letter that Tubman's nephew wrote in the 1930s, that she actually went to the slave cabins. And this is where the most vulnerable people would have been, very elderly people, young mothers with infants, she went to the cabins and made sure that those people got out. Everybody running, everybody running for their lives. Ed of Fields Black and how Harriet Tubman helped lead the Comeby River raid. Next time on Clear and Vivid. For more details about Clear and Vivid and to sign up for my newsletter, please visit allenolder.com. And you can also find us on Facebook and Instagram at Clear and Vivid. Thanks for listening. Bye-bye. [BLANK_AUDIO]

Podcast Summary

Key Points:

  1. Gary Marcus expresses concern about powerful individuals recklessly developing AI, acknowledging significant risks of harm but dismissing human extinction as unlikely.
  2. He criticizes large language models (LLMs) like ChatGPT for being statistically driven, unreliable, prone to hallucinations, and lacking true understanding or reasoning (akin to "System 1" thinking without "System 2" deliberation).
  3. Marcus advocates for a hybrid "Neuro-symbolic AI" approach, combining neural networks with symbolic AI to achieve reliable reasoning and abstraction, and stresses the need for AI systems to build robust "world models."
  4. He argues the current generative AI investment boom is a bubble, citing high costs, unreliable outputs, and unmet promises (e.g., ChatGPT-5's failure to deliver AGI), predicting an imminent economic downturn for the sector.
  5. Positive aspects he notes include scientific applications like AlphaFold and a growing recognition in the field of the need for more rigorous, science-driven research beyond hype.

Summary:

In this conversation, cognitive scientist Gary Marcus discusses the risks and current state of artificial intelligence. While he worries about powerful figures developing AI recklessly and the potential for significant harm, he considers human extinction unlikely. He is highly critical of large language models (LLMs) like ChatGPT, arguing they merely approximate language statistics without genuine understanding, leading to unreliability, hallucinations, and an inability to reason abstractly.

He contrasts this with human cognition, which combines intuitive "System 1" thinking with deliberative "System 2" reasoning. Marcus advocates for a "Neuro-symbolic AI" approach, merging neural networks with symbolic AI to create systems capable of abstraction and reliable reasoning, and emphasizes the need for AI to develop internal "world models" to understand context and reality. He believes the massive investment in generative AI is an economic bubble destined to burst due to the technology's inherent flaws, high operational costs, and failure to meet exaggerated promises, as exemplified by the disappointment surrounding ChatGPT-5.

Despite his skepticism, he acknowledges positive trends, such as valuable scientific tools like AlphaFold and a growing shift in the AI community toward more substantive, research-focused development.

FAQs

Gary Marcus believes AI poses significant risks of causing harm, such as spreading misinformation or unethical behavior, but he considers human extinction very unlikely. The primary danger lies in the unreliability and potential societal damage from flawed AI systems.

He appreciates AI's scientific applications, like AlphaFold for protein structure prediction, and notes a growing recognition in the field to move beyond hype toward genuine research and innovation.

LLMs merely approximate language statistics from internet data without true understanding, leading to unreliability, hallucinations, and a lack of abstract reasoning. They operate on reflexive, statistical thinking (like System 1) but lack deliberative reasoning (System 2).

LLMs lack world models—internal representations of entities, facts, and causal principles. They mimic understanding through data patterns but cannot maintain consistent, accurate knowledge about reality, as seen in errors like misattributing a person's nationality.

Neurosymbolic AI combines neural networks (good at pattern recognition) with symbolic AI (good at abstract reasoning) to address the limitations of each. This hybrid approach aims to create more reliable and generalizable AI systems that can handle novel situations.

Marcus argues there is a bubble in generative AI, as the technology is often unreliable and costly, with unclear profitability. He predicts a market decline as investors recognize these flaws and shift away from overhyped AI investments.

Chat with AI

Loading...

Pro features

Go deeper with this episode

Unlock creator-grade tools that turn any transcript into show notes and subtitle files.