Is AI Making Us Stupid?

Hi, I'm Meryl Horn filling in for Wendy's Accommod, you're listening to Science Vs. This is the show that pits facts against frying our brains with AI. Today on the show is AI destroying our ability to think? A lot of people are saying, "Yeah, it is," and they say that we've got the science to prove it. We've been medically verified that Chatchy Pt makes people stupider. Chatchy Pt users could not even remember what they wrote. They basically stopped thinking entirely. Every single time we give it a prompt, our brain cells are burning. And if this is true, we might be in for a brain cell bonfire because a lot of us are using AI. One recent survey found that 62% of adults in the US say they interact with it at least several times a week, and AI really seems to be booming for students. A bunch of surveys suggest that around 80% of high schoolers and college students say they're using it for school work. And nobody's really worried about this. Obviously, I'm kidding. People are panicking. We have no idea how our students will ever learn anything or whether universities have any future. One professor graded papers and discovered every single one was AI-generated. The whole system is cooked, cheating is the new major, game over, it's just f*cking game over. But there is a flip side here because some people say that AI is going to be good for us. That can help us save time by breezing through busy work, get information for us, and accelerate scientific progress. It can do away with a lot of the really annoying sort of tedious tasks. It's helped me boost my productivity by like 300%. AI is driving a transformation across all fields of science. Buckle up tech enthusiasts, the future with AI is bright and it's happening now. Wait a minute, buckle up tech enthusiasts. That last one was AI. How did it get in there? But seriously, who is right here? Could AI help us be more productive and lead to a boom in science or is using AI the equivalent of shoving your brain into the microwave? So do we stop using AI so our hippocampus doesn't turn into a hot pocket? Buckle up science enthusiasts because when it comes to AI, a lot of people are saying, "Chat GPT makes people stupider." But then they're science. And full disclosure, some AI companies do advertise on science versus. Science versus AI is coming up after the break. Welcome back, I'm Meryl Horn and today we're going to look at whether using AI, stuff like chat GPT is bad for our brains. I have senior producer Rose Rimmler here. Hi, Rose. Hi, Meryl. So you recently mentioned to me that you have been using chat GPT more and more of these days. Hi, I guess that's true. My under oath. Curious like do you feel like it's changing how your brain is working? I do sometimes catch myself about to ask chat GPT something that I could do and I'm just too lazy. Do you want to see what you use chat GPT for recently? I asked how many Roma tomatoes equal 500 milliliters of pure tomato? How to clean my velvet armchair, I spilled milk on it. I asked it to tell me if a couple of bottles of wine I had on Thanksgiving were dry or sweet, I think, dry or juicy. Would you say it's like your go to now for just looking up in phone online? Yeah, it is kind of becoming my go to. It is. For me, also, and I've been curious about what kind of problems this might lead to. So let's just address one thing first. We do need to acknowledge that one big problem with this sort of AI is that it can get stuff wrong. I actually did try to find some numbers on this. How often is it wrong and when chat GPT first came out, it looked pretty bad. A lot of the research was finding that it was only right, roughly half the time. I remember that from the early chat GPT exploratory stuff. Yeah, I made up a lot of stuff. Now there are a couple of reviews which compare like chat GPT 3.5 to chat GPT 4 and they do find that it's gotten better. It's still not 100% accurate, so bottom line, it's just hard to tell whether it's telling you BS or not. Yeah, that's true. So, yeah, I do feel like that's one way it could be making us stupid. It's just by feeding us incorrect information. Yes, just like the government. But now, on top of that, there's this other fear that it's bad for our brains to be using stuff like chat GPT or other LLMs, large language models. Like if we let these things do a bunch of the thinking for us, then we'll lose our ability to think on our own or even be creative. So, let's dive into all of that. Okay. And I want to start with this one study. It was done by Sheri Melomod. She's an associate professor of marketing at the University of Pennsylvania. And Sheri looks at how tech is changing us. When to do first, get the idea to start looking into AI. Yeah, I mean, as a person who studies technology, it was difficult not to. Study AI, right? It's pretty prevalent at this point. So yeah, Sheri just published a huge study on AI altogether. It looked at more than 10,000 adults. And the goal was to see what happens when we use LLMs like chat GPT to try to learn something new. Like, how does it compare to an old school Google search? So here's what she did. First, she got some people to do a fun little research assignment. I told them to imagine that a friend came to them asking for advice on the topic. So, for example, how to plant a vegetable garden. Other times, they had to research something else, like how to lead a healthier lifestyle. And so they did some research on this thing. And half of them had to use chat GPT for this, where the other half had to do a normal Google search. Like, no, AI summaries, just links. And their next job was to write up a little blurb based on the stuff that they had just read, as if they're writing up advice for that imaginary friend who needs their help. And this is what Sheri was really interested in, because she wanted to see if the advice was any different when people used Google versus chat GPT, and it was. So when they used chat GPT, the advice they came up with was sparser. It was more generic, and it referred to fewer facts after participants learned from an LLM versus web search. That's interesting. Yeah, so let me play you some examples so you can hear for yourself, like what it sounds like when Sheri says the advice was more generic and referred to fewer facts. Okay. So, this first one was written by someone who used chat GPT to research how to lead a healthier lifestyle. Basically, you want to eat better foods and limit sugar and processed foods. Get at least 30 minutes of exercise a day. Stay hydrated and also check with your doctor as well. What do you think? Wow. It's like a lyric poem. Yeah. You're impressed. I'm so inspired to live a healthier lifestyle. And then she played me another one, which for me helped see what she meant by the genericness of it. So let me put you one more from this chat GPT. Having a balanced diet, exercising regularly, staying hydrated by drinking water, getting enough sleep and avoiding stress are ways to live a healthier lifestyle. To get more details, ask chat GPT. And this wasn't just copy and pasted. This was a human. Remember they did the research and then they actually wrote just as chat GPT. So yeah, now let me play you an example from someone who used the old school Google search for their research. So there's no AI summary. There's just a bunch of links to click on and they can look at as many websites as they wanted. And again, the prompt is how to lead a healthier lifestyle. So here's Sherry with one of the responses. Start with focusing on the outer, then inner of your body. It's recommended to be active a minimum of 30 minutes most days of the week by engaging in healthy movements such as walking, riding a bike, yoga, sports, or even dancing. From there, the inner workings make sure to stay hydrated with at least eight glasses of water a day, avoiding sugary drinks, and focusing on consuming a well-balanced diet. Out of a variety of foods to your diet, from vegetables, fruit, seeds, and whole grains, while avoiding foods high in sodium, and also avoiding foods high in saturated fats. Make sure to get plenty of sleep each night, at least eight hours or so. Always wear sunscreen before sun exposure to limit the chance of skin cancer. That's more engaging for sure. Basically the same advice, but it does sound like a human wrote it. Even though I know humans wrote the other examples too, but it doesn't sound like they did. Exactly. Yeah, it sounds more human, even though it had information that is wrong in it. The eight glasses of water a day thing isn't actually real. That's not based on science, if you remember our hydration episode, there's nothing magical about having eight glasses of water a day. Yeah, that's true. But even with that, I still is just more charming, even with the flaws. When I took a look at the other examples that Shiri sent me, a lot of them were like that. Here's Shiri. It's unique to the writer, right? It really doesn't come off as generic as the chat GPT pieces. Yeah. And it sounds like they're having fun with it, but they're so excited when they thought of that inner versus outer, even though it doesn't really make any sense, but I guess they had their own love. Yeah, it does read at least like the writers sort of put more of themselves into the advice. And in Shiri's experiments, she also asked the people who wrote the advice, how they felt about it, and the group that used chat GPT felt like they learned less compared to the people who used Google. And then she also showed the advice to a different group of strangers. And people basically liked the advice from the Google group better. They said they were more likely to take the advice and said that it was more helpful, more informative. Okay. Shiri, by the way, she also did the same thing with that Google's AI overview and the results were basically the same in both cases. The advice kind of sucks when people use large language models, whether it's chat GPT or Google's AI overview. Mm-hmm. Wow. The difference between how much people felt like they learned, it wasn't huge. It was statistically significant and Shiri found it again and again with different groups of people. So it does seem to be real. Mm-hmm. Okay. And if you think about why this might be happening, why people are learning about the topic as deeply, it does sort of make sense because when you use something like chat GPT for research, you are probably skipping over a bunch of steps. And it turns out those steps are actually pretty important when you're trying to learn something new. It's the process of going through the links yourself, reading them, you know, digesting them yourself, interpreting them, that leads you to at least feel like you're learning more. But also, we still find these differences in the content of the advice that they're right, which suggests it's not just like the sense of learning. It's actually differences in learning. So that's the important part is like doing the work of actually getting information from these different sources and then synthesizing it in your brain. Exactly. Because essentially, these syntheses that LLMs provide are transforming learning from more active to a more passive process and that's what we're losing. And we reached out to these companies, Google and OpenAI, which makes chat GPT, to get their take on this. OpenAI didn't get back to us, but Google told us that the AI overview is supposed to just be a jumping off point because you do still get those other links. But moving away from the study, there is other research that backs all this up and sort of gets at what might be going on in the brain. So the study kind of went viral. They looked at people's brains when they're using chat GPT. So it's called your brain on chat GPT. I don't know if you remember this one. Yeah. Do you think I saw that headline? Yeah. Yeah. It's just a pre-print. And it's pretty small. So take it with a grain of salt. But it was interesting. So they got around 50 people and then used EEG to measure people's brain waves. Those are like the little electrodes on your scalp. Yeah, exactly. Then in some people used chat GPT, sort of an essay, and other people used Google or just their own brains. And they found that when people were using chat GPT, brain connectivity was the weakest, which is sort of a measure for how much different brain regions are talking to each other. Wow. They could actually measure that. Yeah. And so it just seems like maybe people are just less engaged when they're using chat GPT. Yeah. It makes sense to me that you need to do some of the like trying and failing and then succeeding to make these connections work in order to really remember and process and add your own thoughts to what you just read or learned about. Well, it's funny you should bring up memory because there are also studies on a couple pre-prints that have found that that can also be worse when we use stuff like chat GPT. So yeah, if people use chat GPT to write something, they'll remember less of like what was in that work when they use AI. Yeah, I mean, I think sometimes that's okay because I might be using it because I don't really care that much to learn it myself or like whatever, I don't really need to commit this to memory for all time. You might not care about how to tell the difference between a juicy wine and a dry wine for like your life in general. And if I forget the answer, I'll just ask Chad to meet again or look back at my previous question. Yeah. Okay. So what she's saying is that it messes with our learning process and that is scary to think about it when we know how many students are using it. Exactly. Yes. Which they are, right? Yeah. Which is like a lot of what I hear about and I think it was in the beginning like, oh my god, the students are using chat GPT, they're not going to learn anything and that's bad because like that is what you're supposed to be doing at that age, it's like learning, learning, learning, learning. Yeah. Exactly. And a big idea is that this learning forms the basis for critical thinking, you know, doing research, putting together your thoughts in a meaningful way. And it does look like maybe these LLMs are getting in the way of that. Like there was a big survey of Australian university students that asked them what they use AI for and most of them said that they were using it to do stuff like answer questions for them or create text that I can use. So like that doesn't seem great, great text I can use cheat on my own, they're conveying to it. And so Sherry is also worried about this, about how AI will affect the next generation since most adults like we didn't grow up with this stuff. We had to figure out how to do our own research and write essays on our own. At least you and I have that foundation, but I'm really worried that younger generations won't be able to establish those foundations because it's so tempting to outsource all of that work to LLMs and AI in general. And do you think that we have cold heart data that says that students are getting worse at that yet? It's hard because you need longitudinal data and these things are only introduced fairly recently. But I do think that we currently already have data that is at least pointing directionally at what the effects are going to be. I mean, people are cheating back in my day too, but now they can do it even better. And just one more science tidbit, if you want to get a little bit more freaked out about all of this rose, let's just face it, finish it off. So there's this fear that AI will lead to something called de-skilling. Make us forget how to do things that we once knew how to do. The fact I had when you said de-skilling was, there's a term for when if your skin ever gets peeled back off something, it's called de-gloving, so I'm so sorry. This is the equivalent for your brain because you just like, yeah, we'll just lose these abilities. Thanks to AI. I did find a study that looked into this. So it was on a group of doctors doing colonoscopies and they started using AI to like find little spots on the inside of the colon that could become cancer. But then for the study, the researchers had them stop using the AI and it turned out that the doctors were then worse at finding the little spots on their own compared to before they ever started using the AI. Yeah, right? Not great. Right. Like, what if they can't access it? Or if something goes wrong with AI and worst case, the doctors might start missing these little spots and those can eventually turn into cancer. Which would total that would be the worst case scenario. Yeah. Interesting. I mean, it does all make sense because it's like, you get better at what you focus on and you forget when you don't focus on and that's supposed to happen because your brain can't retain everything. And yeah, otherwise you'd have these like gigantic, you know, these gigantic heads, you don't have to have expandable skulls. So of course, of course that would happen. Yeah. And now I do though think when I'm doing something with AI, I'm like, do I care about losing this skill? Yeah. Because otherwise I don't really mind. But other times I do. So it's helpful to think about that. Yeah. I guess it's like if it's something I really want to improve or get better at or retain, don't outsource so much of it to AI. Yeah. So that's sort of the science that I found that supports some of the fears around what AI is doing to us. But next, let's look at the counter-argument here. Okay. The claim that we have lots of gain from using AI, that it can kind of take care of the easy stuff. So our brains can do the hard stuff and ultimately use AI to do more and better stuff than we ever could have done without it. Today we're finding out whether AI is making us stupid or smart. Rose Rimmler is here with me. - Hello, Merrill. - Oh no. - I have been replaced by AI. - All right. - No, I'm still human and I'm ready to hear why AI is good. So yeah, we've talked a lot about the scary science, but I also wanted to understand the pro AI arguments. So I called up Aaron French. He's an assistant professor of information systems at Kennesaw State University in Georgia. He's also on the advisory board of an AI company. And he was basically like the effects of AI are gonna depend on how you use it. - For some people, it's absolutely gonna make them, I guess, I don't wanna say dumber, but they're not gonna learn or improve because of that. Other people, they're gonna be able to do more with AI than they were able to do without it. - So you could either use it as like a crutch or like an enhancer, is that the idea? - Yeah. - So one obvious way that it could enhance our work is that AI could take over the mindless rot tasks for us. - What are the repetitive tasks that consume a lot of time that I don't need to be doing? And can I use AI to handle those tasks that allow me to spend my time in a more valuable way? - Just the busy work. - Yes. - So yeah, the idea is that once AI does the busy work, the human brains can swoop in for the harder, more complicated parts of the task, you know, critical thinking and analysis. And so there is science that does sort of back up this idea because it does look like AI can save people time for one. Like there are studies finding that this is true for all sorts of careers. So dieticians, computer programmers, people who run clinical trials. The effects were pretty big in some cases. Sometimes it took people 30% less time to do something. Thanks to AI. Sometimes it was 80% less time, depending on the thing they were doing in the study. - Okay. - And an obvious caveat here is that you do need to, like for this to work, the AI has to do a good job at the thing for it to really replace. But we're hearing that in some cases it is helping. Teachers in particular have said that they can save a lot of time with AI. One survey found that some teachers who are using it were saving an average of six hours a week. - What are teachers using it for? - Putting together materials for class, sometimes grading. Basically, it's the type of stuff that teachers say is just hard to get done during their normal hours. And Erin's like, yeah, if you can get AI to speed some of this stuff up, it could really help. - As a professor, if I can use AI to accurately grade, that would be great because instead of spending five, 10 hours a week grading assignments, if AI can do it and provide proper feedback, I can spend that time working with the students, given them more engagement. Teachers do always talk about how much time they have to spend grading, right? And so Erin spends this saved time engaging more with the students. But this does raise a question, which is like, are most people actually gonna spend the extra time doing stuff like that, like doing their jobs even better or engaging with the world in a meaningful way? Or are we just gonna use it for like, mindless scrolling or watching Love Island? - So, well, that's your prerogative. You get to do whatever you want to do with your extra time. So judgey. - Well, but it's like, for the point purposes of this episode, whether or not, AI is gonna like lead to this like, you know, new and improved humanity, it might not do so well on that front. If we're just-- - You're not having time. - Like, take up the slack at some like, rope part of your job, and then going home and inventing a new kind of flying machine, like you're married to venture or whatever, you're just like, flying on the couch. You know, looking at Instagram. - It feels very likely, right? And I went looking for a paper on this, like how are people actually spending the time they saved with AI? And the only thing I could find was this one early study, not peer-reviewed, and if they got 83 managers, so these are people like vice presidents and sea level executives. And first, it asked them, do you save time because of AI and the vast majority said yes, it was almost three hours a week on average. And they did say that they often used the time to do stuff like continue working on my tasks or take on additional projects, but a lot of the time was also wasted. - And they admitted that? - Yeah, so within this group of managers, 36% of them said they wasted at least half of the time that they saved. (laughs) - Okay, wow. And they asked a random group of adults about this too and got similar results, which feels pretty on point. Yeah, we, as long as our bosses don't find out that we have all this extra time, we can just use it for watching Love Island. So that's to keep it quiet. - So stop calling out these surveys. Honestly, people, what are you doing? (laughs) But there is one other thing, a different way that AI might actually make us smarter. - Okay. - The idea is that AI will open up new doors and let us do things that we would have never been able to do without it. And the scientists who think that this is possible often bring up an analogy. They calculator, which does have some interesting parallels to AI because on the one hand, you can imagine that maybe people will get worse at math 'cause, you know, it's doing the work for you, but on the other hand, maybe we'll be able to do more things like harder math because of it. And one calculator is where first introduced to classrooms back in the '70s. It generated a similar controversy that AI is now, like some people staged protests. Some math teachers were worried that young kids would get hooked on calculators to do basic math problems. - To write blue bliss. - What? (laughs) - Yeah. - I've heard over and over again. - I forgot about that. (laughs) - And they even had a name for these students who were a calculator dependent, Calcuholix. - Oh my God. - Yeah. (laughs) - So the specific fear was Calcuholix. - Uh-huh. - But be so dependent on their calculators, they could no longer add and subtract and divide by it with paper and pencil or in their heads. - Exactly. - Yeah, they would lose those skills 'cause you'd just type everything in and we would never learn anything. - So that's cool because we've actually had plenty of time to test if that has happened. - Yeah. So we have science on this and when, so I looked to see like, okay, what did happen? - Yeah. - And it seems like the answer is, everything was basically fine. Like there is a meta-analysis of over 50 studies on this. - Oh. - Yeah, which looked at what happened when kids started using calculators in the classroom and it found that first of all, kids basic math skills didn't really get worse. Like if they got to use the calculators for learning, but then they took them away for a test, they didn't do any worse on that pen and paper test. And if they got to use the calculators for both learning and testing, they showed improvements. So their problem solving got better. The graphing calculator led to improvements in visualizing things and understanding graphical concepts. - Oh. - And some studies found that kids attitudes towards math was better when they got to use the calculator, maybe 'cause they got the right boobless. (laughing) Or because these were kids who have some interest in mathematical stuff, but not arithmetic, which is basically, which is just like two plus two. And that's the tedious part. And so they could outsource that part and dig into like imaginary numbers and trigonometry and stuff that gets more interested. - Yeah, exactly. And so yeah, overall the review said that the science supports using calculators in elementary and high school classrooms because our basic math skills didn't seem to get worse and now we can do harder math since we can use it as a tool. Is there anything we can say though about AI specifically in the classroom? 'Cause it's pretty new and it's different, you know, from a calculator. Well, like we said in the first half, like some academics are worried that if we use AI too much, you know, it'll kind of inhibit learning. But then there's also a ton of papers which show that it might be helpful in the classroom and in particular ways. So like there's tons of studies that just try having like chat GPT gives students feedback for like, you know, writing something and they'll find that like, look, it can help them polish their writing. And there are some researchers that are making like customizing chat bots to help students with specific learning disabilities and think that there's a lot of potential there. And then there are some studies that have students use chat GPT kind of for good, like as a tutor. So rather than just having it give them all the answers, they'll just have it help them learn how to do like, like a particular kind of math problem. And then like it's okay at that. Like it can do as good of a job as using a textbook to learn math. So like there's potential here and Google told us that they're working on tools like this too. I mean, sure, but also all those examples are the stuff that teachers are supposed to do, you know, like human teachers, that's kind of the point. So I guess theoretically, but often they're spread so thin, you know, I think the idea is that this can help them address all their students needs when like they're often just, you know, don't have enough time to do that. Yeah, fair enough. So yeah, maybe I think the dream is that this will be like a helpful tool in the toolbox. Yeah. And maybe, you know, yeah, well, we're just spitting out here over nothing. And then there are people at the top levels of academia who are using it to solve all sorts of problems. So not just, not just talking about LLMs, but AI more broadly, like machine learning. So for example, in biochemistry, AI is really good at predicting protein shapes based on like amino acid sequences. And physics has been used to help find black holes and analyze data from particle collision experiments in real time. And according to a group of 1,600 scientists, more than half of them said that AI tools will become very important or essential for their fields over the next decade. They make sense to me that they're going to be use cases where you're not going to have a physicist piece through 200 million images of the far reaches of the universe looking for black holes. They're just like, that's not even humanly possible. And it's certainly not possible in the span of like getting a PhD or something. So it makes sense to have the computer do it. And there's lots of things we outsource to computers or machines that we just generally feel isn't that good. So yeah, I'm convinced that there is, I'm convinced that there is a role for AI to be positive in personal development and like human development. Yeah, yeah, I agree. For me though, as I think about all of this research, the main thing that I've been finding myself worrying about is that homogenization effects. The fact that what we create is more generic when we use AI to do it. Like that really boring, yeah, saying healthy advice. That really stuck in my brain. And so now I'm like trying to avoid using it for anything too creative because there's so much now about how AI can be used for creativity in all these different ways. And I'm always now like, ooh, I don't know if I want it trying to do the things that I care about sounding like me. But I do, I still use it just for like looking up stuff that I don't really care about that much. Like in all of the academics that I talked to for this episode, including the ones that found negative effects from it, still use AI. Like Shiri are scientists from the beginning. She's like, you know, if you just want to look up something quickly and you don't really care about getting a super deep understanding of it. - Use an LLM, right? It'll make your life a lot easier. But to the extent that you actually care about learning more deeply about something, you should really try to avoid starting off your research with an LLM because it's too tempting to stop with the synthesis that you're provided. I even see it in myself. I've studied this stuff and when I start with chat, it should be to learn about something. I find it really hard to motivate to keep learning more. - So, Marilla, let me ask you, are you worried? I mean, you have a little kids too. Are you worried about this generation coming up? Are they going to be stupider less skilled than I don't know us or previous generation because of using AI, especially using it in school instead of doing their homework? - I don't know, I'm not that worried yet. It's a couple decades ago, people were freaking out about the internet making us stupid, googling stuff too much. But I think we're still okay. And even beyond that, humans have existed in so many different contexts throughout history when we were never learning how to write essays in school about pride and prejudice. - That's true. - Like, were we really all totally stupid when we were a caveman? - I don't think so, I think we were probably okay. - For whatever that context required of us. - Yeah. - Yeah, and now we'll be a new context 'cause AI will, I guess, change the world in some way and everything will just be different, right? - Yeah. - All right, thanks Rose, thanks, Mal. - That's science versus. This episode had 59 citations in it. Check out our transcripts if you wanna see all that science. And one quick note, we've heard from some of you wondering why you have heard a little less from Wendy, the past few months, she has taken a bit of time to be with family, but you'll be hearing more from her again in the new year. And the show will take a few weeks off for the holiday, but we will be back in your ears in January. And we have some amazing episodes in store for you in the new year. We'll tell you the secret to happiness according to science. We'll dig into relationships and tell you whether yours is toxic. And we're gonna give you the science on one of our most requested topics, running. Should we really be doing it? Plus the weird science of something called sad nipple syndrome. 2026, it's gonna be great. (upbeat music) This episode was produced by me, Merrill Horne, with help from a Kedi Foster Keys, Michelle Dang, and Rose Rimmler. We're edited by Blythe Cerelle. Our executive producer is Wendy Zuckerman, fact-checking by Erica Akiko Howard, mixed in sound design by Bobby Lorde, music written by Emma Munger, so Wiley, Peter Leonard, Bumihidaka, and Bobby Lorde. Thanks to all the researchers we spoke with, including Dan Yella Fernandez, Dr. Martin Romantic, Professor Michael Henderson, Dr. Tim Sindelka, and Professor Vito Mir Kovenovicens. Special thanks also to Sebastian Peleato, Chris Souter, Elise Indelen, Jack Weinstein, and Hunter. Science versus is a Spotify Studios original. Listen for free on Spotify, or wherever you get your podcasts. And if you do listen on Spotify, follow us and tap the bell for episode notifications. Fact you next year.