Ep.1 with Professor Wayne Holmes | AI in Education: Framework, Policy, Risk and What Comes Next

So recently Cher and I noticed something really quite unsettling. Everyone around us has been using chat GPT, Claude, or other chatbots for virtually everything. And we've realized that we were becoming genuinely dependent on it, that ability just to sit with a problem and think through it was slipping away. So this week, we sat down with Professor Wayne Holmes from UCL, who not only advises the UNESCO, but also the Council of Europe on AI in education. We talked about why there's almost no evidence that these tools actually help students in education. The policy's being built right now, across Europe to regulate AI, and what AI literacy really should look like. There's a lot again, too. I'm Melissa, and this is Cher, and you are listening to the Not a Stupid Genic Age podcast. So welcome, and thank you so much for joining us today. Would you like to give us a brief introduction about yourself? Sure. So I'm Wayne Holmes. I'm Professor of Critical Studies of AI and Education in the UCL Institute of Education, University College London. I also hold the UNESCO Chair in the Ethics of AI and Education, that's with the International Research Centre for AI, which is based in Slovenia, and a lot of my work involves working with UNESCO and the Council of Europe. Nice. Nice. Yes, to get started, then. So there's this term called "brainwrote," and it was named Word of the Year by the Oxford University Press in 2024, which describes a mental fog that can come with from endlessly consuming digital content as someone researching in AI in education. Do you think this is becoming more of a real issue for students today in the age of generative AI, what's the problem you see using generative AI in education nowadays? How long have we got? I think the reality is that we don't really know the impact of generative AI on people, let alone on students. It's still pretty early days. It's been used in multiple different ways, and I'm always astonished at how lots of ministries of education, for example, and school leadership are pushing, pushing, pushing to get generative AI into education, into schools. When, as I say, we don't really know the outcomes. I would argue very strongly that we don't have independent evidence of scale, that these tools are effective, that they do what they claim they do, or that they're safe. And so every time a ministry or a school chooses to push these into education, or essentially experimenting on children, and I think that's usually problematic. As an academic, I'm happy to experiment on children, but under rigorous ethical controls. But these tools are being pumped into education without any of that. Now I said we have no independent evidence of scale that they're effective or safe, but we are increasingly seeing evidence that these tools are being negative. So for example, a recent study from Turkey looked at the use of generative AI with students. They put the students into two groups. One group had access to the tool, the other group did not, and at the end of that period they discovered that actually those who had access to chat GPT did better than those who didn't, but then they took the tool away, and they discovered that after a few more weeks working, that the young people who never had access to the tool did better than those who did have access to the tool. So the inference from that is it might be a useful crutch in order to get you the marks you need. Yeah. But if you're interested in learning, it doesn't seem to be that helpful. And that's just one study. There are many others. Yeah. I feel like you have mentioned this actually in your research that there's a difference between learning and educations. Could you briefly go through the difference between learning and educations for us? Yeah. It's a good question. It's an important one. So when people mostly think about education, particularly in the West, but also in East Asia and other places, education has thought of as the process of learning. It's about we have classrooms and the teachers come in with books and the information is in the teacher's head or in the books, and the job of the teacher is to get that information into the student's head so that the students can regurgitate, that the end in a standardized test or whatever it might be. But that's actually a very narrow way of thinking about education. Dutch education psychologist called Gert Biesta. He describes three key functions of education. One of them he calls qualification and that's what we've been talking about. It's learning stuff. And that is what most people think. But the other two functions, one he calls individuation, which is about helping each individual student to become the best human being they can become. And the third is subjectification, which is about helping each individual student to understand and build upon their place in society, in the communities that they're a part of, so that they can contribute effectively. And if you think about education, that's what teachers are interested in. Yes, learning stuff is important, but so are the other things. Helping individuals become the best they want to be, in the spaces they want to be. Be it sport, be it computing, be it history. It doesn't matter. It's about helping them develop to become the best. But it's also the reason we're in schools, the reason we come together in education establishments is to make sure that these young people experience collaboration, experience, teamwork as you guys are doing today, so that you can build your role in society when you graduate and go further. Now, the problem is, and why this is important, is because all of the AI mediated tools that have been developed for education, they focus almost exclusively on that learning, that learning. And they don't even begin to address or even consider those other functions. I think that's a problem. Yes, I think the words you mentioned, individualization, or we can say, develop human subjectivity. It's not also very popular on the social media. We often say it is very important for a people to have human subjectivity and things like that. It is actually for me, it's kind of a broad and abstract words, if the question comes to you is that if you can give us a specific example in terms of what kind of behavior would demonstrate that person have the human subjectivity, what would you say, what kind of behavior, and why is it so important? We all make choices. We make choices in our personal lives, in our professional lives, in our social lives, and it's about enabling people to make choices that are consistent with their beliefs and views. It's not about us all doing the same, so it depends on the circumstance. That's why it's difficult to give you a specific example, but your decision to study what you're studying, that's about you being aware of the kind of questions that the subject brings up, and you're interested in those questions. You study a topic because you want to learn about it, you want to become familiar with it, you want to become expert in that area, so it's about you becoming you, we're all a bundle of stuff when we're born, when we're little children, and we have our hopes and dreams. As we grow older, as we develop, it's about helping those individual children to become well-rounded in control of themselves, and to be able to follow their own interests, their own dreams, in ways that help them become interesting people. Yeah. Well, it is really interesting, you mentioned the importance of finding your own interest, but do you think there's really a systematic way to help students find their interests, or it's like a spontaneous way, because when we are reading a profile, it is very interesting that from 1991 to 2013, you spend your life as a script writer and editor, and then you decided to do two PhD at the University of Oxford. So I guess there's kind of like switch interest between them, then would you say like it is more like a fate as spontaneous thing? It's a really good question, I mean, my background I wasn't as script writer or actor, I was a producer developer editor, but you're absolutely right, it's very different from the work I'm doing now, and in many ways the work I'm doing now feels to me, you know, with my obvious age, that I finally have come home, and so the work I'm doing these days for me is exactly allowing me to do what I believe in, and ways in which I can help. So, you know, for me, I'm interested in this space because I want to help young people and their teachers to be informed, to be empowered, to be able to make good decisions in this world, not for me to tell them what those decisions are, but to help them with that. And that's what I'm doing now, and I'm very, very, very happy about it, but for me it took a long time, you know, a lot of people don't realise this, and I was kicked out of school because I was a bad school student, and it took me a long time to get back, and so I kind of recognise that struggle from someone who didn't succeed, and now looking back on it, I can recognize. how important it was and how what a loss it was that I didn't achieve that then. So yeah, it's taken me decades to get to the point where I feel that I am an individual and I'm able to think and talk about things that make sense to me and to develop my interests. - I see. - But it took a long time for me to get there and I don't want young people to go through those struggles. - That's why you're trying to build a better environment for them like you've had a education system. - Yeah, I mean, I've been in education all my life, right? I've never left it, either as a teacher or a student. I mean, the odd week here or a week there, I wasn't, you know, for the most part. And I believe in education, I love education, I love the idea that we're helping young people to develop to become the best they can become but also to see their role in society. These are, I think, for me, are fantastic. And at the same time, the reason I love working in universities I get to meet young people who I learn so much. So it's not a finished thing. It's about continuing. And I think that, you know, back to that subjectification of the, so the individuation we talked about earlier, and it's not a, it's not a fixed thing. So, you know, when you're AT bank, you got it, you're now an individual person. It's an ongoing thing throughout life. But what education is partly about is helping young people to recognize that that's worth pursuing and helping them to pursue it and giving them the opportunities to pursue it. - Yeah, that's really interesting. I mean, like you mentioned a Dutch study and those three dimensions on standardized tests, subjectivity and individuality. But it feels like right now, schools and society are placing an overwhelmingly emphasis on passing these standardized tests, which is usually, well, like you said, a really narrow way of thinking about what education should actually be. I mean, what's really interesting is that we're already seeing some examples of this, like Gabriel Peterson, who's in high school dropout. He was then hired by OpenAI as a research scientist because he essentially times off a PhD level of machine learning through chat GPT. And regardless of how naturally talented he might be, does it raise a question for you that we are heading towards a future where former education might matter less if these two are making knowledge freely accessible to anyone who wants to learn it. - I can see why OpenAI might say that. It's nonsense. It's utter nonsense, you know, from the beginning. Problem is, say that young person was already somebody who had this interest. - Yeah. - He might have used the tool chat GPT to support him, but without his individual interest, and that would never have happened. And so it's always, you know, straightforward to find somebody who bucks the train. I'm not interested in individual people per se. I'm interested in large groups, you know, the student body of the USA, the student body of the UK and East Asia and wherever, right? It's not about individuals. And I think, I mean, Sam Altman has also said that he doesn't expect his own child to go to university. I think the sadness is for his own child, you know? So has Sam Altman been successful in what he's chosen to do? - Yep. How has he done that? In my opinion, he's done that by wall-to-wall lying throughout his career. And yeah, unfortunately in our world, people who are very skilled at lying often can make great progress in their own personal career. I don't like that, I'm sorry. I don't think anything that Sam Altman says about life is worth listening to. He's a. He's so. - Already an expert in his fields? - Yeah, what's his field? His field is making a business and conning people into giving him money. He's very successful. So I wouldn't take any lessons from those people at all and this notion that in the future we won't need education. He's nonsense. And I can take you back. Have you ever heard of Thomas Edison? - Yes. I invented the light bulb. Many other things as well. Well, something that's not known about Thomas Edison's he declared very, very strongly. Then in the future we won't need schools because everybody would learn everything they needed from film, which was the new technology of the time. - Yeah. - Yeah, it was obviously nonsense. - Yeah. - And go back to the 1950s. You've probably heard of the psychologist B.F. Skinner. He invented a machine that he said would mean we don't need schools. It was wrong. So the idea that we don't need schools and we won't need education, we won't need universities in the future is just wrong. It's nonsense. It's the kind of myth that they like to perpetuate 'cause you don't need school. You just need our product. I wouldn't believe a word they say. - Okay, that's a really nicely sort of like a note to have when people like go out there again and listen to this type of narrative. Something definitely to be. But it's slightly because they have the power. They have money. They have the ear of government. It's hugely problematic that they have so much power in today's society, but we have no democratic control over them. - Yeah, it's just, it sounds very interesting that you have mentioned when it comes to the invention of new technology, they're always trying to do like a revolution in the field of educations. Like why is education such a field that they want to pursue? Why is it all about education? - It's a beautiful question. The answer is because they know education. They're experts. Why? 'Cause they went to school. And they think because they went to school, they understand education. But they don't, you know, their picture, their window on education as a student is a tiny fraction of what education's about. The biggest tech company, Khan Academy in the world, has received, you know, $600 million of grant funding, money given to them to do with what they chose. It all started because Sal Khan did a couple of videos for his nephews because they wanted to understand something about maths. So he did a couple of videos. They lied to them, ooh, he then manages to convince. And this is what these people are very clever at. They're convincing people with money to give them money. So at the beginning, Sal Khan managed to convince Bill Gates to give him $100 million. That's why it's a company, nothing to do with what they do. And that company has pivoted so many times to where they are today. So they're not, you know, they don't understand education. They really don't. And what's interesting, you know, picking on that one company is there was a study published just a couple of months ago. Two of the authors were from Khan Academy. And I won't bore you with the stats, but basically the study showed firstly that students on average only use Khan Academy for 11 minutes a week. And in so doing, the stats demonstrated clearly they learned very little. Nothing worth talking about at all. And then they said, well, look, if we project it to say the students used it for 30 minutes a week, even the outcomes of that that they were projecting were tiny. So these tools, as I said, no evidence that they do anything useful at all. And that was based on 200,000 students' worth of data and at least two people from inside the company. And with all of that, they still weren't able to even predict if those pesky students didn't use it for 11 minutes a day, how naughty are they? But that's used a week, sorry. But that's used it for 30 minutes a week. Even that prediction was weak and pathetic. Yeah, then why is the use when we use generative AI in the process of learning that we didn't really acquire the knowledge? Well, it's referred to as cognitive offloading. So instead of thinking about it yourself, you get the tool to generate something. So I have a slightly later rival here today because one of my students from a country in East Asia wanted to discuss, because they are developing AI literacy work for their teachers and students. And she told me, said, well, in our country, the teachers are encouraged to use these tools for developing reports, for developing lesson plans, et cetera. And it saves them time. And I know, OK, well, I'm not convinced it saves them time, but OK, that's a shame it does. What happens if the tool goes away? Will they still be able to do that? Are these individual teachers developing the skills that are necessary for them to be able to judge whether the outcomes of the tools are worth having? And we're also seeing at the same time in the world of coding that young people who are coming into coding into big companies are so dependent on the coding generative AI tools that when something goes wrong, they don't know how to fix it. They can't stand the logic of it. So it's this cognitive workflow, not bothering to study. Now, you know, right? You've got to where you are. You know that actually studying anything, but whatever reason, whatever outcomes, is hard work. And if you don't put effort in, it's not going to happen. So using generative AI, it might get you to the finishing post quicker. But it doesn't necessarily mean it's going to help you learn. learn what you need to learn. Let me just give you a quick metaphor. This is a pictorial one but I think it works well. So the question is why do people go to the gym? To get fitter. To get fitter, right? And when they're out the gym, one of the activities they do is they lift weights. Yeah. And that's all about getting fitter. Yeah. So the equivalent of using genitive AR for writing your essay or helping write your essay is going into a gym with one of those machines, the fault lift truck that can lift things. Yeah. You get them in warehouses. And so the person will go into the into the gym and use the fault lift truck to lift the weights. So the question is are they successfully lifting weights? Yes they are. Are they getting fit? Not really? No, they're not. Their little finger maybe but that's about it. And that's the same kind of thing in education. So are they getting the immediate results? I have an essay. Are they learning? Are they preparing themselves and benefiting from this experience? Yeah. So that their futures are more interesting and exciting? I don't think so. That's really interesting. You have highlighted a lot of the problematic reasons why we should try and avoid it. But I mean since the introductory of like the first model of chat GPT in 2024 has been about two years or something. I would say it's still like pretty deeply embedded in just students I guess even for teachers or for people who use it in work as well. Wouldn't it be really hard to try and reverse what has been done already? I'm enjoying this conversation. That's another good question. So when I grow up we're in London right and you've probably used the tubes at the underground train. We will use them. Cinemas, airplanes, when I grew up in each of those situations people smoked cigarettes were everywhere right the floor was covered with cigarette butts. You got into a tube train and you stepped into an astray and it was a fog of smoke. It was horrible. But as a young person I'm thinking well yeah this is it. This is the life I have to lead. It just seemed unthinkable, impossible. That could ever be got rid of. And here we are now. You don't smoke. You can't smoke in the underground. You can't smoke in restaurants. You can't smoke on airplanes. So that magical future that I thought would never happen did. So we should never think that just because something looks embedded that it's here forever it depends what we do with it, how we react to it and what we think. And what we're finding is so there's a studies from lots of people from MIT, the Washington Post published something just last week where they're looking at companies using these tools and companies that invested huge amounts of money, tens of billions of dollars in the implementation of these tools and their business. And what they are finding is the MIT studies showed that these companies were getting zero return on their investments on AI and what the Washington Post, the American GDP and also had zero improvement because of these tools. So you know business leaders are going to wake up one day and say well why are we spending all this money on these tools that there's no return. So clearly these things do happen in different ways. And a story I read the other day about you think about computers when computers first arrived some company spent a lot of money got computers in but they didn't see any returns. So maybe in decades these tools might bring a return but at the moment they're definitely not. So don't think that these tools are here and here forever. It's up to us to decide how we want to use them and whether we want to use them and maybe reject them. It sounds like it is really important to have like proper regulation and policies and like that. That will be a really interesting topic for us to discuss later. And I have a question let's go back to you mentioned the cognitive offloadings like the loss of friction in the process of learning. I've read an really interesting article on the Guardian saying that so far as technology has developed. There's a lot of kind of friction and offloading cognitive offloading happened so far. For instance we are the foods online now we don't order in person and also we kind of Google a thing instead of trying to drag it out of our memories. Yes these are all kinds of frictions but do we need to keep them all like what really is the boundary between the healthy cognitive offloading that make our life more convenient and versus the kind of cognitive offloading that will degrades our competence. I have an iPhone and iPad and Mac I love them. I use them all the time. I find them helpful and enjoyable to use. So I'm not a games technology and I think you're right there are times when offloading can be helpful but we're talking about or we've been focusing on education and so that's very different what you might do at home when you're having fun or whatever when you're trying to learn something that's when the cognitive offloading really becomes important and I think I mean all you know we the reason we use any technology because we want to offload some effort you know that's the point and sometimes the technology helps us do things that we couldn't do without the technology that's also true. So this is not about being a games technology and about offloading effort to technology. I'm not against that at all but we're talking as I about education and I think that's where it becomes really really sharp. So it's about like the special thing about education because education is not about measuring how many efforts we have put in or like the standard result is about wider things. That's why using generative AI and cognitive offloading sounds so dangerous in the circumstances there. Yes and I think it's interesting to use the word dangerous because I think it is these tools are literally dangerous as well. One of the ways what thing we have we see a lot is the anthropomorphisation of the technology so using human sounding words. Now we've already lost the battle it's called artificial intelligence but they're not intelligence. There is no correlation between what these tools are capable of doing and what humans are capable of doing. So it's the wrong word and it's misleading. We have other words, machine learning. Again these tools they don't learn like we learn. Another one we hear of the time hallucination right? Well they don't hallucinate in the way we hallucinate but they do generate nonsense. So the problem of this the reason that this happens or the reason of reason is just computer science is talking to computer science. It's very reasonable in each domain we have our language and that's fine but the point is that this technology has escaped the computer science lab. It's out there it's in everything we do every time we walk down the street there's face recognition cameras and I'm used your phone you go online to buy stuff you go on the tube train everywhere everywhere and so it's no longer their domain and the problem of allowing those human words to be used it means a lot of people misunderstand the reality and kind of feel that they are engaging with something very human-like and they're not. The appearance the companies work so hard to make it appear that we're working with human but we're not and so we've seen lots of problems there and the danger comes in here where young people around the world for various reasons are self-harming committing suicide because of their over-assumption that these tools are human and their engagement with these tools have become so intimate sharing fears and hopes and loves and all the stuff that we share with people suddenly being shared with these machines and when the machines go wrong I do something different it's really problematic and we have lots of examples of that happening around the world and we have examples of these tools encouraging people to buy and use more drugs for example who end up dying through that conflict so they are dangerous you know and so coming back you know you mentioned the regulation so I think regulation is fundamentally important and we don't have any so if if you want to use a drug and you're an ordinary person you're not a mad person let's leave those out you're an ordinary person you want to have a drug you're not feeling well you want someone to give you something's going to help you feel well and the drugs that you would get access to are drugs that have gone through a robust process of trials evidence checking to make sure it both does what it says it does it's effective but also that it's safe yeah right and then apart from the really simple ones you would typically only get a drug from an expert a professional a doctor who says look I see your condition this is the drug I recommend we know it's safe because of these trials that's how we do it right we don't do that yeah with education technologies and AI mediated education technology you read about it on the news or you go to an exhibition you hear about it or you somebody tells you about it in the pub then you think okay that sounds interesting I use it with my students but again without any evidence of that effectiveness or safety I think there's speaking of the policy there's an interesting thing you mentioned in your research is that OECD are funded to include the measurements of AI literacy into the piece assessment in 2025 and you said that there's potential risk associated with it. Can you help us do go through what kind of risk are associated with this? Yeah, it depends what you mean by AI literacy. This is the problem. AI literacy is complicated. Recently I contributed to a report where we looked at AI literacy frameworks around the world and we identified 115 of them. But the vast majority of them all emerged from three original ones and the vast majority of them are grounded in these ones that are developed by computer scientists. Now I like computer scientists but their expertise is in computer science. Their expertise is not in ethics in humanities in social sciences all that stuff. And so the problem is that what they do is their focus is for me is very wrong. So one of my students pointed out to me that AI literacy shouldn't be about helping people to use AI. It should be about helping people to thrive and flourish in a world where AI is impacting. That's a very different approach, right? Yeah. But the vast majority of these frameworks they're on the form. It's about how do we use it? So in work I'm doing with the Council of Europe we divide AI into three dimensions. So the technological dimension, the practical dimension to human dimension. The practical dimension is about how you use it but it's more about how can you make informed decisions about whether to use it, let alone how to use it. But if you decide how to use it then how do you use it reliably safely, responsibly effectively and so and so forth. So that's one dimension. The other dimension and this is what most of these frameworks they mentioned focus on is the technological dimension. What is AI, how does it work, what's an algorithm, what does machine learning mean, etc, etc. But the third dimension is the human dimension and for this we mean the the role of humans in AI in its development, in its control, the big companies, etc. The fact that AI emerge from military research, the fact that it's being used in military conflicts today as we speak. It's impacts on human rights, on child rights, it's impact on the environment, it's impact on relationships within the classroom and outside the classroom, it's impact on the deep fakes and the lies we're seeing in the news, etc. So this is hugely important and it also comes down to who is AI, literally four. So the first ones, they're really for people who want to become computer scientists or AI developers or data science whatever. I'm not saying those people shouldn't be supported, they should be supported, but that's not AI literacy, that's teaching in that domain. AI literacy as I say is for everybody, but for me the focus has got to be on that human dimension and that's the problem. So when I say that I worry about the OECD, they five months ago published with the EU what they called the AI lit framework and that is principally about the technological dimension. It was developed by computer scientists. Now of course they mention ethics, of course they do and they mention privacy and things like that, of course they do because that's what they all talk about, but they don't mention any of the things I mentioned just a couple of minutes ago. Yeah that's a very comprehensive understanding of AI literacy because I think privacy when we talked about it, we weren't very clear about what AI literacy should be meant. Yeah as I said I think it's that key sentence that my student gave me that I'm now I've tweaked it but I'm using it a lot you know it's that AI literacy is not about helping people to use AI tools, it's about helping people to thrive and flourish in a world what AI is increasingly impacting and I think that's what we should be doing. So I've never ever said to any of my students don't use these tools, I would not do that. It's not what it's about but equally I'm not interested in train them how to use these tools either. I mean I think in fact you know this writing a prompt thing is fascinating because so much effort goes into it and yet we're told the tools are pretty intelligent but you know if they were so intelligent why do we need to put so much effort into writing a prompt you know when you use when you work with Siri on your iPhone or Alexa or whatever other thing you might use you have to change the way you speak yeah this conversation we're having you couldn't have one of these tools they wouldn't get it it would collapse right yeah but why do we have to change if these tools are so damn smart which they're not but if they were then they should be fitting around us and they're not yeah the computer science they will hang on a second you know you'll talk about now these tools are quite new this is true and all the problems are going to be sorted quite soon but don't buy that either you know all innovation start off that's what makes them innovations but if you look at the history of innovations they then do that they flatten out and that's what we've seen so the difference between no chat GBT and the first chat GBT we got access to was massive and frankly you know what the computer engineers have done to build these tools is incredible yeah so impressive but the difference between that first chat GBT and what we now have is incremental it's tiny and they still have fixed the fundamental problems because the fundamental problems emerge from the way they actually work so they're not going to be solved anytime soon there's a lot of work out there and looking at how AI can be developed further yeah and they're saying well you know the the large language models the chat GBT that it's a dead end and lots of the leading AI researchers in the world are now coming out and saying this they weren't saying this two years ago but they're saying it now it's a dead end and they need to be something else yeah so I'm not predicting what they'll be in 10 years time maybe that will be able to do all the things I'm pointing out they can't do now but I'm talking about it now yeah the problem we have about regulation is that the people who are doing the regulation don't really understand the background I've been telling you about either so in the European Union they released the EU AI Act somewhat two years ago and they have to stop the development of that why because chat GBT arrived and they haven't taken to account that they didn't understand it and their definition of AI that they use there the OECD uses even the Council of Europe uses it's not very helpful it's a very technical definition and it makes no reference whatsoever to the role of humans but humans wrote the algorithms humans trained the systems humans are impacted by the systems so regulators need to be AI literature as well and they are not but particularly in that human dimension yeah and they need to understand this stuff and yeah it's really tough yeah yeah then I feel like the problem you have mentioned so far is that since the rise of chat GBT it has been two years we have seen many rise of productivity was like cloud urban clouds so far things like that but yet we haven't seen one proper AI education tours what has cost us is it because the lack of consciousness that you have mentioned I think there are hundreds and hundreds of tools designed for education that have AI technologies built in hundreds today as we speak there are schools around London that are using these tools in not good ways because they see them as a way of saving teacher time as a way of personalising the things I've said they don't do yeah I think that's problematic but it's not just the schools for the teachers for you know the policy makers are at fault as well the DFE the Department for Education here in the UK are constantly releasing stuff about Jones of AI which constantly points out to me that they don't know what they're talking about you know they see as a task it's this fear of missing out if we don't do it other countries will we must do it really quickly ourselves otherwise well as pointed out that that's not necessarily quite as terribly things they don't focus on the thriving and flourishing that I've talked about and I think that's a major mistake is that because we are so dependent on what we have already and is that why we don't go towards those off-year tools I think there's lots of things I think first it's the ignorance ignorance and I think also it's the apparent convenience yeah and it's also the push the push from industry yeah who are constantly making these over confident claims that are just not true but they're pushing pushing all the time so I think you know it's all that kind of stuff that's that's getting into that meaning that so many governments and ministries are pushing it so hard yeah but it's not just there you know we're from UCL right and I've been involved in committees at UCL talking about this and my colleagues might hate me for saying this but the problem is like we were saying before about education everyone's an expert in education well today everyone's an expert in Jones of AI because they use it and they can see the wonderful things it does but they are not experts and it shocks me how an institution like UCL which is all about helping people develop expertise has no time for expertise in this field they're more interested in people's day-to-day experiences of it they don't understand any of the background and that's hugely problematic. And the situation across UCL is very complicated. We have a set of rules. The rules are not very well understood, they're not very well implemented. It's a very messy situation. It's not just UCL, it's other universities as well, but it's just the university, I'm based that's why I'm mentioning it. And I think it's usually worrying. You know, we need to depend upon expert insight. I'm not saying that the experts, the people that have a good understanding, a broad understanding of these technologies should make all the decisions. But those who don't should at least listen to those people. And that's not happening. The decisions are being made based on, well, I use chat GBT to write my emails and it suits me fine. - Yeah, in this case, then, as a policymaker, what can we do? What kind of regulation and policy can we adopt to solve this kind of problem, the people's ignorance to see towards the dangers of AI and all the sort of things you have mentioned? - I think it's a difficult one. So the council of you, we are doing exactly this, we are developing a legal instrument to regulate the use of AI in education, which obviously is called gender AI. The idea is not to stop people using the tools, right? The legislation is not about that. But it's like I was saying with health, before a school or a ministry decide, yes, we're going to roll this out, is it not reasonable to expect them to have robust evidence that these tools are safe and effective? Well, I think it is. Rather than what something I read about, what I found or somebody showed me, we don't do that with drugs. And why are we doing it with these tools that by definition are supposed to help learning, and are supposed therefore to develop human brains. So a developing human brain is not as important as our physical bodies. - Yeah. - That makes no sense to me. So I think that's what the legislation is about. It's about ensuring that the governments take proper, controls proper understandings of what these technologies can and can't do. So I don't think our legislation is ever going to say to a government, you must not use this tool. It's never going to say that. It's more likely to say, you as a ministry or a government whatever, must have the evidence that make you confident that this is safe and effective. Now, then it's up to them to work out what that means, but I think that's the approach. And one of the other side students today was saying to me, well, the problem with this is too much regulation, it undermines innovation, we need innovation. Well, that's a false logic that's been perpetrated by industry for years. It's never been shown. And two things there. Firstly, if we're right with this legislation, if we're successful with this legislation that we're talking about in the counts of Europe and they say it's still a long way to go, what it would mean is that an EdTech company based in the UK, if they tick all the legislative boxes, could sell without any problem to 45 other countries with no further legislative problems hurdles to get over. So that's how's that bad for innovation, right? With suddenly massively increasing their market. At the moment, that's not the case. But the other thing is, you say metaphor is the pharmacist's going to increase again. You know, the pharmaceutical industry is probably the most regulated industry in the world. It's probably the most profitable, or one of the most profitable industries in the world. It's probably the most innovative in the world. So why do we say, therefore, that regulation stops innovation, not happening pharmaceuticals? Now, I'm not saying pharmaceuticals is the way to go because there are lots of problems there. But yeah, it's not straightforward. So you mentioned this idea of trialing EdTech tools before they hit the market. It's almost like how new drugs go through phase one, two, three, four, et cetera. Like you have given in the example right now before they get approved and pushed out to the market. Could we apply a similar logic to educational technology? Like if we put tools through multiple rounds of evaluation like that, what would some of the key criteria look like? I want to ask you for a checklist. - Yeah, I mean, firstly, we've been doing that for years in the development of curriculum and textbooks. So curriculum takes years to develop and people disagree and argue and debate and develop and refine. It's an ongoing process. It doesn't just happen overnight. And textbooks, right? I'm not arguing for a textbook, but textbooks typically go through many stages of development to make sure that what's going into a classroom is true accurate pedagogically sound, et cetera, et cetera. But yeah, when we get to ed to it, none of that's important apparently. So the kind of things I was saying before it's about, can we be certain that this tool is safe, that it's not going to undermine children's development mental health, that it's not anthropomorphizing. It doesn't mislead children into thinking that they're working with a human in some sense. Is there robust evidence that this actually does what it says it does? So it's not wasting the children's time, you know? So there are things like that. - So when it comes to that pilot stage before rolling out a tool to more widely across different schools or different countries, who do you think we should actually be testing it with first, like which students or under what context should be the starting point for this? - This is long been a problem. So one of the questions that's asked, if we have a tool and we want to do a proper experiment a control group, a treatment group, and we give it to the treatment group, other control groups suffering because they haven't got access to this lovely tool. Well that's kind of put in the, so they have ways of doing it where they swap it over. So the same children will then do the other way of doing it. But I think we need to go through these processes. And I think we shouldn't assume that these tools are good. There's the precautionary principle. We should assume that these tools are not good until we have evidence that they are good. - Okay, it is very interesting that you mentioned, you don't think it is likely for a government to say, let's bend this AI tool. But we might see something similar is that the Australian government, they bend the usage of Instagram for kids, is this kind of education policy sounds like a normal things for you because they completely bend the usage of one app. And let's do say, I don't think anything is definitely bad, we're definitely good. Then how can government evaluate the cost and benefits, for instance, in Instagram that they decided to just bend it? - Well, it's, to be fair to the Australians, it's all social media with some small exceptions rather than just Instagram. But you're right, what you're saying. You know, how can they make that decision? And they're not alone. And the British government are looking at this at the moment and making decisions as to whether they'll do the same here. I'm not arguing for that at all. As I say, I've never stopped any student of mine using it. But it's about, and also we must distinguish between the use in education and use outside. Outside of education, we do have rules. So for example, we have rules about the age of which you're allowed to smoke, the age of which you're allowed to drink alcohol, the age of which you're allowed to get married, the age of which you're allowed to drive a car. So we do have these rules. This is not unheard of. And we do this because we recognize that there's requires a certain level of maturity before you can make informed decisions about what you do. So I think, you know, pulling social media into that is not massively different in many ways. But we're talking about that. That's what's happening outside of education. And that's an important issue, et cetera. But that's not what I focus on. I focus on in education. And in education, it's very different because children don't have a say about going to school. They have to go to school, right? They also don't have a say about what they do in the classroom. Right? They are told what to do in the classroom by their teachers. But we're now getting to a situation when they're being told what to do in their classrooms by engineers sitting in Silicon Valley, you're disempowering the teachers to make proper choices. So I think that's when it becomes problematic. And the teachers need the support. For a teacher to say now, I'm not going to use this talk. I don't know, I don't feel right to me. It's very difficult for the teacher. But for the teacher to say, I'm not using this talk because there is no evidence that it is effective or safe as the rules say, then that's much easier. So that's putting the power back in the hands of teachers. And so I think this is, I'm not a big, we need to regulate everything kind of a person. But I think there are some fundamental things. I'm more interested in not regulating the users, but regulating the developers and the providers of the systems. That's where the weight should lay. And people say, you're being unfair by putting this weight on the shoulders of the teachers. And so I say, well, that's why we're arguing for AI literacy. I'm not saying teachers need to be AI literate so that they are able to know whether it's good or bad because they can't know I'm saying I believe they should be AI-literate and be supported and helped to be AI-literate, as I've explained it, so that they can make those informed judgments about what they're being told. But still, the pressure should be on the developers and the government. The teachers just need to be in a position where they are better able to continue to be the professionals they are. Yeah. You have mentioned that, yes, the regulation are very important, but you said there's a long way to go. So, what are the friction that we're facing so far for pushing those policies? Well, legislation itself takes forever. So, in September 2023, there was a conference, a regular conference in, I'm held in Strasbourg, which is the seat of the Council of Europe, and you had around the table 46 ministers of education from 46 countries or their representatives. And the resolution put before them was that the Council of Europe should develop the legal instrument and also develop AI literacy. So, that was three years ago, almost. Three years ago. Two and a half years ago. Wow. And since then, we have been slowly but surely developing this stuff, way that legislation works is you need to come up with the idea of what we're legislating. You then need to work out, well, what does that actually mean? How do we write that down? What do we try and represent in that? And some it like, you know, any parliamentary system, it goes back and forward. So, they bring in the experts and one of that team and we write something, we hand it over to their committee. Our job is to then develop to the next stage, based on what they've said. And so, that is a long process. And we're at the stage on the AI literacy, where in the next few months it will go back to the committee and hopefully they will agree it. And then hopefully that will then be slowly filtered out to the countries, at least they will have something to focus on if we're lucky. Because it could be rejected by the members of, by the member states. But if we are fortunate and we are successful, then it goes out to the countries. And then as a regulation agreed by the four to six member states, the member states then have to implement it. But they don't have to implement it overnight, they will have a period one, two years for them to implement it. So, it's a long process. But yeah. It would be exciting to see how they implement it and into the each and individual system because there's so many cultural differences in everything in different countries. Completely. This is part of the problem. So, during the development, you try and address as many of those issues as you can. So, you know, if it's said once in this recommendation, then it must be localised. It says it several times, right? Because this is fundamental. It's not about telling individual states how to do their business. It's about trying to give support in this complex area. And one of the other things we're doing with this is we're saying, look, you know, the Council of Europe in their work together have been looking at education for 50 years, right? They have lots of recommendation about education. And what we're doing does not change any event at all. Equally, for the past 10 years, they've done a lot of work on AI. And we're saying what we're doing again, it doesn't change any of that. What we're looking at is where these two clash, where they come together. And that hasn't been looked at in detail, and that's why this is so important. And for me, and I said at the beginning, I like being, you know, a teacher in the university, if we are successful, however far we get with this, I will feel really good that I've helped with this process. I mean, a small part. But if I've helped with this process, and we get it to the point where it is implemented, then I will know that we have protected millions of young people across Europe. I quite like that. But I think the problem is that we're living in a lawless world when it comes to the use of AI in education. And you think about, you go back in health, go back 150 years, there were, you know, cracks as they called them on every corner, who would do all sorts of amazing things, you know, put leeches on you and cut you into all sorts, give you snake oil as they call it. That was very common. But over the past 150 years, it's been heavily, heavily regulated and standardized, and it's not perfect by a long way. But it's an awful lot better than it was 200 years ago. But we're still in that early stages of education technology. Yeah, I feel like this conversation has so far has really inspired me in terms of the dangerous when the technology runs before the regulation and the importance of their illiteracy so far. So then I think we're basically done with a technology part that we come down to the life advice part. Yeah, a small one. Light advice part for the young generation from your experience. So you have mentioned that the experience of you actually in a young age getting kicking out of the score and then you manage to do two masters in one PhD, which still is a really long year in academia fields. What really happens to you, what really changes to you than from a kid's who may be kind of traumatized by standard education towards someone who really devils into the academia field. It was slow and there was no, you know, one morning I woke up at all. But I fell into teaching early. So I did, as I say, I fell out of, I got kicked out of school, but then I went to what we call when the UK further education. So a college and between the age of 16 and 18. And I kind of recovered a bit of my ambitions there and that's when I got into the the film. And then I fell into the teaching of film and photography and had fun and just built and built and built the teaching. What did they do to make you recover the ambitions at that film? I think I just realised that how important education was. And you know, I'm not being critical of my particular teachers. I'm sure they did their best under difficult circumstances, but it just helped me recognise that education is not simple and people are not simple and young people are not simple. You know, the moment a child walks through the door of a school, think of all the baggage they've got. You know, the chubbles they might have at home, their relationships with their siblings, their friends, lots of children go to school in the UK without having breakfast because the families don't have the money for it. It's a complex, complex thing and we have other students who, you know, have really comfortable background. We have homes that don't have things called books and other houses. They have lots of books, right? So it's a very complicated thing. And I think for me, it was just that recognition of how important it was and how exciting it was and my pleasure is talking with students because I learned so much all the time. I had a comment teaching this earlier today. I'm teaching later and I know that I will, oh, I haven't thought of that. And I really enjoy that. That's part of what makes me tick. Well, then you again emphasise on the idea of the importance of interest, finding what you're really interested in is really important. But again, you sound like a really resilient people and you're really good at dealing with uncertainty, kind of, you recovered from your early, let's my say, failure in education, things like that. If you're to give advice to the young in terms of how to deal with uncertainty, what would you say when you're not clear of where your life is heading to? But you sound how got the interest. I'm hesitating only because it's a really, really interesting question. So as I mentioned at the beginning, my title that I chose is Professor of Critical Studies of AI and Education. If you speak to my friends and family, I've always been the question, the person who asked the question, why? My mother would say, you always say, why? Just do it. I said, why? And I think that's what it is. So I've always had an interest in trying to understand things and trying to move forward. And I've always taken a critical perspective, but I didn't have that label before. And for me, I kind of see that it's all kind of come home in the sense that that criticality that I've always had bubbling under in my world and my head has a very clear focus. Because I've always been interested in technology as well. And when I started, you know, when I started my PhD, which wasn't that long ago, I was a huge enthusiast. This is amazing. I want to do this. And it's only by researching it that I've become critical. Yeah. I see. I feel like uncertainty is rising, especially on the AI tools. Like, students now don't even know if their job is going to be replaced in the future. Yeah, I mean, I think the time we're at now is in my life the most uncertain time I've ever experienced in terms of, you know, what does it mean to be in school and the impact of these technologies? What does it mean for careers and where direction I'm going to go? You know, a few years ago and the recommendations, you know, study data science because you always have a career. No, you won't. You know, that's really under threat. So we have all that. We also have the uncertainty in the political world, the military world, you know, around the world what's happening today. It's hugely, hugely difficult at my own side. constantly blames me as representative of my generation for the situation he finds himself in today and I kind of get that. Yeah, I think if we get back to those ideas though about being literate, not just in AI, but literate in the world, understanding and being able to question that and having some understanding of how the world actually works, not how the world appears to work. I think is hugely important and that it's only by that getting those skills and that awareness of those different issues, then hopefully young people can move forward, but my hope, I put a lot of weight on the shoulders of my students in the sense that so one of the groups of students from Australia, we're interviewed by their teacher and they came up with some really interesting ideas that we're dealing with AI slop, that this slop actually is, it takes an awful lot of effort to use it properly and is it really worth that effort, you know, we'd be better off without it, and that the slop is actually being perpetrated by demagogues of ideologues, Elon Musk, Sam Orman, right, and what does that mean for society? And the fact that these tools are being used them to prop up militaries and then to kill jobs, this is the reality we have at the moment, and my heart, you know, I'm lucky at the end of my career, my heart goes out to anybody who's trying to get kickstart into a career now because it's really hard and I completely accept that, but I think building those literacy skills and those critical skills is the best thing you can do because who knows what's going to happen in three years time, but if you've got those skills you're just going to be better able to, not adjust to it, but to deal with it to turn it to make sense for you, so yeah, you sure, an awful lot of it, yeah.