Using Generative AI in Education with Dr. Philippa Hardman

Sometimes we kind of underestimate our role in like how human AI is or how much it relies upon the human in order for it to be effective. And in my mind, AI is a tool in our hands rather than a replacement. And I think it will mean that jobs are lost. I think it will mean that jobs change for sure. But whether or not that is a negative thing, I don't know. Welcome to Trending in Education. This is Mike Bommer. I am joined today by Dr. Philippa Hardman, who is doing a lot of really interesting work around artificial intelligence and learning science, data science, leaning into some of the new tools that are emerging. She's a great follow on LinkedIn. We're going to talk more about her background in a minute before we do any of that. Philippa, welcome to Trending in Education. Thanks so much, Mike. It's so nice to be here. And I'm excited to talk about learning science, which I think is something that we don't talk about enough, but has some really interesting and exciting implications, particularly when we think about it in line with emerging AI technologies. Absolutely. I always think of it as a both end, in a lot of these cases, where you need to have skills and expertise and competencies on the harder digital technical skills. But then at the same time, you need to complement that with the human durable skills. And a lot of that involves learning about learning and understanding how humans think and how human behavior relates to all of this. I was really excited to discover your work where you're really hitting on all of those things. Can you catch folks up to start just on who you are and how you got to this point in your professional life? Yeah, sure. I think I'm an unusual hybrid kind of beast. So I'm an academic by training. I worked out of the University of Sheffield in Cambridge in the UK. I did a little bit of time working out of Harvard too. And by training I'm a historian, but just so happens that as I was doing my PhD at Sheffield, Sheffield was a real great centre for digital humanity. So thinking about how we can use technology to increase access to and the quality of human knowledge. And I was really lucky to be involved in a couple of huge early projects. So there's a project called The Old Baby Proceedings online and one called Plebeian Lives, where we move huge volumes of historical records online so that for the first time we could both increase access to it, but also do new exciting things with it. So start to collate that data into new themes, understandings. And so what happened really is I started to transition more and more out of academia. I'm still an affiliated scholar at Cambridge, but I'm really interested in how we can use technology to increase both access to really great learning content and learning experiences, but also use technology. And this is where I think there's huge potential and technologies who really so far failed to disrupt education. But the big question I'm pursuing at the moment is how can we use technology to deliver better learning experiences? And one thing I've learnt very clearly over the 20 odd years that I've been in education is that we do understand what the formulas are for brilliant learning experiences. I would define that as a learning experience that is both motivating. So as a learner you really want to do it. This is exciting, it's relevant, it has real world value. I've got a North Star that I'm aiming for and it feels like this is the way for me to get there. But also it's designed very intentionally to develop my mastery, to make sure that I understand the right things in the right way, but also can apply them and then use that to be creative and original. And I think, as I say, we know a lot from learning science research, so research around how the human brain works, how humans behave, how they're motivated, to be able to design really great learning experiences. But for some reason, and I'm very happy to talk about this, because this is what I'm fascinated by, it just hasn't infiltrated. There's this gap fundamentally between what we know about how humans learn and how we design learning experiences. And I would say that's agnostic of mode. So quite often we say online learning isn't as good as in class learning, but fundamentally if the pedagogy is wrong, both of those things are equally bad and all of the research shows that the mode matters less than the pedagogy than the science that's underpinning it. So yeah, that's my area of research. I've now transitioned mostly out of higher ed. I've been in the leadership teams of a couple of ed tech companies, and I've recently founded my own company called Don's. My mission is to make it easy to apply the science of learning to the art of learning design. You're someone who's really bridging anywhere from a leadership team who's trying to reimagine their learning strategy and think about how they use AI, what's their AI strategy, what are the humans relate, what's the future technology there, all the way over to someone who's an instructional designer, a learning professional, who's a teacher, someone who's rolling up their sleeves and designing materials on a day-to-day basis. Your audience really spans all of those contexts. Yeah, absolutely. The reason that it does is that fundamentally, a great learning experience is a great learning experience. And what I've been able to do is to distill hundreds of pieces of learning science research, like what is motivating, what leads to mastery, and to develop an evidence-based process, which I'm finding has value at all of those different levels and in all those different contexts that you just talked about. So yeah, I work in higher ed, in L&D, with individuals, with huge organizations, as you say. And I think fundamentally what the process is that we need to, rather than designing the learning experience that we want to design, fundamentally what we need to think about is who is our learner, what's motivating them, because if they're not motivated, basically everything is futile. You can build a beautiful thing over here, but if they don't want it, they don't want it. And I'm talking here both about people who might be buying a course, but also in the higher ed context, it's really important. I think sometimes we take for granted that while you're here and you've paid, so you'll do this, whereas actually we found that if we apply the science of motivation to those kind of experiences, people persist, they hang around more, they're more engaged, they get significantly better results. And I was lucky enough when I worked at an ed tech called Aula, to be able to lead a huge project there with the University of Coventry. And what we found is that when we applied learning science principles to the way that we design 1200 modules there in 12 weeks, it was a very big, very rapid project. We saw a significant positive impact on motivation, engagement, on student satisfaction scores, student completion scores. And yeah, I mean, they had more perfect educator feedback scores than ever. And so it's just about understanding what do the learners want and need, and then what are the principles that we need to apply based on their motivations on the subject that we're trying to teach and on what we're trying to get out of the end. So we're trying to drive understanding of a concept or we're trying to deliver skills. That then impacts the pedagogy that we select and that then impacts what the learning experience is. And by following that process of like discovery, writing objectives, mapping out the experience, and then storyboarding it all out in detail, that's the DOM's process, we optimize for both mastery and motivation with some really quite impressive results. I like thinking about both the technical skills and the human durable skills where data science is one area that I know you've been highlighting as an area to get conversant in. The other is learning science as you were describing. And then the last really is more around the tools that are emerging. How do I tap into breakthroughs like chat GPT and other emerging technology to get better at delivering on those objectives. But it's a place where best case the human is intentional about how she's engaging with the tools that are out there. You're not just using chat GPT as the voice of God, where whatever it says is obviously correct and or is done. Instead, it is the beginning of a process where when done right, using learning science and all the principles you're describing, the human is actually in charge. I love to roll this quote out, but it's very important that people often ask me what's the biggest danger of AI in education. And it is that we become better and faster at designing really bad learning experiences, which is a very real risk. And already I see when people talk about AI, in the context of education, what they tend to talk about is how we can very quickly generate video, for example. So content creation processes are going to be made more efficient. Now that, of course, could be a brilliant thing. But I think more fundamentally what we need to get right in the first place is to think about what is the content and what's the purpose of that content. And so what I'm interested in more than those kind of content building tools, which I think I'll get to later in my process is fundamental like right now, how can I use tools like chat GPT, for example, to design a better experience to apply pedagogical principles so that the content that I create is the right mode, the right length covers the right kind of stuff. And so I'm much more interested in trying to raise awareness around this big part of the learning experience that's been quite included for many years. When we think about learning experience, we often think, right, what's the content? And if you think about LMS or a content authoring tool, these are the tools of the learning designer, whereas in fact, that's at best like the middle. Actually, it should be toward the end of the process. The beginning is this process where we need to understand what we're trying to do and how to apply the science of learning to optimize our plan before we build the thing. For me, it's all about making sure that we don't just start to use AI. We underestimate the power of technology. I think we've done this repeatedly with education, where we've used it to make ourselves faster, to make things easier, but to reproduce what is fundamentally a knowledge transfer pedagogy, where it's like, here's a load of content, and I'm going to ask you to regurgitate it. So that might be a lecture in a physical lecture hall, or it might be a MOOC with video and then a quiz, but it's fundamentally the same pedagogy underneath. And I think for me, the most exciting thing about chat GPT, and for many, I know also like the most scary, is that it fundamentally challenges this knowledge transfer pedagogy, because we're now in a situation where if I set one of my students an essay question, they can use chat GPT to generate that. And I've done some really interesting tests where I can upload a mark scheme from the University of Cambridge and say, can you please write a response to this essay at a first class level, according to that mark scheme that I just uploaded, and it can do it brilliantly. I can then say, make it a bit smarter, make it a bit shorter, add these footnotes. I mean, I understand why that's scary, because it means that fundamentally we need to change how we think about assessment, teaching, learning at all levels. But what's exciting is that we've always promised, particularly in higher education, but just generally, that the education is about generating skills like innovative thinking, original thinking, creativity, the skills that really do develop the economy and make people happy and fulfill human beings. And I think what chat GPT is doing is saying, well, we've automated now those bits that are just about being able to recall and regurgitate something. And so now, we all have to think a little bit more creatively about the learning process and think, well, actually, how do we use those tools to develop, recall, understanding, whatnot, but then actually use the learning experience to push us to those higher order processes around being able to apply something, to critique it, evaluate it, and then use it to create something original. And so it's almost like chat GPT is going to make us deliver on a promise that we've been making for decades, if not hundreds of years, that education is there not to reinforce what we already know and establish power structures, but actually to encourage people to be creative, innovative, original thinkers. Yeah, and ideally, there will be a lot of positive disruption, but disruption nonetheless, and it does bring the question of what does it mean to be human more front and center? What are the things that add the most value as a human is another way to think about it as well, where, you know, for me, I worked in test prep for many years, and I did a lot of writing, instructional writing for Kaplan when I worked there. And a lot of that writing was a grind, you know, I had to generate a word count or you're writing a reading passage, not the most intellectually stimulating experience when you're doing hundreds of them as a human, as opposed to designing a curriculum or thinking about adaptivities, you know, testing an algorithm that's actually out there, those are things that are much more intellectually engaging. And I think that's really the challenge for us is to think about work where we're doing higher yield, more intellectually challenging, more creatively challenging work, more of our time, and then perhaps working less, there is almost a, maybe it's slightly utopian, but there is a world where folks are doing more engaging creative work 20 hours a week, three days, you know, out and off a more flexible schedule. But when combined with AI, they are generating more value. And I think that's particularly true in learning context, there's been a lot of sausage making and instructional design, there's been a lot of like, let's make sure we write some more learning objectives, we're now chat GPT is just going to fuel the engine to some extent. I mean, you've been doing a lot of this, you know, active exploration of what chat GPT is good at and where it faces challenges, specifically around learning type work, there's a post we'll share of yours that's getting a lot of attention about some thought exercises and things you've been working on. Can you catch us up a little bit on how that's been going? Yeah, it's really interesting. And I totally agree that, again, just to go back to that analogy of like the research associate or teaching help AI is something that can make us all more like able to focus on the things that matter about our work. And I appreciate that not everybody's work has that element to it, but perhaps it should. And perhaps this is going to be liberating. And I think a really great example of this is what's happening in medicine. So the world of medicine is a little bit further ahead than other industries when it comes to technology. And what we're seeing there is not like the training or the employment of fewer doctors. But we're seeing doctors being able to focus more and more on, as you say, the high value items. And that is things like checking anomalies. So a machine like AI can, I think, better than a human now we found, for example, review scans to find problems, whatever. But it's then down to the human to go through that kind of condensed curated list and to look at things in more detail, be able to engage more with that individual, etc, etc. And also more doctors are being freed up to conduct the research that then forms how the machine like scans a scan. And it's the same in learning design. So I think what we will see is a shift from learning designers spending hours building content, making videos, you know, those kind of things will become automated. And I think that's a liberating thing. But they will still be absolutely critical. And this is where my research has been focused without expertise. We don't know what video to build very quickly. Same in higher ed, educators, professors, I certainly experienced this firsthand, we get so bogged down with things like grading. And there's already some really interesting AI driven tools that have been around now for more than a decade that have been able to automate an amount of that. And I think we'll see more and more there. But again, that's liberating the professors to spend more time with the students to spend more time researching and this kind of thing. And yeah, I mean, what I've found so far using chat GPT is that it is powerful, but only with the right prompts. And so if I just give a very simple example, but if I type in, you know, write me some learning objectives for a course that six weeks on the theory of gravity for 10 year olds, it very confidently gives me a list of learning objectives, which to be fair, are pretty solid. I would say that they were probably better than average compared with my experience of my instructional designers and academics. And that's not to criticize them. It's just to say that it's like a tiny part of a job that's quite powerful. So they're not able to specialize in it. So there's value I'd immediately. But what I've found is that if you understand how the AI works a little bit, you're able to coach it from that initial response through to something that is like as good as an objective can get. So we know from research that like the best objectives are, I'll just give you some examples, like achievable within one hour are directly addressed to the learner. So they're in the first person, they include a verb and they include a very clear description on what they will do and why they will do it. There's basically a formula. And by putting in that formula and saying to chat GPT, please write me a set of objectives using the following criteria, it can very rapidly create objectives which are exceptional. And so that's where it gets exciting. But it's another example of where on its own, it is relatively powerful. But with expert domain knowledge added to it, it's doubly powerful. There are some versions of chat GPT that are being developed right now have been developed already in the medical world. So there's a biomedical science example where its domain knowledge is very up to date and very specific. So I think essentially they've fed it abstract from biomedical science journals. There's an interesting implications there potentially. But definitely so far, my experimentation has shown very clearly that it can be coached to generate great evidence-based learning design, but the human needs to understand what that looks like and what those formulas are. Yeah, and the related idea, taking a step back, I do think there is some systems thinking and design thinking, understanding of workflows that humans will need to get better at so that we can, as individuals, as members of teams, as leaders, we can think about these things I want to hold on to as things that I will have humans do. These things are actually better served using these emerging tools. And then maybe there's a blend, you know, maybe to your point, maybe part of what I want my humans to do is to be training our own conversational AI that will be specific to our domain, feeding it just the data that we want. To me, it is a mindset thing where if you're coming at it as the designer of a system, who may also be a participant in it, but that way of understanding seems to me to be very closely tied to how you think about applying your system across the different contexts, whether you're leading a learning organization, you're working in higher ed, you're an individual who's trying to monetize your course, you're a course designer. Each of those stakeholders, each of those personas is going to want to engage in the broader system in a different way. But there is a level of understanding of how the system is going to work and how AI will factor in and how the humans will factor in. We're at an aha moment where if you can be in a role where you're tapping into that, in some ways you'll be on the right side of this revolution. Yeah, absolutely. And I think this concept of technically it's called like an intelligent tutoring system. So I think there are lots of ways, and this is where I've been focusing so far to think about, well, how could we use AI to make it because both more effective and more efficient at what we already do. So what we already do being designing classes that are delivered by professors online in the flesh hybrid, whatever. But I think there is a world in which if we zoom out a little bit, as you say, we've got the potential to rethink how learning happens. And one thing that I find very interesting is that, you know, 30 odd years now, more of really robust research into learning science, kind of through that, there's a definite scene which emerges, which is that learning happens, regardless of who you are, what you're learning, what age, whatever, like there's a thread that runs through out. And that says that learning happens through a combination of experience and dialogue. And within that, there will be an amount of content, but it's actually like, it's effectively thinking that like, coaching session is much more effective than a lecture. So if I give you a problem and coach you through it, you take a problem based approach, I don't give you like a lecture up front, I just set you something to do, and then I help you through it through a combination of our dialogue and just sending you the right information at the right time. That's the dream in a way, like if I could build anything, I would build that because that's where learning happens. And so it's interesting to think about what that might look like in the future, you know, that's less a learning platform populated by a load of content and more a dialogue based interface, where maybe even it's totally feasible, we could do it tomorrow with the right resources, we generate content on the fly according to what happens in the process of learning. And the other really interesting thing related to that, I think, is assessment. And so at the moment, all we tend to do in the learning experience, wherever it happens, is we do a thing and then we assess it at the end. And we might have formative assessment where we like, we see how we're going. But we always have these moments of like stop and assess. And we only really assess outputs, because they're kind of easier to measure. So like, you know, the output being an essay and then a grade. But I think one thing that the AI will enable us to do is to kind of forget the stop and assess thing. And we're going to be able to assess students, learners, humans, like in the process of learning. So I'll be able to see, for example, the methods that you took to solve the problem, the amount of time that it took you to get there, the number of different pieces of content, and prompts that I had to create in order for you to hit that goal. And all the time I'm learning all the time, like what does Mike, for example, prefer? What are the highest leverage interventions within this experience? And so we're collecting data, but we're having more of a real experience. And as learning experience is driven by dialogue, more than it is content and examination. I think again, that applies across all of the different contexts that we're talking about for all those different people. And we've already gotten intelligent tutoring systems happening in the world, like Georgia Tech University have been developing there. Jill Watson, yeah. Forever. And it's interesting. And again, we've not seen, you know, Georgia Tech getting rid of professors, but we have seen potentially an increased efficiency of those professors who use technology like that and increased ability to help those students who are falling behind or to do more research or to develop new learning experiences. So I think when we think about the impact of AI and education, it's important to look at innovations that have been happening over the last 10, 20, actually AI ed has been around for about 30 years. So, you know, I think there's lots of lessons to be learned from that, including that we shouldn't panic necessarily about being taken over by the bots. Right. Although it is also perhaps a Gutenberg moment where what was previously only accessible to a select few is now widely available to all of us in which case, you know, post the Gutenberg Bible and the printing press literacy rates went through the roof. So it does seem like we're at a stage where being able to make things with AI as a new set of skills and competencies that are real and they're here today. That's where for me, the most important thing at this point in time is to just get your hands dirty and get used to making and the messiness of making and the complexity and then also doing so critically in that there are a lot of biases and perhaps too passive mindset when engaging with these tools comes with more risk. We haven't gotten that dystopian yet and I know my listeners like when we get into some of the darker scenarios will end with a ray of hope at least. Where could this stuff break bad? For me, I'm mostly concerned that there's some of us out there who are being trained to be obedient and conformist and when the confident but wrong AI tells us something, we're going to start doing things. What do you see out there on the horizon? Where might there be some risk? Yeah, it's a great question. I think there is real risk to get very dystopian and a little bit scary but we know that AI has killed people. The allocation of medical treatments, the prioritization of those treatments was based on algorithms which were fundamentally biased as we are towards white people and that led to increased risk and actual death for people of color. We should be very, very aware that AI is dangerous, that it is as flawed as we are as humans, that it is inevitably biased because we have built it and therefore that we need to be very intentional about recognizing that. This takes me back to my history classes at school, it's like a photograph is as biased as a painting. We need to make sure that we understand that there is a level of, as you say, criticality to this and I think if I could put mistake in the ground and say one thing would you do tomorrow to get the most from AI, it would be to make sure that everybody gets really great education about some of the risks and the mitigations of those risks, what we might need to do in order to ensure that the AI is developed in a way that is not biased because we could choose to do that but then there are huge philosophical questions about who decides what is the right way, what is not biased and all of those things. Yeah, in schooling systems, again, the allocation of additional support, I think it was a case from a New York school that was using AI and again, in a similar way to the medical scenario, it was just reproducing biases towards certain types of underperformance which were specific to certain racial groups compared with others and this kind of thing and so we need to be very aware of this for sure to introduce a ray of hope. Again, AI is not new, we have some really great ethical frameworks already in place for AI in education specifically, things around student data protection, things which restrict by law, the purposes to which we can put AI and also provide recommendations for how to overcome biases in the data that we generate and analyze. So, it's a real risk and I think, as I say, one of the most important things as far as I'm concerned, particularly in AI in the world of education, is to make sure that people are very critical consumers of AI and realize that there's a lot of potential risk for them, for their students, if we don't approach it with a critical mind. But again, ray of hope, silver lining, the World Economic Forum tells us that one of the key skills in order for the economy to survive is critical thinking and so for me, rather than thinking in terms of banning them, getting kids back in a room to do an exam so we know they've not used chat GPT, what if we instead use tools like that to, on one level, develop critical thinking. So, asking students to generate two different answers to the same question and investigate where the differences came from and which one is more reliable than the other and why. But at the same time, also teach them about if they're using chat GPT to do those kind of activities, it does this job of AI education, of making sure that they understand this thing is so dependent on input and that some of the outputs are biased just in the same way that it would be if you asked somebody out on the street. Yeah, it reminds me of the point you made at the top about learning science where a lot of the breakthroughs in behavioral economics and places where we know humans natural tendencies are not rational, they come with filters that are pointing us towards making decisions in ways that aren't always optimized. It is interesting to start thinking about intentionally designing to counter some of those things, leveraging some of these new tools. I still feel like we're all going to be traveling about with virtual familiar at some point. We're going to have little pets who we train up, customize, and they're going to be nice versions of chat GPT. And to your point, I feel like we're going to have to allow them into class with kids ultimately, especially if one of the ideas is that education prepares you for a world of work. One of the challenges we face post higher ed now is to ramp people up on the skills they actually need to do their job. What if we start building those things into our educational experiences sooner? Lots of interesting questions on the horizon here for us. Phil, we do want to get to your closing remarks as we wrap up. Hopefully we'll get you back on to continue this conversation. I does feel like this stuff's going to continue to be happening. You're also delivering workshops, a great follow on LinkedIn courses. If you're interested in what Philippa is doing, we'll be sharing all of that through the show notes. But as we conclude here, I always like to give guests a chance for closing remarks. Use it however you'd like, but it's been great having you on. Thanks again for joining. Thanks, Mike. I think as a closing remark, definitely just to repeat this point that education has always failed to be disrupted by technology, repeatedly, repeatedly, repeatedly. The example of the MOOC, for example, it was going to be the future, it was going to be disrupted, it was going to change everything, but it didn't. And I think there's a really interesting question here around why it didn't, how AI might be different, might be more interesting. And I think for me, the difference with AI is that, well, it invites us, it gives us the opportunity if we want to take it to, as I say, think about education less as a process of recall and regurgitation. So tell me back what I taught you, and I will give you an A to actually delivering on this promise to encourage, to teach people, to be able to think independently, innovatively and creatively. And I think given how dependent the economy is on that, there's a lot of force in that direction, it'll be interesting to see what happens. The other thing I'd love to just underline is that the way that we design learning experiences, regardless of whether it's in an L and D, HR department, an Ivy League university, you know, K-12 school, the way that we as humans design learning experiences is broken. And it's broken because the science of learning is impenetrable, it's expensive, it's effectively locked away behind both a paywall and then like an ivory tower of coded conversation. And I think one thing that's really, really exciting for me is that AI offers us an opportunity to use that to open up access to that understanding. We can get more access to that expertise now because the technology can help us get there. But also because it liberates people who are at the moment maybe too distracted by things like making content or marking exams, to spend the time instead understanding actually what are the mechanics of how people learn and how do I apply that to this learning experience. And for me, that's why this is potentially much more disruptive than the MOOC for example, which was opening up access to something that we already were doing. So that was a transmission project. This is more fundamentally a rethinking potentially of what and how and why we teach. And that is like incredibly exciting. Yeah, we may be in the midst of a revolution in which case it's time to wake up and pay attention because you don't want to be caught on the wrong side of history here. Really interesting stuff. Dr. Philippa Hardman, great follow on LinkedIn, folks whose interest is peaked. You'll have more to chew on if you go to the show page. Philippa, thank you so much for joining us on today's show. Thank you. It's been great to chat with you. Awesome. Hopefully our listeners enjoyed what you heard. If you did, please subscribe, write a review, do all the good things. We'll be back again soon. This is Trending in Education.