11. AI and Assessments: When Students Ask "Does This Sound Like Me?"

[Music] Hello and welcome back to the AI Ethics Now podcast. My name is Tom Richey and I'm really pleased today to be joined by Dr. Charna Gonzalez, Senior Electro at Marketing and Program Director for the MSC International Marketing at Kings Business School at Kings College of London. Charna's research examines how generative AI is transforming critical thinking and academic writing in higher education. Her recent work explores what she calls epistemic offloading, the process by which we're increasingly delegating reasoning and justification to AI tools like chat GPT. She's exploring what this means for authorship, legitimacy and agency in learning. Charna, hello. Hello, thanks for having me Tom. No, great, how you doing? I'm good, thank you. I'm really excited to be here to have this conversation. Great, thank you. Well, let's get started then. What brings you to this conversation on AI Ethics, please? So I've been using chat GPT since the day it became available and I really wanted to be part of the discussion about how people ask academics, the public, but also especially students, how they were actually using generative AI in practice, rather than how we imagine they would use it. So I think the conversation about AI Ethics is, you know, it's very multi-dimensional. There are a lot of aspects for us to consider, but I really wanted to be part of the conversation in being able to translate what students are saying they're doing as opposed to what we think is happening in working groups, where they're very separated from the reality of it. Yeah, I think that's really interesting. And we've done previous podcasts with students where it's almost like the secret life of students with AI, where we make a lot of assumptions, I think, as academics. When the reality is that we know students are using these tools in ways that we perhaps don't think about, haven't considered. And I'm also aware that for lots of students, they are slightly nervous, AI explainers, and so they don't want to say they've used it. And so it makes it really difficult with things like assessment design, with course design, not to make sure that students aren't using it, but to actually help them use it in an effective way, and to help them develop that critical AI literacy, I suppose. And on that point, your work speaks about the concept of epistemic offload. I wonder if you could explain what that means and why you think it matters when we're using tools, like chat GPT for academic writing or assessments. I think just linking to your previous question before I explain that. I think one of the things that I was aware of is that a lot of our discussions do kind of center on the idea of detection and misconduct. And I wanted to show a little bit more of the more complex reality of how people were using it. So this idea of epistemic offloading kind of refers to the parts of the reasoning work that students are now offloading to the tools, not just the effort to have AI generate a piece of text for you, but how much of the thinking about what you're doing are you delegating to the AI. And I wanted to understand where students felt that they needed to do that thinking themselves, versus where they thought it was appropriate for AI to take those reasoning decisions on their behalf. And some of what I found through interviews of students were them saying that they're using AI to not just summarise reading, but also entire curriculum, for example. So when they have comprehension activity, they might have been given an academic article to read. And if you comprehension questions, they are using AI to decide which parts are actually appropriate to give in response to those questions. So it's now not just the summarisation effort. They were using it to generate argument structures and, you know, they'll outline the essays and pieces of course work. And again, allowing the AI to decide what might be valuable to submit every edit piece of that piece of work, identifying key points that they should focus on and almost pre-valuate in the quality of their draft as well. So I wanted to focus on this idea of academic offloading because what I saw was that this is shifting the response of the initials of the calling for intellectual work from the student. And the students have accepted that AI has become a part of their reasoning process, not just the convenience tool. They were also discussing how they consider what counts as intellectual efforts or what do they think is appropriate in response to the task that we set them. And I thought actually that is why this matter is looking, we don't know what students think is important for them to do themselves. We're making a lot of assumptions about what we want them to do themselves and why we think that's important, according to our curriculum design, our accreditation, the way our programs are benchmarked in terms of intellectual levels, for example, but that doesn't translate to the students often, they're not privy to a lot of that information. And they see it very differently. So that's kind of where this idea of academic offloading came from. And I thought it was important to really dig more deeply because there's a huge goal between these two sides on that on that topic. Yeah, absolutely. And I wonder, is there a point where you think offloading becomes problematic, or do you think it's more how it's done by students, that people are finding problematic? No, I think it can be problematic for various reasons. I think one of the most obvious point is that sometimes, you know, as academics as instructors, we want the students to go through certain processes in order to learn. We want them to be able to make connections, we want them to achieve specific goals. And if they're skipping most, it's an ultimately we haven't achieved our intended outcome. So that is problematic. I think the other thing that's problematic is that we don't explicitly tell students they have to go through. And AI has kind of revealed that if students skip these steps, again, we don't achieve our outcomes, but it also means that we're now not sure of what we're not achieving from the academic side. So I think there are those reasons. And of course, for the outcomes of students learning, you know, we don't know which of those steps are detrimental for them to skip. We've not yet had the chance to evaluate that. So I do see offloading as problematic, but I think at this early stage, I think it's more important that we try to understand it. And it is a very complex topic. So I think there are so many different ways that we could look into it in order to better understand it. And then maybe we could have a more informed conversation about when and why it's problematic. And I think we as academics really need to consider students are using this, whether or not they're going to tell us, we know they are because it's a tool that helps them release some of the pressure that is put on through. Yeah. Of course, it's being very expensive. The needs to succeed at any cost. And the kind of systemic problem academia seems to have with the concept of failure. And then we frame it as a really bad thing. So I get why students are using it. And you argue in your work that AI tools are changing, not just how we write, but how we think and how we take responsibility for what we produce. Could you share some examples of what that shift looks like in practice? Yes. So students conveyed in some of my interviews that they're increasingly treating AI explanations as provisional interpretations that they should build on. And sometimes they think deeply into them, but other times they accept those uncritically. So that I think that is a very clear example of it changing how we think because if you had to do all of that work, maybe you'd be making all of those decisions by yourself. Then some of the students were explaining that they do rely on AI generated outlines, for example, as the backbone of their assignments. And that shapes how they conceptualize a topic or what they are likely to go back to and refer to in terms of the material that was taught. So I felt this was quite interesting because we could give students 10 weeks of lecture content and accompanying reading, but actually the way that they pull that together mentally is now shaped by AI. So if AI put emphasis on certain aspects of those 10 weeks as opposed to others, we get a different outcome. Then actually one of the really interesting things with students saying that they'll give their draft AI and ask for feedback on the kind of grade that AI would suggest they might get according to their rubrics or the instructions that they've been given. And they would allow teachers to be see to tell them what's mixed in. And I thought this was interesting because actually it now shifts if I'm with the judgment. So how students understand what they think is good or what they think is an appropriate level to sit this way I can. And actually the machine is playing a huge role in that, not just the human. So I think those are the three things that stood out to me. There was this one other aspect of students describing, losing their sense of authorship. They would allow AI to rewrite an entire section of the draft, for example. And they would prompt it with questions, you know, like, does this sound like me? And they were conveying in the interviews that they were asking themselves, who wrote this? Did I write this or did the AI? Is it still my own work? And that kind of spoke to taking responsibility for the work, but then they said if it didn't feel enough like there weren't words. Actually, they know not necessarily what they were doing, but there was different paths where they thought that that was more or less appropriate. And this is kind of where I came to decide that actually the way we're thinking about what we write as well as taking responsibility, which is determined by a lot of the factors when you're going through this process and working iteratively with AI. And I think particularly your point about how it changes the way that students evaluate what's good is a really interesting and challenging outcome that perhaps we don't think about. It is that an AI will read a rubric in a certain way that perhaps we who written it don't mean in that way or maybe overemphasis is something and so students are then submitting things that are not quite what we thought. And then they are then surprised because they're thinking, "Well, the AI said I'd get it first." Yes. And then they don't. And so that can lead to a number of challenges, I think, in how students' satisfaction, I suppose, can be affected because they, if you trust the AI is telling you the truth in quotes, you think, "Well, I will get it first because it's what has told me. I'll get the top marks." I think that's a really important point, actually. I do. I think, you know, without sort of taking us on a version, I think academic understanding of how these large language models work is also important there. There was a really nice paper by Margaret Berman and colleagues. She wrote this paper, "A Development Designative Judgment in a Time as AI." And I think having seen that work, that was quite early. I have understood from colleagues actually because they don't necessarily understand how these models process take. They don't necessarily understand why an AI would ban certain aspects of writing over a matter. Now recently we were marking master's participation. And I had said a lot of discussion with my students about what they thought was appropriate for being AI versions of their rewrite. This is what they think I've been asking for. And one of the things that came out was that because AI prizes polish over content a lot of the time. So when the AI is as a co-author, it does interpret different aspects of the rewrite can be more important because those are things that need to better polish, not necessarily better intellectual debt, they'll better be content. And I think, you know, there is a lot more that we could explore going forward to understand, you know, how AI is interpreting our re-bricks and how that leads into the skill or development of a variety of God-worn among the students. And I think particularly that there is a challenge there with encouraging staff to explore them. And it's not, I think, the staff are unwilling most of the time. I think it's the lack of workload space to actually really sit and understand these things. It would take, you know, people a long, long time. And as someone who works in this space, I don't fully understand how they work all the time. And why they pick what they pick and the kind of the black box nature of AI tools makes it very, very difficult to kind of completely understand that inner workings. I agree. But I do think a few more of these, just very simple conversations, these kind of corridor conversations about what we've observed, what we think is strange, what a intrigue. I think a lot more of that would be helpful to move than I can say things forward, actually. And it's interesting being in the US at the moment, but also having worked in the UK is you can almost see some of the reasons why it doesn't doesn't happen where in the UK there's a much greater apprehension about speaking about this. It's almost like AI is a dirty word that we don't necessarily want to talk about because people are afraid that if you're seen to be using it, then that's a bad thing as a staff member. Whereas here in the US, the universities that I've engaged with while being here have had a lot clearer sense of policy around it and a lot clearer approach. It doesn't necessarily always translate to staff, but you're seeing that staff are at least more open to discuss it. It doesn't mean they want to use it, but they're open to the idea of it. And I think that is a real difference I'm seeing between the UK and US higher education landscape. In the UK, it feels very reactive, and in the US, it feels a lot more proactive and that people are wanting to plan how they can use this in, whereas in the UK, it's this thing that we're not quite talking about yet, and though it's definitely an issue and we know it's come up. You mentioned your work, the concept of epistemic orchestration. So this is the idea of learning to delegate question and reauthor AI contributions, which I think is something that we're also seeing from students where they feel like if they can just change a few words, take out the word dwell, take out the word delve, take the ZZ out, take the M dashes out, that they feel like that's then reauthoring. Can you walk us through what true kind of epistemic orchestration looks like as a process, please? So I'd say this idea is not pretty in government, right? Again, I'd imagine from a lot of this data. It involves for contingent iteration. So first of all, what is it that the student thinks is that are okay to delegate? A lot of what they suggested so far, you know, anything that seems very routine on the counter core, where they feel that they've returned conceptual controls over the work, but actually getting out and execute this productively. Then the second part of that is it does involve some interrogation. So they check him for the logic and the actual thought as well as the alignment of the output with the task as they had understood it. Then the reauthoring aspect where they're considering whether it is reflect your voice and the voice that they would like to portray in this given submission and how that might be interpreted by the reader, but also the argument that they think is appropriate to put forward. So the reauthoring part is really the tone of voice as well as the interpretation of the argument. And then with drawing or avoiding some use of AI in certain aspects, so if they think that the task is not appropriate for AI to be, they will resist using it or avoid using it. Or if they think the task really requires personal insight and use them, like reflective activitions, for example, then they will rethink. So this orchestration is like, you know, going through these four stages of asking all of the questions about is it appropriate to delegate? Can AI hand them it? What I might have to be the output? Do I need to change it? Or should I just avoid it completely? And you also in your research speak about the impact of AI on critical thinking, you link to Bloom's taxonomy. And I wonder based on what you've just said, what have you found about how AI is affecting these different levels of cognitive processes when these decisions are being made? I think the data tells us quite clearly that students are regularly offloading what we consider to be those lower level tasks according to Bloom's taxonomy. So anything that just requires some general knowledge, understanding, conveying that through text, so where they're just paraphrasing and extracting the key ideas, they're quite happy to upload the lower level Bloom's taxonomy tasks. And at the middle levels though, they're using AI in slightly different ways. So even at the young level of understanding, they're using AI to help them understand. They're in constant breakdown, pieces of text or bigger ideas, conceptual ideas for the student, and then using it for application by helping them to interpret their understanding according to the task, carefully, and again, like, propulsion and who work in the structures that they're sort of experimenting with. And then we get to the higher levels. Now I did one specific study which students were doing a coursework project. So they had to come up with a new brand, they had to come up with the brand an after-under product and then design a strategy around it. They were using AI at every thing. So I think it's better to say that this was quite a creative task. It had a number of steps and a number of subcomponents as well. And at those higher levels, the students work in doing that. They find ways to use it at every stage. So it could help them brainstorm, it could help them evaluate various ideas before settling on the one that they're most happy with. But they also feel quite a protective instinct around what they're happy to share as part of their submission. So they're using AI, but they weren't completely relying on it, which was in fact and so my argument there was that it can be hard to create a thing because they're using it in the rise, rather than simply setting the output and using it in their work. But the thing that becomes concerning in that that's higher levels is whether those students are willing to offer those judgments and using a model and in many cases that is not what we want them to be even if it's not as necessary. So I don't think it eliminates people thinking and I think Rune's text on in these days gives us a really easy framework that many of us have familiar with across the spectrum, but it does also indicate where it might be changing how that goes with the pattern and the students. And Chania, you'll work also examines how risk perception and text having this influencing students' adoption of using Genai in assessments. Are there any patterns you're seeing and are there any equity implications that we should be more concerned about in terms of our more text-heavy students getting an unfair advantage? I think this is hard to want with regards to an unfair advantage and I feel like I have the caveat that I keep marker seeing the way practitioner goes with the saying that they expect students to be using AI. So the equity discussion goes both ways. Are we the inequitable of we withhold that experience from the students that we prepare and then for equal chances that a graduate job? I think this is again a more complex discussion. I think there are people that we've found from that particular project that students who've believed themselves could be more than text-heavy, but means they should be using AI and they go into use it with or without permission because they don't want to feel that they are behind their competition when they go out and work in well. And I think this is really a cause to academics to think about how this claimed out in their particular area to be most used to a business subject, maybe you're in fact, maybe going to be very different from those teaching humanities, but a board of social science object. But I think actually the expertise of academics is required here to answer this question about what types of each students might do in that context. I think also then we can ask the question about equity. So how does that look across those types of students that we can and how does it look in terms of their graduate reality? I think for us the equity implications, it really suggests that we should be case-or-ending to all students' potential user AI. I think it was better for us to foster this transparent discussion so that those who feel that they don't want to use it or have reasons that they can't are able to surface, but it's a menu we can think about the very specific situations that we need to address, whether that's institutional, sort of infrastructure that needs to be put in place, maybe accessibility considerations, whatever it is that I would say at the moment you don't feel like you know what those are and therefore can't address them in a reasonable way. At the moment the students are the best people to tell us, but actually we are largely designing our education for the masses, we're pursuing with our large cohort within the care use. So I think we should be designing under the assumption that all students are at least going to be using free burdens of these authors, access is equitable and then if there are other ways that we can make it more equitable we go from that, we'll move forward from that case life. It's a really important point I think on this idea that we need to be catering that all students are going to be using at least the free versions of these tools and you've mentioned new work about this need for a transparent and reflexive use of AI rather than an outright ban and I think that's a really important distinction that again speaking to colleagues in the UK and working with others there, there is a feeling that we should just ban AI, just ban it outright, seems to come up so often in conversation. But I wonder if you could speak a little bit more on what reflexive AI use would look like in practice and how we can make thinking more visible and accountable when AI is involved. I think for both students and the academic reflexive AI with name, articulating why we've used AI at maybe a given stage in a process or for a task as a law, what use chose to delegate and why, how you detect and re-aussure the output, what maybe critical criteria where you hold in the output to what we think was important before you saw the output and did its feature and also then which decisions remain yours. So I think there's this why, what, how and then which decisions do you think are important to use the free version? Maybe because AI wasn't capable of delivering on those but maybe there are other and broader contextual reasons why some of that decision-making has to remain our own. So I think that sort of reflexive use would be a slower poster in some way. At least I think it would require the very important reflection and interest action on how we make decision post, you know, post event or poster. I think also then it would be easier for us to explain what the process was that we went through and what we have now as academics is this idea that we can assess this process, not product but we don't really know how to do that. So I think we need to go through this transition stage where both as as the academic particularly those of us who are sections, task and assignment. I think we need to go through this process and we can hear how students have gone through it and then we can understand where the thinking had become visible, maybe the part that we also looked for the part that we underestimated. I think then, you know, having the opportunity to review people's rationality, I did that those would be fairly sure but even this idea of talking through something that you've done, saying you can explain what new thought was important. I think that would be helpful on both sides. Maybe annotating things, I'm not a fan of this ideal submitting expensive lists of prompts and reflective logs, I'm not a fan of that, but the reality is in all regions of the world, we're already over wealth in higher education with assessment and, you know, not there, sort of the admin path that accompanied that. So I wouldn't, I always try to encourage a sort of approach where maybe we have a feeling it's a conversation with students, maybe we capture that in a recorded fashion so we can have a transcript, but you know, that wouldn't take up a huge amount of time, it just took us to sample for example. Or have these tech points where students made their approach maybe a quarter of a class that they've been sitting where I've seen what have you done up until now, how do you approach it? But they're a number of ways that we could go through trying to understand that reflective process. And that would move us a little bit more from trying to police them to cultivate in this more, I wouldn't say mature and accountable, sort of communication around AI. And I also think for academics it would, it would encourage their movement of the conversation from to group to to more normalised. Some of the things that I have a cross about for more risk of our students are very hesitant to try and also to communicate what they've done. Partly because they don't really understand our integrity norms and our assessment norms, they're very concerned about this information that needs to get them. So they might restrict their road use, but they're also even if they don't, they're just not going to tell us. And I think academics have an important role to play in fostering this conversation at the class really level where they really are the decisional teams and the gate teams to grade. But also then in heart, in their own literacy, through hearing what students are doing and feeding in that. Well, I don't want you to move it for that part of the talk because I'm trying to teach you how to be X or I would really like to see those things process so we can diagnose where you get stuck. There are so many reasons that are completely legitimate that I think the reflective use requires us to appreciate the academics that use a lot more and foreground it and then for people to be having more of this kind of faith when at the ground level. And it feels like that context is sometimes lost in teaching. So on the modules I teach on I ask students to when they submit an assignment to do an AI declaration and just say what you used and a couple of sentences on why you used it for that essentially. But I think this conversation between staff and students is missing where we don't probably explain why we don't use AI. We know you can, but we don't want you to and here's why. Here's the skill set we want you to test or develop or engage with in some way. And I think it's that that's the discussion for me that's missing quite a lot of the time. And I appreciate what you're saying about this kind of more reflexive use, but for both students and academics the only is not on students here all the time. It needs to be on us thinking about why are we teaching what we teach, why are we assessing it in this way and what are we hoping to get. And this bigger question about the impact of Gen AI on assessment is going to continue. But with all of that said I wonder, China, what steps do you think we can take to ensure a positive future with AI please? I do feel like realistically the answer to this has to be quite short term because things are changing so fast. But I think we can experiment with it a lot more. I think we can be more positive about the use of AI if we understand it better. And I think actually back in 2023 when this first came about we had fairly small but important proportion of our colleagues. We thought, oh, you know, I put my assessment into chat to the team just to see what it would do when I was intrigued. I think we still need more of that. I think we can enhance the progress of the CEO of future with AI if we have a better understanding of what we think is important, why we are trying to maintain certain human skills. And if we know where the barriers are because AI can do things reasonably then we can try to address them. And I don't think we have clarity about what we're trying to address right now. I think teaching AI literacy in a way that doesn't just center on tool use and trying to keep up with the hundreds of tools that become available and updated on a weekly basis. I think it would be better for us to think about this in the in the bigger sense of how our judgment is changing, how our supervision of tools is changing, why we create things in a certain way and why it's important for that to continue to happen particularly among students. So I think it needs to be a bigger discussion. And those people who offer more philosophical perspectives who offer the bigger picture perspective actually I think they're opinion is very important at this stage. I think supporting students to learn how to use AI as a system. I think we talk about this around these metaphors very often but there's very little training that is being offered in that way. And actually I think saying to somebody right a poem with Katt-T-T is not particularly helpful at this stage anymore. I think there needs to be a little bit more discussion about how for example a colleague might allow students to use Katt-T-T in class and what they would do in Katt-T-T in our generates all of the answers they had for the discussion that they were expecting to host. You know what they do now. How do they overcome those challenges? I think that would eliminate a little bit of the fear around engaging with it. Yeah and I couldn't agree more, Charna. I think particularly around the point of when we talk about AI literacy we need to stop talking about it is the ability to understand the different tools that exist because they change so quickly that we would never ever be able to keep up. And your point about the judgment that we bring to using those tools I think that's a much more stable ground to begin having these discussions on about why we're using them and as teachers what are we wanting us to get from using them or not in different contexts. And I think that's a much more productive conversation than well, Chatt-T-T does X but Gemini does Y and Claude does Zed and that's why we have to use Gemini. You know that that find but they change so quickly that it's very very difficult to keep up with with what's going on and I think that in itself could be a full-time job because I don't know about you every time I look at LinkedIn or I go online and look at AI articles there's always something you know drastically new that's completely changed particularly now with the rise of a gentick AI and the impact that's going to have as well. But thank you so much Charna. It's been amazing to speak to you and really really good to learn more about your research and what you're doing. I'm excited to learn more and maybe have a chat with you at some point in future about where your research has gone to as well but just wanted to say yeah thank you so much. Have lovely rest of the day and I'll speak to you very soon. Thanks so much. Thank you. Bye.