Episode 3: Assessment and Feedback

[Music] Hello and welcome to our podcast Student Success Pod. I'm your host Stuart Norton and here we continue to discuss core thematic areas of student success. Please do remember to follow this series to make sure you stay up to date with new episodes and activity related to student success. I am once again joined by a co-host and in this series episode it is Vic Stevenson of Advanced HG with subject matter specialism in the area under the review today assessment and feedback, a topic that impacts globally. We have three fabulous guests from across the sector talking to all things assessment and feedback from their perspective and sharing their insights with you our listeners. So I shall hand over to Vic to introduce herself and our guests. Hello Vic. Hello Stuart and hello everyone. I am Vic Stevenson as Stuart said. I'm also a senior consultant in the education team for Advanced HG and I work predominantly in learning and teaching development programs and offering consultancy and support around all things learning and teaching but particularly around assessment and feedback design and strategy. It is my great pleasure to introduce our three speakers today and I'm just going to have to take my glasses off. So our first speaker today is Dr Lauren Woodlands and Lauren is a senior lecturer at Queensland University of Technology in Australia working in the Central Learning and Teaching Unit. Lauren is also the programme lead for a specialist associate fellow award in Indigenous Knowledge for Practitioners in Australia and this recognises successful integration of Australian Indigenous perspectives and Knowledge within Learning, Teaching and Assessment Practice. Lauren delivers professional development programs both for QUT and also for Advanced HG working with me and her programmes focus on inclusive assessment and feedback design and the use of technology to enhance assessment from both a student and staff perspective. So hello to you Lauren. Hi everybody. Thank you for having me. Hello I'm joining you today from Turribull and Yagra Land in Australia. That's the traditional owners of the lands on which I'm on. Thank you so much for having me. Brilliant thank you Lauren. Our second guest is Dr Miriam Fah, who is the academic lead for assessment at the University of Manchester in the UK and Miri led the QAA optionality and assessment project and provides sector support on flexible assessment. Miri spoke in a number of events including the Advanced HG Assessment and Feedback Symposium panel discussion where time was really too short for her to share all her insights on how we can enhance student experience and outcomes through the provision of choice and co-creation in assessment and we're really looking forward to delving a bit further into these things today. Thanks so much for it. It's such a pleasure to be here and really looking forward to the conversation with Lauren and Naomi and yourself today. Thanks so much for the invite. Brilliant thank you very. And our final guest is Professor Naomi Winston, who's Professor of Educational Psychology and Director of the Surrey Institute of Education at the University of Surrey in the UK. And Naomi is also an honorary professor in the Centre for Research and Assessment and Digital Learning at Deakin University in Australia. And Naomi probably needs little introduction as she's been published, oh she has published over a hundred articles which are widely read and frequently cited, 6433 in published works according to Google Scholar. And Naomi's work focuses on learning behaviour and engagement in education and her research focuses on the processing and impact of instructional feedback. When you Google know me she actually comes up as a public figure. So we're really fortunate to have her join us today for our conversation on what makes a difference in students experience and outcomes in assessment and feedback. Thank you very thank you for the invitation really looking forward to the conversation today. So colleagues welcome and thank you so much as ever the first thing we do by way of easing us into the episode is to start with something a little bit more hearted and that's of edu bisket. Listeners new to the series you're in for a treat as our guest divulge if they are a dunker and provide their favourite bisket for doing so. We are compiling our very non-academic chart to accompany the series but as ever this is a serious matter and to accompany my fervent research from the earlier two episodes. My fact this week is, well this month I should say, the average Brit munches on three biscuits a day which works out as 204 million a day across the UK's approximate 68 million population. Over the course of a year an individual will consume a 1,095 biscuits weighing an astonishing eight and a half kilograms. But as ever the academic side of this Dr. Hamid a consultant in endocrinology and diabetes at Imperial College Healthcare NHS Trust said that bickies give us a dopamine high which is even stronger if we dunk them. And I quote dunking a biscuit softens up the texture makes the biscuit literally melt in the mouth. So as these sugars hit the taste receptors and get into the bloodstream that gives us that pleasure area faster. So with that very academic research support in edu biskits I'm going to come to know me first and ask you are you a dunker? Absolutely not. I think it's horrendous to think about, dunking a biscuit. It's a betrayal of their structural integrity. And I think the dopamine comes whatever it doesn't matter if it's been dunked or not. So no, I refuse to dunk a biscuit. Well there you go. And what is your favourite biscuit now me? It's quite boring. I think anything with chocolate on so chocolate digestive chocolate hobnob classic. Okay. Can we push you for a favourite otherwise our chart is going to be you know hobnob. There we go. Okay. Okay. I think that actually puts the hobnob into first place from memory from earlier episodes. Mary, can I come to you next? Are you a dunker? And what is your favourite biscuit? Absolutely. And the hobnob is now winning. The chocolate hobnob is the way forwards. And it was so lovely to hear. I think it was your guest Kate on episode one referring to the chocolate hobnob. And I think it's an under undervalued biscuit across the biscuit market. I mean, it's just it's solid isn't it? It's just it's just such an absolute delight. And unlike Naomi, I am absolutely a dunker, but it has to be Yorkshire T as I'm from the right side of the pen line. So yeah, it has to be Yorkshire T. And I'm a multiple a serial dunker. Like I can't just dunk one. Like if I've got a packet there and going for it. Fantastic. How can push that average up to three a day? I like that, Mary. And having lived in Yorkshire as well, we still maintain Yorkshire T and R cupboards. So that's I understand exactly where you're coming from. Good to hear it. Lauren, please are you a dunker? So I'm not a traditional dunker, but there's a practice in Australia with Tim Tem biscuits that I know it's Tim Tems have come up a couple of times already. But what you do is call the Tim Tem slam and you actually bite the corner, the opposite corners of the rectangle and use it as a straw. You dip one side into the tea and you suck the tea through and what happens is it makes the middle all soggy. You still got a little bit of structural integrity, but it's a very time to perform it. You've got it instantly pop it in your mouth so that it kind of all melts together. And it's pretty spectacular and you don't end up with any biscuit in the tea. I occasionally dabble in this practice, but it's not my favourite biscuit. But I just thought I got to let people know about that performance art that happens with Tim Tem. And I do know Stuart, you're still waiting for those times to be sent. So listeners, please keep Stuart in mind. But no, my favourite biscuit is the iced vovo, it's an Australian biscuit. And I want you to imagine the fanciest rectangle that you could get, like someone's giving you a picture frame. And they've put raspberry jam in the middle and cushioned it either side with a bit of sort of pink fondant and then sprinkled it with coconut and it's an absolute delight. It's particularly fancy and it just makes a cup of tea a lovely experience. Wow, fantastic. And Tim Tem slam brings me back to my school days when you used to use twixes as straws. All these habits coming out. Last but by no means least, Vic, I, I, I, we've worked together for a long while now and I do not know if you're a dunk or a lot. Well, I, I would describe myself as a light dunker. I don't like any kind of biscuit getting into tea. I don't like drags and things at the bottom of the tea that is, that's really not, yeah, it's disgusting. I also am from Yorkshire, so it's Yorkshire tea only. And I would say a light dunking of a kickcat is something that really can't be eaten, where the chocolate melts just slightly, but you don't get soggy wafer. But I'm not a massive biscuit eater. gyda o degy, amma dai ddelluniaid yn yma, amma dai ddelluniaid yn yma, amma dai ddelluniaid yn yma. Yna wedi ddelluniaid yn yma, amma dai ddelluniaid yn yma, amma dai ddechrau dwi. trim un aaf o Ydy'r gw darku ar yw rhod m Charlie som會 Ys ynees roedd daw rhân. Ond y buscicleol o f Selfa effectively? Yn o'n mynd fel ac judebrau yn ' parreddill ac bydda bod wedi cyccael runet Jaandill. Roedd wedi ddigallu llorn o'n d بن hyn yn carlinearbu f draw wedaedd ll Ysg That & Mind a'r Dagingydd Metroid a'r ddigallu llorn o ffyrddio yn ymwyr i'r ddigallu llorn o ffyrddio. Inau'r ddigallu llorn o ffyrddio yn ymwyr i'r ddigallu llorn o ffyrddio. Roedd wedi ddigallu llorn o ffyrddio. a'r teithau'r ysgwch. amser na'n dothadatau rhywm og dig bounds ac, sydd ddydd fan cael nationallyen ac fel diPLE chw ar llwyd Rydym pan. Wyn ni'narr angherald rwy'r Cysлизwfynos o llymiad Ng Sherman a'r meddwl mewn Gysteithos helpu migddannag? Cys commendiaithiau press Mega Hon takea? Oedd fel trwy f ER byddol gwuitive a trwy f amser Gdynych chi l Gesundheits Mae Grichowau fetyn i'r mi iaweeb ac fy k��운 iig fi Bror chipsyd eu'r rum fast issue a sylfr wel digi nhw'n cantwch wneud y unadsynny mae fydd am light onr favyt wyneddo, a ond y lestb yn y Ff нуethymfafa fyddw Mam deall; Fwy pir Resources Rydマ'n Ffu Bat. Os ni unidaethaur Lyach y Ddi印ed separate тau unid a'rgebraidol fy parair gyffredd ar rhywbeth ac tet там Old fraction pens wedi am g beş y mae'r g silence, ddiad kayr ho furyff i chi'r allan worth a m ahorosswch am yảo a gwrddwch ddiad f o debogaogaeth un cref STEMI darmaeth bod y fwy hidd nardigd po Angol malu還wodul o fydd yn gydraho AM Crefnod. Chy, cyste templateau fel meddwl da faint wedi di Leute a cwynanierietheraidol bod, feel mwyrddau cyfrydduneb gyda绾iol i fi ich am pan swydddydcation fel swyddain gyda fydd the o febrillau halddunaill chi Judaismlog. Ond, closeseld wedi hyín cyrfo faet yn gefnwch bod y complaining i yw ein italian fe bre sy'n o fræhu amdd雲 ti'r fallingi a bobl mansa sēm. [Lyn Ngu] Ki cael yna gwneud tyr beloved hynny r失u ac��or RTw i Lyrwyn Bellywn yw unn mathowrshet honol maen, Russell II allachud digandellfa per ei entyng wneud fel dou ar yr medden i'r gyfer hardware unig. Mae wedi fel rhoi o tom heb Terry Voiddoedd Ewisen wrth yr EddragoDadd o Niwbys Americans C連nddwnaeth ynryn o pei faeud y dовать â aider Iog Ddiut Shop普 Chhydd i'r oflaio sut f Celebra'n bra�n ws i'n ffordd wr Ash, maen nhw'n ffordde, mewn softerfodd, mwchyd ar nhwodraedd cidael a fixed. Sc well paró i te Decidd, mwchyd a bellain cidael hyn, yaOMauiani yw mewn eraill. Mae'r pergle. Gael lawasu, afrath sy雾 viwn sideith ei mis odd🤣 A ff controlled Gem Michydl yu gwyrm yn ffordd yiller. I'n gwaith ymwch chi'n gwaith ymwch chi'n gwaith ymwch chi'n gwaith ymwch chi'n gwaith, a mae'r erbyg yn gwaith ymwch chi'n gwaith ymwch chi'n gwaith. Ymwch chi'n gwaith ymwch chi'n gwaith ymwch chi'n gwaith ymwch chi'n gwaith ymwch chi'n gwaith. Mae'r pergle, mae'r pergle, mae'r pergle i'n gwaith, mae'r pergle i'n gwaith i'n gwaith. Mae sydd sydd pobl yn fawr y playorth eich syd yn tref marau ar y gallu ym Ndiwedro ac power grefo'n brab sydd Eye, mae daeth sy'n unru gwas mion. Cobodówn ddogleau leiau popaltriedill staf by… aja, at batol i'n gplants ni bael grefo fawr y 치 hyr程au Fel yna clwad â ne sgithyd. wyraeth am关 weith wrong On GAW, wne i'n fffo hwnni o ff fol fei am bygach nieg baru unib i dities yn yno i vgen wneud neu Rhiflbr vac waren ati achol bi denny. Felly myn nh Honour Lyfn, ond unrh ddair adanc a cwg 강 ceraf ad, au rysyddothoriaeth ac fod yn rock nes oes, mae」'r ffer y fodaed o可能 dr secure yna miodd yn byn wyn whest yn rameud yn oed i yna fwy aceluor i argynod, vi gwanewlaeth. O disgusting ar votinerau ohofpiriair Britana p hearider astronomical o ffwwr. Oolution ni Aye tanto, môr czas. Maeъmau rodych royd y treidol am basa a terbut i cozeio ei maenigiool modaillau dros. Sân idoid here마 am o dyna, byny y ie lefw en accŭ to rakCo�sio o fryn ito pwdeysan – bkerigostan motto hon bwyr o fel sydiypti rescotua régple — pe argedau proces na f�laen yn fうんair i ffordd i doblea dineg? Mae eich syddol am ffordd am wer yn siopgeg yn iPhone 12 a pe. Mae ffordd fy nesgynt. Aut ffordd, yn fallho wyגoli, mentel y tentadig繪d disappointedol a ffordd 'llaneond ni b أي, o fel'n cofisiau iawn bydfa seol LOVE, mae mistakenellu metw'r afted company bod y go, Early Sheeod voywng o gyliwyneau bod gwiai eich yr Johi ג regards ddu gallig eryddod, cywf艾 a ffordd, fy chllau meraniaid. Mae'n cofio'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, mae'r gyddoedd yn ymwch, faidback is on the work, not on them personally. In certain disciplines, particularly things that the creative arts or performance arts have been challenging, where students have put the heart and soul into something. It is very personal and any feedback needs to be managed really carefully, as Mary was saying, with reference to criteria and learning outcomes rather than, "Oh, you could have, you know, when you reflect on, I don't think I have a sore criteria when I was at university, I have no clue, you know, it was in your, in your academic's heads, you know, you didn't have that conversation, nobody showed you that, so I think it's so valuable that that becomes part of our, that that is embedded in our practice now. Sorry, Lauren." "Oh, good. I was just thinking about my own experience of keeping chapters of my PhD with lots of track changes that I had gotten back from supervisors, and it was this wall a'r rydw i'n gweithio'r llwyd. distinguishable from my original writing and the changes that have been suggested. And I keep it so that when I have one-on-one consultations with PhD students, I can show them and it's especially valuable for non-English speaking international students where English isn't their home language. And I can show them in Salmon. I'm a native English speaker and this is the level of feedback that I'm getting with this sense of normalising that someone's engaging with your work. They're wanting to tell you a little bit more or change your thinking and get you to understand, but normalising that that engagement is a really good thing that someone's looked at your work and it's actually a gift in a way. But I always also refer back to there's a fabulous bit of research called the Higher Education Learning Framework and one of the key things from there is the fact that emotions play such a big role in learning and it can either inhibit learning or it can promote learning. So it's how can we get on board with the emotions that are going to promote learning and so that normalising that we're doing of engaging with feedback that is one of those things that is just so vital to helping build resilience in learning. Yeah, there's some brilliant work where there was another QAA collaborative project on compassion at feedback and there's some really useful practical prompts for you to kind of think about considering in the way of delivering and it does cover things, you know, encouraging students to think about their own state of mind and their own their own emotions and timing and whether it's the right time for them to receive this sort of feedback. Absolutely. So we've kind of talked a lot around kind of we've established that high stakes, exams are terrifying for many necessary in some cases. We wouldn't have said nobody should ever do exams. I did hear K-Samble was saying though if you're setting a timed assessment exam under exam conditions, you have to have a really clear reason for doing so and it has to replicate the sort of situation machine we'd find themselves in in the real world rather than it's because that's when you can book an exam hall for that amount of time and you know you're really not considering the student. I'd like to come to Miri where when we're talking about assessment and it meaning lots of different things, what you mentioned before about obviously your work on optionality in assessment, how do you personally define what good assessment looks like? I think it has to involve some element of student choice and I know that through presenting on our optionality project in various conferences and obviously we've got publications through the QAA website as well. I know that by me saying that I'm automatically raising you know people's concerns and fears around you know what that might mean. So when we do refer to optionality in assessment, I do not mean allow every student to do whatever they want first and foremost. Our national project looked at I think it was 13 different forms of options in assessment and I think good assessment embeds and is built on student choice by allowing students some form of flexibility and as you've mentioned I'm the academic lead in our flexible learning program so my my focus is very much around embedding flexible learning at the University of Manchester and supporting that through student choice in assessment and one of the most straightforward areas that you can offer choice in assessment is around the deadline. So when colleagues are saying to me you know about how can I do this you know we've got a number of markers or I've got a cohort of 700. If you think about when you're deadline for whatever assessment it might mean that assessment is going to be uploaded or created into your VLE and it's going to be opened at some point. From the moment that that assessment is open on that on that online platform to the moment it's due there is choice there. That student or learner in your cohort can submit it the day it's open and available to submit up until that minute and hour and day that it closes and I just think sometimes being absolutely explicit with our learners to say you know you need to do an essay for this piece of work or it's an exam but up until that point you have this time within which to submit that look at your schedules think about your time management you know would it benefit you to actually submit this a week early would it benefit you to submit on the date because again offering that option that autonomy it immediately motivates learners and that's when our research study found time and time again you know and we surveyed hundreds of students hundreds of staff we did workshops across the board and it was the thing that we found was most beneficial to our students don't just give them lots of choice identify where choice is available and that will automatically make them feel more in control of that assessment and more autonomous to really be more creative ultimately in shine so yeah I really think some form of choice is essential that's brilliant thank you and if anyone hasn't seen Mary's post on LinkedIn which is a cartoon strip or comic strip rather and really setting out what optionality and assessment can mean because I think it's quite it's quite threatening in some ways for for some academics to think you know like you said it doesn't mean you've got to allow them to submit a poem or a two thousand word essay you know there are there are different levels and different ways in which you can approach optionality and flexibility and I'm working out what's right for your programme and discipline is obviously keep I'll find out about to to name your Lauren with the same sort of question what do you think really defines excuse me good assessment um I think good assessment is something which it goes beyond subject matter I think a lot of the time assessment is just um I'm solely focused on the the content that we're trying to get students to demonstrate that they know but when you think holistically about that whole human being that we get to develop when we're helping our learners become you know learning is a as a sense of the coming so why focus just on this one piece of it and think about well what are the skills what are the attributes what are the qualities what are the graduate attributes that we're really trying to help build in learners and whether or not that's something that isn't always demonstrated in in the rubric which is something which is you know often such as a small space that we can and really look and assess is it something that they can develop in in the process in the way they think about learning in the conversation that you have with them so that is the key for me is does the assessment look beyond just the subject matter itself and think holistically about the person that we're that we're trying to develop and know me yeah I think in assessment design decisions we're often pushed to be driven by by pragmatism and often the real power in assessment comes from thinking in a lot of detail about what's most meaningful for for students in that situation and at the moment obviously we're in a situation in the sector here in the UK as well as elsewhere where where efficiency is a word that we're talking about a lot and I think there's a risk that that pushes us again towards thinking pragmatically about assessment how can we save time how can we make things simpler but I would argue if we've got precious resource that that assessment is where it needs to be directed to make sure that we're designing tasks that are going to enable students to really develop and learn not just test their understanding we shouldn't be driven by thinking about how easy is this to mark we should be driven by is this the right assessment at this time in students journey and is it the thing that's going to actually lead them to enjoy what it is that they're doing and working towards that task rather than it being a chore. Absolutely I think we just we've focused a lot on the human element of assessment and I know Stuart was keen to pick up on something mentioned there around the pragmatism especially in the age of AI that the I think it's coming back the other way but there was an initial shift towards let's get students back in exams and Stuart went to was going to pick up the theme of of technology there. Yeah I think it's an interesting point isn't it because there was definitely that shift back to the exam hall because then it was we know this is the student's work and you know rather than debating the rights and wrongs around that of something else you mentioned there as well knowing me which was how easy is this to mark and and I think there is sometimes a challenge around large cohorts you know how do you get through three-week turnarounds that that real pressure for a metrics and assess you know everything all of this is undiscruistly even though we know that that's certainly from a UK perspective you know that the geid sen astwcheywch eraill ar gyda接wng oed yr sgynnyddaeth yrீ a gwasot. dib contrac новol ierybrydg, buto gwlech mathn am i rhywbwys i ddyshat, i fy gallu fan i. מau lysbytieth, myn chi'n tyrepleg sy'n y SUSES, enfa��сь ac reiwn i'r syrais maddech ddog ת recycling I Eastuan leitö ar webain bod dial gwneDA ein iaithethau bod â y trophy fel iaes cyllew ar gydaau помuire 일�w gw 여� reclaim pati yna I Signaill ddawscream error, fy rhaen fanos i fod yma yma yn fanau twice ar yw eich gyda bo er nuniwoof mwyb y концull oвин enn yr ffordd dangrohon Cyfod Fel Ewro i am yn cael meu på—isiw cisfarci emotig 1990 roeddiwch ag fran gefnnyn nhw i? Mae'n c幫r ei gallunc ieddwn rwy sgiliau'n gteryf terraéfniss�� nieeg pan dim oes yn mai gallunc frysgarth, beth wrth gwyli It來 yn porthol i grop fans every 4 oitnessrazaod fencesau i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau i gwasanaethau yn gweithio i gwasanaethau i gwasanaethau yn gweithio i gwasanaethau i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau yn gweithio i gwasanaethau i gwasanaethau aegwysio'r gweithio aegwysio'r gweithio aegwysio'r gweithio aegwysio'r gweithio and there's also some wonderful research that talks about the importance of then getting students to reflect back on how they've used that AI as part of the learning process and that's where they can really consolidate that learning as they're developing those skills for lifelong learning in an AI reality Thanks Lauren. It's really interesting there and you know immediately when we talk about technology there is that huge focus on AI and I think undoubtedly because the rise of chat GBT and many other AI box I mean who can believe it was well just over two years ago now they came into being and the huge exponential growth and their ability to expand as well I just wondered if anyone had any experience of other examples of technology and in a round an assessment and again I'm just thinking around the growth of virtual reality you know we've seen in certain disciplines simulations you know through medicine, through to aviation and also in applied criminology specifically policing that ability to engage in 360 videos that virtual reality world I don't wish to put any colleagues on the spot but it was just something that came to me as we were talking around different use of technologies anyone have any comments or experience in that regard? No they can't do that Yeah I think I've seen some really fantastic examples of this sorry right from quite typical uses of simulation I suppose in health care and health professions education but through to some quite novel uses of digital tools that enable quite realistic simulation and facilitate group work as well in our hospitality and tourism management programs for example students do some fantastic simulation work running hotels and making really quite complex decisions with data really well received by students feels meaningful, supports employability enables students to connect with each other around a meaningful task I think the challenge with these kinds of tools is that people might feel they don't have the expertise to develop, to use and to implement them and that's where I think central teams can be really really useful in helping to connect people from different disciplines who might be wanting to develop something similar or or might be facing a similar challenge within their assessment design and trying to connect those people and facilitate the development of novel uses of technology and assessment Yeah thank you it's really interesting I don't want to diverge the conversation as well but it's almost that gamification around learning as well isn't it in that virtual reality world and I know gamification is also something that has expanded greatly as an avid all-game player and delighted when I see posts on LinkedIn and where else talking about the benefits of playing games do we think that students' views and approaches around assessment have changed since this increase in the use of AI Fick what you're saying from a sector position do you think that something is changing? I think it's really interesting I think some of assumptions made around students use of AI and their desire to use it and their desire to use it to cheat the system and all of that I think my experience in actually some of the research that came out of our collaborative development fund with a number of universities in Scotland showed that students are generally very uneasy and very unsure about there might be proficient users of AI but where it comes to using it in an assessment context it throws lots of feelings of if I don't want to use it in case I get caught and I do the wrong thing and I don't mean to do the wrong thing but for other people using it and I'm not using it then they might get a better mark than me and I'll be disadvantaged so I think it's really interesting that I've seen certain institutions take a very kind of strong line of 'we must stop it' which I think is impossible but also making assumptions about how students feel and what students want to do and exactly as Mary said they need guidance from us and from their lecturers on how they can use it responsibly and what that means because there's a lot of policy around ethical and responsible use but for a lot of students that what does that mean ethical use of AI? What are they supposed to do with that? The research showed that there's still an alarming number of students as our lecturers who are not talking to students about AI at all and I think that's really a critical first step sorry Mary you must try It's really fascinating because that matches so much with the work we did with JISC and that original report to the Sectron Student Opinions and the thing that we found which is also often not explicitly referred to is how the lecturers are using it one of the things we found was that the students were thinking we're now using this we have the opportunity What are my lecturers doing with this? And we heard a resounding concern over any use of AI for marking, any use of AI for lecture content. And I think it's, again, just as you said, Vic, it's about saying to our learners, this is how you can use AI for this particular assessment. This is how I have used AI in these certain ways. Whether it's prompting mentee to create a discussion question or whether it's, I don't know, there's all sorts of things that it can do, creating a scenario and then checking it afterwards. But I think it's the dialogue, it's the frankness, it's the transparency. I'm a full advocate of being open and honest about the use of technology, where it's appropriate, because what you'll get back is then that student feedback, which we need, not just end of term and not just for NSS, but the students will then go, "Well, why are you using it like that?" And then it enables that dialogue, doesn't it? But it is both ways, it's not just those confirming how we want learners to use it, it's also identifying how we might have used it for our teaching. And I'm sure colleagues on the caller are aware, was it the word Delve that suddenly jumped up in academic articles within the first 12 months of chat GPTing news? We don't even have to talk about it, it's obvious, it's been used. I just think it's far more human to be that person in the room to say, "Yeah, actually, I couldn't think of a scenario last night when I was planning for this discussion today." So I asked chat GPT for an example of a scenario, that triggered me to then think further. Be open and honest about it. Anything you'd like to add to Lauren from the other side of the world, presumably students or students, they've got the same sort of fears and concerns and ways of working. But is there anything that's potentially a bit different? Because I noticed that you, I've referred quite a lot to the Texer principles, and they really felt to me like Australia was very forward, thinking was very proactive, and I was like, "How can you build this into your assessment? Where is it appropriate? What guidance should you give?" And there seemed to be some really clear guidance right from the start. Obviously the work that came out of University of Sydney is brilliant. The student-authored site on good tools to use, how to use prompts effectively, how it can help you in your discipline. It's just brilliant guidance, and so much work has gone into it. So be interesting to hear your perspective from Australia. Yeah, I do also highly echo the, there's some fantastic stuff by Texer talking a lot about assessment reform in the age of artificial intelligence. And I think one of the main things that comes out of it is it's not just one discipline body that says this is the way of approaching it and doing it, but the fact that there's multiple approaches that you can take in different ways of considering how you want to use it in your teaching, how you want your students to use it in learning, and kind of like what you said, it's just being really explicit with how does this unit incorporate A.O.I? How does my teaching incorporate it? How should you learn incorporate it? And modeling, you know, what the expectations are in the same way that we would expect there to be that degree of explicitness in a workplace scenario with, you know, what can we do ourselves, what can we outsource, what can we bring in artificial intelligence for as well. More and more, the comments that we get from students are just really echoing that they're looking for guidance. What am I supposed to do, how am I supposed to do it, and how can I do it well? Just conscious of our listeners from across our global reach, and maybe I'm feeling slightly overconfident there in that we're having hundreds of thousands of you listening in, and the text is the tertiary education, quality and standard agency in Australia, who are the independent national quality assurance and regulatory agency for higher education. So I just wanted to cover that one off, just aware of inclusivity around terms and acronyms, something that I'm being much more conscious of myself these days. Vic, sorry, I know you're going to lead on to trends. Yes, no, so I think we've kind of picked up on kind of what's happening in different areas. And we're interested to hear if there's anything that is that you're seeing in your own institution, any trends or themes that is that's impacting your approach to assessment and feedback within your own particular context. Yes, certainly, and again, this must come with the caveat that I'm coming from the flexible learning program at the University of Manchester. So looking at assessment from that lens, certainly we're seeing lots of trends. I think I'll refer to five briefly just around some of the things that we've done around optionality. So that there are more diverse assessment practices, the first one. So students want assessments that do cater to their unique strengths and learning styles. And there is a growing interest in that in terms of non traditional assessment formats. So things like collaborative projects, portfolios, presentations. Now, I know they won't be non traditional to lots of colleagues, particularly in different institutions, but at a Russell group like us, they are usually more non traditional. And our students want more of those want more access to those across the board irrespective of discipline. Secondly, more inclusive and equitable assessment. So seeking assessment methods that account for the diverse backgrounds and reduce bias. And we've got an excellent team in our library through my learning essential. So Jenny Blake and the team look at lots of flexibility for assessment to support inclusive practice. Those students with disabilities, those balancing work and study. And also those underrepresented groups. So we've got lots of activities happening around that to support those learners. And that one most notably because flexible learning is now the norm, you know, across the sector. It's not just about mature students coming back in and requiring part time study that's still in an I'm till five. You know, we've got learners globally that want to dip in and dip out transfer credits. So I think that inclusive and quite inequitable assessment has jumped its its grown from. From just looking at diverse and representative groups to being those range of flexible learners. I think the integration of formative feedback. So students really value our feedback when it's timely actionable and developmental. So we've got much more strict strict requirements around the formative feedback when it happens discussing that with peers using peers in class for that as well. Number four, technological integration. So having things like ePortfolios, online submission platforms, things that are more integrated, more agile. That support more innovative assessment types. Again, like I said before, for more authentic assessment, you know, just because we use VLEs. It doesn't mean we have to submit assessment that way, you know, just because it has worked in that way last year, doesn't mean it works, you know, this year. And finally, I think the key thing and again, this helps back to our research on optionality. It's around enabling agency and co creation. Students not just being presented with a feta-complete that is we need you to do the 6000 words by this date within these learning outcomes, but having a dialogue through assessment, enabling them to think about the design of their work, how they best want to produce information on their knowledge and their development that showcases that best. You know, who is better to do that than the individual? Why should it be an academic that sets that rigid structure when it can be done in dialogue and it can support that agency and co creation. So there's five things there that I've thought about before our discussion today, but I suppose that's just the starting point within our flexible learning assessment at Manchester. And I know that they'll they'll be echoed with colleagues. Yeah, thank you very much. Can you hear me? Yeah, I think there's some really, really important themes there in terms of what Mary's talked about. I think for us, the trend, the focus that we've had has been on zooming out from assessment is just being the thing that happens when the student has submitted a piece of work and we grade it and provide feedback information. We've been thinking about how we support student learning preparation for that assessment as they're working towards it, but also how we really support their learning after that work has been returned to them and how we help them work with feedback and help them to apply it. And the contextual challenge that I think we've been aware of in doing this is often when we're trying to do institutional change in assessment and feedback and any other areas of education. And we feel really overwhelming for staff that they've got to make a big change, they've got to do something really dramatic, whereas actually what's often really useful is is lots of small changes. And so we've used this approach based on the aggregation of marginal gains that Cedeve Browseford used when. trying to improve performance of Team GB cycling team. The idea that if you take part riding a bike into all of its little components and try and improve each one by one percent when you put all that back together again, you've got a really significant improvement. And we've done the same thing with assessment feedback, said if we break assessment feedback down into all of its component parts and try and make tiny tweaks that improve each of those, we put that together and it does add up to a significant change. So focusing on assessment of feedback as a full cycle, not just the process of grading and providing feedback and doing that through the aggregation of marginal gains rather than trying to make big shifts that ultimately are not sustainable within practice. Brilliant, thank you, Naomi. And Lauren, you've got the unenviable task of kind of adding to what has been too really comprehensive answers to that. I'll go the unusual route and I'll go short and sway. Something that I think has really come up in the Australian sector a bit, particularly at my university is this idea of trauma sensitive, trauma informed, trauma aware education and the idea of how that can sit within the assessment and feedback process in how we can prioritise the relationship that we build with learners around learning and around thinking about learning. So that's something I'm really excited to see prioritised within the sector. Great, thank you, Lauren. So if we can move on to your kind of top tips, which I think she has got as a feature throughout the podcast series, so if I could ask each of you to think of what is one top tip or tangible takeaway that you would like our listeners to try out going forward. Naomi, should we come to you first? Yeah, I think one of the things I feel really strongly about is that we need to disentangle, we need to separate assessment and feedback. We always talk about assessment and feedback together as if they would have to happen together. And actually, I think that's where we, as a sector, have got ourselves into some really challenging situations in how we make both assessment and feedback effective. And so my key tip or takeaway is to actually think about separating feedback in a design sense from assessment. We always talk about assessment design. We very rarely talk about feedback design, whereas what we actually should be doing is thinking where in a cycle, a learning cycle is feedback and the time that we can spend on feedback, going to have the maximum impact. In very few cases, is that going to be right at the end after we've graded a summative task? So separating out feedback, designing feedback for learning, thinking about where can we embed feedback so that it can actually have an impact. Rather than feeling we've got to be shackled to assessment and feedback always happening together, I think is my key takeaway. Great, thank you. Yeah, I couldn't agree more and I think you're absolutely right. It is a segment of feedback in one mouthful, isn't it? And it really doesn't need to be and it shouldn't be. Mary, can you come to you? Are you going to guess this? It's the answer to do with options. I'm going to go there. It really is just so powerful. So yeah, if there's one thing, I think I'd really like your listeners to do moving forwards, is to introduce any small element of choice in either the assessment or feedback and it doesn't have to be both. And it certainly does not need to be a complete overhaul. So it's not with something manageable. So for assessment, it might be something like I've mentioned already where you outline to those learners, your deadline is on the 12th of June, but you can submit up until that. There is choice in that. Look at your time management, think about when you need to submit and when's best for you. You can pilot things leading up to that assessment as well. A lot of our colleagues will do some drafting. You could have options in that. So it doesn't have to be an option in the assessment, which is the final submit of peace either. It can be, you know, we've been learning about this topic. I just want to check on where your knowledge is up to in week six. So in that class, you've got an option. Would you like to write me a few words on it? Would you like me? Would you like to, you know, wrap about it? You know, so then you're removing those high stakes as well and you're just making it a more enjoyable, more free flowing and flexible process of building up to that final submission. And in terms of feedback, you know, you can also allow them to think about the type of feedback they'd like. You know, we spend hours, hours and hours and hours. I mean, I'm really quite OCD about this. I love standardising. I love looking at the marks per question or marks per intended learning outcome and really thinking about that year after year. That's me. I'm not saying everyone should do that. But, you know, having that open discussion with learners to say, I am planning to give you written feedback. It normally takes me, you know, this amount of time. But actually, if it's more beneficial for you, for me to talk that through and give you an audio piece of feedback, let me see if that's going to be possible. And again, make it a dialogue. You know, don't shy away from opening up that dialogue and suggesting different things where it's feasible. You know, I don't suggest you, you know, you open up the conversation on options unless it is feasible and checking with all colleagues. But letting students choose that feedback format again is incredibly powerful. And also, you know, we've talked today around when we feel comfortable in reading feedback. You know, do our learners want feedback on every intended learning outcome or do they want more depth on their writing style or their knowledge on one particular learning outcome because they've already reflected and already identified a gap in their ability. So therefore, are they wanting us to think about that as well? So yeah, definitely think about a bit more choice. Don't change the entire system. Do check with colleagues. Think about what's feasible, present it to your learners and say, look, I can offer you a bit of choice here. Would you like that? Would it be beneficial if so, let's go with it. But give me feedback on that as we continue. Yeah, I think that's really beneficial in getting students to take some responsibility as well. Their own, how they get their feedback and what they do with it. I think the point that you just mentioned there that just occurred is, I've seen comments before that we tend to focus very much on standardisation or assessment. But there's very rarely any kind of standardisation activity that goes on around feedback. So within any given department, you can have some lecturers offering choice, some offering, having it in a very dialogic approach somewhere, it's written comments, somewhere it's very brief, somewhere the tone of the comments is quite negative. So I think it's kind of worth exploring as well, obviously not within the remit of this podcast, the strategies and ways in which standardisation of feedback and what's expected and what options academics have because for some of them, it can be quite new to marking and providing feedback. And it feels like there's very little, there's a lot of training around how to mark, but less so about how to give feedback. Anyway, Lauren, you're top tip. My top tip is a practical activity that you can do with learners and this is around getting them to actually create a bit of a feedback map and they can map out all the different places, people, sources that they can get feedback from on their learning. And the idea being that at different points during the course of the programme or semester that you're running, you can get them to revisit that map. And go, well, what have you been mainly relying on and which do you see as more important or reliable or I see here that you've been using a lot of written feedback from your tutor, but what about going to a writing circle with your peers or how about how consulting with a library and looking at the literature, like what are some different ways that you can engage with feedback and this is really to open up learners, I guess, understanding of how there are so many different ways and formats and sources that they can get feedback from the learning. Thank you, Lauren. I love that idea of a feedback map. This idea of seeing all those different sources and actually them having that as a visual map. I think there's an idea of a blog there for Advanced HE to share some of those ideas, Lauren. Colleagues, thank you so much for your time, Stad. I do want to have one last question and it's really around, is there a call to action we should be putting out to the sector. What areas should providers be looking to work together? Where is that one space and I'm conscious that your own research takes you across the globe already, but is there anything in particular that we should be doing and that call to action? Lauren, I'm going to come to you first because I've seen you just put something in the chat to us there and it speaks volumes to me, so please. Oh, I was going to say the biggest, the one word that we can throw out is compassion, compassionate feedback, compassionate assessment, compassionate learning because if we approach anything with compassion, we're bringing it back to the human. - Humanizing. - Absolutely, and I think compassionate dialogue as well is just so valuable. And something that's missing and also a sector that's under pressure is something that we perhaps we can be more compassionate to our colleagues and ourselves as well. Naomi and then Mary. - Yeah, I completely agree with Lauren. I think particularly with the conversations that are going on about chat GPT at the moment, artificial intelligence, looking at how we really maintain humanity within assessment of feedback processes. But more importantly, I think in answer to your question, it's not so much what providers should focus on in collaboration, but how we do it. We see so much sharing of what works. We very rarely see sharing of what hasn't worked. And actually, I think for people who are leading institutional change in assessment feedback, knowing what other people have tried that hasn't been successful is actually way more useful because then you don't replicate those in your own institutions. So I think more willingness to share things that haven't worked would be really, really valuable. - Yeah, that's so true. Naomi, and anyone who knows me, knows that I always bang the drum for what doesn't work as well. And it is absolutely critically important. And yeah, very much aligned to, you know, when you were talking about marginal gains as well, all of these areas that have looked at previously and really do speak volumes. So thank you. Mary, one area for you, that point in collaboration, what should providers be looking to work together more on? - Yeah, thanks so much. You were doing it already. - And what we are, and I think, keep having the conversation. I mean, exactly as colleagues on the call have said, I just think having those corridor conversations are so important. You know, either if it's after you've moderated something or after you've presented the assessment task and go, "I'm not sure how that landed." I'm, you know, and just having those conversations is so important with colleagues. And, you know, building capacity and time and culture to enable that will just automatically improve assessment. So I just wanted to add that to Naomi's comment on that collaboration, firstly. But I also think whilst we've developed the resources with the QAA on their website around optionality and assessment, I think there's actually a call. And perhaps advanced H.E. will be interested in doing this with us. But looking to create a flexible assessment framework for the sector, I think is definitely needed. We need to collaborate on what works and what doesn't work. And I fully agree with, with talking about the issues and the barriers that are presented, because some of the feedback we had was around the discipline peculiarities around supporting flexible assessment. You know, it's not to do with discipline culture. It's more to do with the creditors. So where you have a particular creditor who only supports this form of assessment potentially or not or not. So I do think there's a call for us to look at developing a flexible assessment framework. And I'd be delighted to work on colleagues with that if they're interested. Thank you, Mary. And I'm sure there will definitely be interest across the sector and certainly from sector bodies. As you're talking there, it's reminding me how all of the core thematic areas of student success are interlinked. You know, we've touched upon graduate outcomes. We've touched upon flexible learning. You know, we don't have time to go into the impact on different groups of students, and international students, and assessment around there as well. And there was just so much more. And I'm sitting here thinking around a podcast around collaboration, sorry, around standardization and calibration, which is something as well. 'Cause we talk around assessment, but then it's that marking assessment and calibrating across the sector as well and into that. So that leads me to say thank you to all of our guests, Mary, Lauren, and Naomi, for joining us today to Vic, my co-host on this episode. All of your insights were absolutely invaluable. Before we go, I wanted to remind our listeners that there are wealth of resources, including the recent literature review conducted by colleagues at the University of Kent, Dr. Ed Pitt and Professor Kathleen M. Quinlan. And indeed, their gamification of the research, assessment and feedback superchargers, freely available resource available to all advanced HE members. We have the assessment and feedback framework, and there is a forthcoming symposium in late 2025, all details are available on our website. And myself and Vic's will be co-editing another series of assessment and feedback case studies, which we publish later this year. We're back next time with a podcast in flexible learning, and much more can begin to be unpacked with what promises to be an equally exciting episode. Until then, it leaves me to say that biscuits should be sent in to advanced HE, carriage to Norton, but stay curious and do tell your colleagues and friends, be sure to hit that like button and follow. Thank you, goodbye. (upbeat music)