Developing students’ digital fluency and employability through authentic assessment

Hello, I'm Jen Delaney and I work as a career consultant within the careers and employability team. Hi everyone, I'm Dr. Gemma Watchert and I'm a lecturing school of life sciences. And I'm Jason Steyer's Communications and Engagement Project Officer from the University of the Pool and you'll listen to the Academy's Development Practice Podcast. Hello and welcome to another episode of the Academy's Development Practice Podcast. In this episode we chat with Jen, Gemma and Jason about their recent Sir Angister Pilketon Award for Teaching Excellence, awarded for their development of life sciences, students, digital fluency and employability at scale through the design and implementation of authentic assessment. We hope you enjoyed. Well Jen, Gemma and Jason, we're really pleased to be talking to you today about your recent Sir Angister Pilketon Award for Teaching Excellence and it was awarded to you for your development of life sciences, students, digital fluency and employability at scale through the design and implementation of authentic assessment. And we're really excited to hear more about that. But before we get started, it would be great if you could tell us a little bit about your background and how you've all arrived at the roles you're in today at the University of Liverpool. So Jen, if we start with you. Yeah, hi, morning everyone. So yeah, my background really is in Recruitment and Career Guards and Centre and a bit of employer engagement. So I graduated from the University of Liverpool, but with the history to go and kind of by accident ended up in a graduate role in Pharmacy Recruitment and Recruitment for the Pharmacy sector across the UK. So I worked there for six years and then I moved to the young persons career service. It's had like a million different names over the years. But basically my role there was looking at the education to business model in Germany and how young people move from education into work and how we could kind of adopt similar ways across the Liverpool City region. Then after that loads of different projects there. I went to work for a charitable arm of a wealth management organisation. And that's where I got my first experience of working in HA. So I worked with academics to embed wealth management and employability courses for students and kind of fully funded stuff. And then I applied for this job on the back of that experience and now I work as a career consultant within our careers in employability team, but I look after the faculty of health and life sciences. Fantastic. So you can see how your journey has built to this role now and what do you enjoy working with the students or do you mainly work with the academics in your role? I really enjoy working with the students. I say my main job as my career consultant is to work with academics and take that strategic approach to making sure that we're embedding employability into the curriculum at large. But yeah, I do some lectures, workshops with students. I love doing all the intro stuff. Kind of intro to employability sessions and things like that with with our faces. But yeah, my background has been a bit mixed like that in itself because although I started in employer engagement and recruitment, I ended up working face-to-face with young people on the different European projects for the career service. So it's kind of like that mix now kind of stakeholder but still student-facing. Fantastic. Thanks, Jen. Gemma, if we come to you next. Thanks, Alex. So yeah, unlike Jen, who's moved around a lot, I am very much University of Liverpool, born and bred. So I did my undergraduate degree here in Zoology and then following that, I worked as a fieldwork technician for a year or so, trapping rodents in the Kiel de Forrest in Northern Berlin. And then following that, I kind of moved on to my PhD and moved more away from Zoology and into microbiology. So that was a bit of a change. And then after that, I went on to work in scientific research for several years on a number of different projects, some based in the UK, one based in Kenya, but still with the University of Liverpool. And then, during that, I did a little bit of teaching and I really enjoyed teaching and working with the students as well as my research. However, around six years ago, I began my lectureship here in the School of Life Sciences and so now I'm a lecturer in microbiology and the programme director for microbiology here. Fantastic. And do you mainly teach undergrads or postgraduate or where's your focus? So mainly undergraduates, quite a lot of the second year teaching and a bit of third year teaching, but I do occasionally have master students for kind of projects and some work like that. Brilliant. That sounds great. And Jason, over to you. Thanks, Alex. So, like Gemma, I've pretty much, other than about 18 months or so after graduation, being a part of the University since the age of 18. I studied my undergraduate course in music here, and then following graduation found employment from the private sector after graduation, whilst building up a bit more experience in another pool music scene at the time, but constantly remain keen to work within higher education and returns again a year or so later. So initially, I worked within student administration support in a front-line role before moving over to careers and publicity to work within what was at the time the student engagement team, which was the initial front-line service at the time. So I helped students with employability resources to enhance their confidence and awareness, and I did that for a number of years, but then, career's employability underwent a period of transformational change in 2018, and that allowed me to move into the role that I work in today within the communications and engagement team, and on any basis, just to provide digital news, tools, and information to keep up students engaged with employability. Interesting back stories there. I mean, particularly, I love the story from, you know, catching rodents in a forest to lecture of microbiology. I think that's fascinating, and we might even come on to that a little bit later, but the reason that we've got you all here today is because you received the Alistair Pelketon Award for your development of life-sciency students, digital fluency, and employability through authentic assessment. So what was the need for the project? You know, why did you choose to set that up? So the new activity was rarely driven by collaboration between our different teams. So the School of Life Sciences and obviously our careers in employability team, and we've seen that obviously the higher education academy really advocate this collaboration as being key really to inform and curriculum, which improves and supports our graduates' employability. So Jen and I have been working together for a number of years on some of our skills modules in the School of Life Sciences and also our Life Sciences Employability Week. However, for this assessment that we won the award for, we decided to introduce a focus on trying to develop students' commercial awareness as part of the assessment, and this is really because we'd seen it, it's identified as a skills gap across the Life Sciences and the wider graduate labour market. So really our teams been led by recent labour market information as well and obviously post-pandemic around 71% of employers are planning to keep digital interview as part of the recruitment process and also we've seen other surveys from people in places like Handshake. They're also showing that a large proportion of graduate employers are going to rely more on social media and digital technology to recruit graduates at scale. And so we thought it was really important for us to address these areas when building this new assessment for our year 2 skills module. Right, so you mentioned in developing their commercial awareness. What does that mean for students in their year one and two? So yeah, when we meet commercial awareness, we're trying to find out the students as well because a lot of the time students believe that it's about recruitability only, so only having an understanding of what they need to say in an interview in terms of what an organisation does and things like that. But really it's about students developing that understanding of how to be commercially aware in the workplace and to understand how their role in that workplace and that organisation is like a cog and you know has a knock on effect and also have a really good understanding of the sectors that they want to work in and the developments within certain sectors and the challenges organisations may face depending on what's happening in the world basically. And this is kind of super important for the life sciences because of the type of industry that it's in, you know, developing in particular kind of, you know, health technology at the moment and on the cut and edge of a lot of new research and development. So there's always lots going on which is why it's always identified when you look at these surveys from employers as being one of the top things that they want students to have or graduates to have when they leave. So yeah, there's probably lots of definitions of commercial awareness, but that is what we were trying to do with this assessment with our students in the first instance that makes sense. Yeah, it does make sense. I think it's, I think you're right, it can mean various things, can't it? I mean, in my time in the private sector it was really obvious to me because I was in a sale, well, it was a sales business to business organisation and you could see the the work that I was doing you know that cog that you mentioned how that fitted in was what at my job was to improve the bottom line It was really simple, whereas I guess I suppose in in the careers that students from life sciences go into it's probably not that simple So I guess that's the reason to actually include that is that fair to say Yeah, well, I think it's basically a skill that is across the board as well I think it's interesting for life science students to be able to do it We'll get on to what the actually assessment is to be able to do this research around what's going on in the sector and the challenges and the opportunities Faking organisations maybe I'm biased. I just think it's dead excited because it's like scientists But yeah, I think basically it's just one of those skills that If you do have it you do understand it and you're able to articulate why you're commercially aware You're going to have that advantage above other graduates when you're not only applying for the job but once you're in Because lots of other skills are born of being commercially aware you know you you find students become more competent leaders More curious, more interested in innovation and things like that. So it's hard It's hard like any skill and it's to kind of badge it like you said much to be honest But yeah, I think timing as well doing this in year two and offering that foundation to to get students to stop thinking about what that might might mean Whether they go into the life sciences or not. It was super important I think it's really interesting what you were saying Gemma a little earlier about I think he said 70% of interviews will now take place online instead of face to face And I've been working with one of my mentees recently around An interview that she was going for and we were discussing how different it is to interview online as opposed to face to face And I guess that's something that we really need to be preparing our students for like you were saying So is that something Gen that you've built into this assessment model Maybe you could tell us more about the the assessment model itself and how you've embedded that kind of digital fluency and employability together Yeah, definitely so that the technology itself Was the basis of how we were going to do this at scale And during this assessment so If I go back to the beginning almost We designed the new assessment definitely using the university's curriculum framework But essentially what we wanted to do was replace the kind of traditional assessment and face to face elements of this cycle that already existed And with a more employer-focused digital approach And it was really to enhance you know existing provision So historically what would happen is that students would complete a job application And then have a mock interview with their academic advisor So again we could see gaps there with authenticity in this assessment And so what we really needed to do was it connected to employers And we were able to do this using resource that already exists which was our life sciences employability week So this is a co-curricular event that happens in the first semester every single year timetable and for as much as it could possibly can be is shut down for our life sciences students We're so lucky we've worked for a long time with our director of employability Zen Lewis to create this face on the curriculum And what we did as Employer relationships with these employers already existed We brought them in as key stakeholders and with assessment delivery and design So what happens now is that students have the choice of researching One of seven employer panelists that attend life sciences employability week And these panels are sector-specific so they range from kind of animal and ecology Type stuff to pharmaceutical to non-linear so alternative ideas for What I can do with my life sciences to great So students go ahead and they go to these panels during life sciences employability week And that's where they develop their knowledge and their commercial awareness Their understanding of this organization and what opportunities and challenges are facing that company So that meant we were kind of getting them to build Employer knowledge at our local and national level In order to kind of complete their summer to assessment So this summer to assessment now is this asynchronous video interview And it's on the platform shortlist meet which jacers our expert on and he'll give you more information about that in a bit I guess But what the technology allowed us to do and to kind of embed that digital fluency for our students Was to replicate an authentic stage of the recruitment process Used by employers such as the NHS and AstraZeneca you know those organizations that they would actually resonate with And of course give us that scalability asynchronous interview You can do at home in your own time during the set time frame that we'd set out And we met the needs of kind of 430 students in this way So yeah alongside the video interview thinking about digital fluency as well And they did submit a tailored job application But they also are asked to submit a link to their updated LinkedIn profile And their handshake profile which is our career's platform Here in the University of Liverpool that connects our students and employers online Coming back to jammers point out area about kind of social media And those platforms being one of the main routes that that employers use to select and screen candidates And so yeah really what we did we aligned and and existing careers event to the curriculum And we used this digital technology to embed the commercial commercial awareness focus questions that we wanted to for our students And so that was it in a nutshell real as I said jace can probably give more information on the tech itself That sounds amazing so being really naive in this field what's an asynchronous interview is that one The interview's opposed beforehand yeah tell us what an asynchronous interview is Yes so an asynchronous video interview is where the interview is set up in advance for students more for anyone who's having this interview so it can be prepared in advance But it is an authentic experience because the employer is the person that the student sees on this video interview So the employer will be on screen Introduce the assessment itself and then ask a series of questions And students have thinking time and then they record their answer back to that question that's set Um It's really good a lot of employers use this I did this for this job that I've got which was terrifying at the time Wish I'd have had this when I was in uni um A lot of employers use this And I'm alright jace is the first screening um before you even get to An interview stage just kind of like a million to you that would come before it's almost to an extent an initial shortlist and process Because it helps to identify those students or those candidates who will complete the application process And especially for for jobs that are required and candidates to be confident on screen or you know for social media For example, it's just an initial sifting process really for students who for candidates who will Not complete them really Sounds fantastic and then um So the students then submitted this as part of their assessment and then who marked these assessments were the employers involved in that or was it just yourselves? The the employers provided video content it's for actually the preparation and the actual video into itself But it was actually tutors who were providing the uh reviewing which they were to obviously by wax accessing through short This may but then obviously completing the mark and um through canvas as well It shows how long I think I've been out of the Job market for because what that it sounds like um That first stage which used to be the telephone interview so when I applied for a job With Barclays, I don't know about 15 20 years ago something like that They asked me to go on the telephone interview and I remember it being absolutely terrifying thinking This is this feels harder than a than an actual interview But it it wasn't after all actually, but if I had some preparation for that I think that would have been really really useful So it's this what most employers are going to now this instead of telephone interviewing They're putting all their sort of work up front and then allowing Allowing people to sort of answer the questions as and when they need to Lot of employers are yeah, I think it's going to be really interesting Might to be honest with you to see what does stick post pandemic. I mean this this idea of um Of digital interviews already existed obviously before the pandemic and other um Faculties have used the same technology to embed in different areas of the curriculum across other schools as well The same technology But I think it's going to be really interesting to say if it does remain Such as you know the norm across all sectors It's definitely um easier. It's definitely a better resource in terms of time and efficiency for employers But it's like anything isn't it you know where planning careers events where we know we can accommodate more numbers when we do it online But we are slowly moving back to some face-to-face I don't know what you think Jason Gemma, but my opinion is that it's always going to remain quite high the technology is there now um It's it's such a good recruitment tool for employers um that we need to make sure our students are prepared for this in the long run going forward Um, sorry, because I completely agree with that. I think the pandemic's obviously had spiked it up But I think the level will be maintained following that process and whilst there may be some sort of relative shift back to kind of like a physical presence with it. I think it is here and remains will remain the case. One of the things we have access to which shortness means what's called the marketplace. Basically that is a non-demand video library of video interviews for students to be able to choose. So as well as getting exposed to video interviews through something like like 2-2-3 which we're talking about today, we do also try to sign post students through our creative students, our creative coaches, who are our frontline students delivering that service to the marketplace to make them aware of the fact that there's a 2-2 there available for them as a preparatory tool prior to completing whether it's a video interview or even a physical interview to kind of recreate the process of a face asking you questions and obviously you haven't so think on the spot in terms of answering back really. Yeah a little bit like what's happening in this podcast right now I guess. It's interesting that you said that because I did see something on LinkedIn a couple of weeks back somebody shared a snippet of a job description and in that post it mentioned that they wanted the applicant to be competent at communicating via Zoom and I thought well isn't that just communicating but I guess it you know I guess it isn't really I think what they were alluding to is they wanted cameras on and people to be you know given as much attention to it as possible but I just found that strange at that time but actually now when we're talking about digital interviewing through asynchronous methods I think actually it makes it makes sense there is a bit more of a nuanced skill in there isn't there instead of just sitting in a physical room. Absolutely matter one of the things we also try to push out to students prior to completing these video interviews as well as it's it's obviously prepare them to sell themselves and sell the content of what they're going to say but it's also making them aware of other factors as well in terms of the setting that they're going to be into complete the video interviewing and making sure that they're in a quiet space to do it and the background looks clean and efficient basically and it's making them aware of time management during the video interview as well so that they're not completely pushed for time or where they might be interrupted to that kind of thing really it's all little little bits which make it which would set them a great way as well as the content that it's actually what I was in the video interview as well. Yeah now that makes a lot of sense that it really does and you know I think one thing that the pandemic has taught us is how to carefully curate our backgrounds as you can probably see so I think yeah I think it all adds doesn't it to that sort of that first impression you know you're not just presenting you in some ways you're presenting what's happening in your surroundings as well so it'd be interesting actually maybe a sub question that is around you know is around blurring your background and the benefits or negatives to what that that means if you're going into an interview with an employer anyway something to think about just well we're talking about you know what that's meant for students can we just talk about a little bit about the impact of the approach to student outcome so what you know what the students said about it what has been their feedback so one of the things I don't think we said earlier was how big this module actually is so there's 430 students who run the module and we've got typically around 90 academic advisors marking these assessments so this was particularly going to cause us a bit of a problem during the pandemic when we couldn't do face-to-face teaching so by using this technology it really meant that we could support students to complete bespoke interviews and with an employer of their choice and also typically in the past this has taken some time for academic advisors to set up interviews and they've spent a lot of their time really on actually the organisation process rather than giving students feedback on their interviews and so by using this process it meant that students get a lot more feedback from their academic advisors on the actual interview rather than time spent on the logistics of it and changing to this system also was a lot more inclusive so students could complete the interviews at home in their own time and like we've just said about the differences between communicating kind of in person or online in like zoom or video interviews and one of the nice things about this software was it it allowed students to reflect on their process throughout the video interview so students could re-record an answer if they thought they had perhaps missed something and I think Jason's going to talk a little bit more about the actual interview later but they had like reflective points and artificial intelligent feedback so it was kind of frightening for students in a way that it was for a lot of the students it was something they hadn't done before I think in our survey there was around 10% of students who'd completed a video interview prior to this but it did mean they could turn off the camera they could re-record their answers so there were lots of benefits to this for students and the feedback we've had from students was quite positive so really to evaluate the impact of these changes we've carried out a student survey and a lot of the students said that they thought this software was really easy to use and we saw increases in students confidence in both completing a video interview and evidence in their commercial awareness but we've seen kind of wider impacts really beyond this so we found that a lot more students are joining the School of Life Sciences LinkedIn community and we've also seen that students are activating their profiles on the university's career platform and our data showed that 92% of students from life sciences had an activated profile on the university career system and this is compared with when we did the the original assessment I think the university average was about 56% but I think Jay she said now it's increased to 65% the university average so a lot more of our students are already taking ownership of their their kind of careers and things and Jen you were talking before about our Life Sciences Employability Week and you mentioned that we'd align this assessment alongside Employability Week and and this was really positive wasn't it we saw increases in the number of students who were actually attending these live panels with the employers and you know this was obviously beneficial for the employers because it meant that they get in access to a lot more students and I think we saw a 60% increase in the number of students attending these panels compared to previous years when we didn't have this employability focus yeah and just to jump in Jamma yeah it was good and and I think the thing it's kind of less data but a really positive thing to say about Employability Week that year was that in those panel events we had a lot of student questions which is so good for the employers and for us when you are hosting those events and you are really willing that participation from students to come through and we we did we and I think that's important the numbers you know we had some employer panels with close to close to 300 students online attending these employer panels but yeah and the feedback from employers that's coming through our employee connections team is that you know students were connected with these organisations on LinkedIn and social media beyond the panel as well which I think is fab so yeah it was that was one big advantage for me from kind of a lion and our event that existed. Fantastic it is fab and I can see why you guys won the Sir Anna Stapilkinton Award for this innovation it's so authentic isn't it and student focused and I can see how it really enhances the student experience I'm guessing there has to be some challenges to this approach so Jayce is there anything that you found in terms of running this intervention at scale that you struggled with and how did you overcome those challenges? We certainly had some challenges and just to give a bit of context a guess to the whole thing we our partnership with shortlist me was formed in March of 2020 so we we pretty much signed the contract a couple days before we went into lockdown but the partnership emerged from a bit of a shared desire really and a commitment to bring authentic digital impurity experiences to students but our use of video interviewing software at that point wasn't new we already had provided interventions for student practice and also for assessments through modules prior to this new partnership or the new platform that we had access to provided much more that we could utilise such as as james mentioned it we could incorporate self-development points into each question for students to take off as they were complete dancers and they could reattempt their answers and we also could provide feedback through artificial intelligence so students could complete the the video interview in their own time and get instant feedback based on their the rate of speech in terms of how quickly or how slowly they speak them and any use of filler words so that might be a guilty that myself saying too much but the interview interview feedback would bring that up and so that allowed with the user experience we benefited from and there was a real pathway to scalable projects such as this but as I say there was a challenge to Mae'r i'n gwaith i'n gwaith i'r gwaith i'r gwaith ysdwyr i'n gwaith ysdwyr i'n gwaith ysdwyr i'n gwaith ysdwyr i'n gwaith ysdwyr i'n gwaith ysdwyr i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes, i'n gweithio'r oes i'n gweithio'r oes i'n gweithio. i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio. I'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes. I'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes. I'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes. I'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes. I'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes. I'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes i'n gweithio'r oes. No, it's all really positive. I think the academics were fine and accessing it. Typically, we do moderation between our different assessments anyway. And I think the first time we rolled this out, we didn't have the academic advisors marking the actual video interview. We just, we gave students some proportion of marks as a kind of incentive to complete the video interview. And we ran the assessment this year. We actually had the academic advisors marking and gave them kind of clear guidance on the rubric about what they should expect for the different questions that students were being asked. So, what was the specific software used for this? I don't know if I might have missed this. What, do we do something into it normally? Was it external software? Yes, it's an external platform called Short This Me. So, it's created by a company called Short This Me, who deal predominantly within solutions for recruiting processes, but this tool called Short This Me is more for proprietary purposes for students. So, we used to use a platform which was a recruitment tool. We used to bend and manipulate it in many ways to allow it to work for student practice. This platform that we took on Short This Me is actually designed for student practice and for assessment purposes such as something like this as well. More bespoke to what we want us to do, although it is an external platform that we work with. No, that makes perfect sense. In my head I was thinking, you know, Flipgrid could probably do this, but the administrative burden of it would be massive. I take it because it's custom built for this type of activity that I was just because I was also thinking you must have hundreds and hundreds of maybe thousands of hours of video footage or based in this system. And then sort of sifting through that, the system must do a lot of that heavy lifting for you. It certainly does. Yeah, I mean, it certainly helped with the scalability of this, as I mentioned before, in terms of the tools, the tools are providing the kind of the user friendly way of working with it from my perspective. It's like being on the administrative side of it and just helps to kind of visualize when we were having the initial discussions. It's visualized how we could work and thankfully it did what we wanted us to do. I'm having competitions in them just thinking about this cycle round because we were like, oh, do you know what we've done it kind of like the better version last year now. We've got all these videos from employers, perhaps if they come to employability week again, we're set, but oh no, we're tweaking it, we're changing it a little bit. So we're going to start again soon, aren't we? We can start putting those requests out to employers for famous last words, it'll be fine. Okay, that's what we said a lot, I mean, it'll be fine. Well, it was, it was more than fine. I think the, you know, the evidences and the awards and the, you know, the, the accolations that you're getting for it. One of the things that we'd like to do on this podcast is sort of highlight some of those great pieces of practice where we have combinations of work with maybe professional services and academics. This is another really good example of that, you know, that is a, it's a really higher, highly effective approach that could be easily transferred to other disciplines. So, you know, do you have any plans to sort of share out this practice to all the areas in the university or even other, you know, areas in the HE. So, as I've previously previously mentioned, shortness me and also handshake, which are in policy performance are already used within a number of examples for academic delivery and assessments. This project, however, shows that scale up and personalizations really possible and are success with the two awards. As meant that often exists in academics who have always engaged within the past are interested in scale up their assessments, fair, but to be create this template because it's so successful, which is great. And I've also been in discussions with some other academics who are interested in uselessness for the first time as well. And that's brilliant news. It's one of the consequences of winning the awards I was really hoping for in the sense of, you know, getting the word out about these digital and political tools, which can be utilized within many different kind of format really. Not long after winning the LTSA awards was approached by an academic from veterinary sciences, and they were keen to work with myself and Jen to create video interview templates based on four specialisms within veterinary sciences for their students. And that's obviously been run out to those. And it's just really good to see that the existing templates that we've created can work really for other departments, you know, the employees will change the questions will change the student workflows can be easily adapted. The backend workflows and processes within the system that we use it easily transferable. Yeah, definitely, Jason. I think, you know, we have it's been really nice to to kind of talk to the departments and and to kind of think about how this would work in different places in terms of where else we've been kind of talking about it as well. Jamma and I were successful with an abstract to go and talk at the advanced AG conference, so that was great. We had a short talk around the assessment and it's impact basically. And there was loads and loads of questions from academics across the UK, you know, some academics there, where from the life sciences as well, which we've made connections with from other universities. Who are interested in in how they can they can do this for their students to so yeah, I feel like we've been we've definitely been talking about it a lot, haven't we? Yeah, and I guess just to jump in, I guess kind of next thing is we need to write the publication. I feel like it's online now we're recorded, we've got to do it. Mae'r gydwch chi'n gweithio ac yn ei ddweud. But yeah, I guess that's, that's kind, but it has been really nice, hasn't it? We've had loads of really positive feedback and I guess interested in what we've been doing and people are really keen to try and implement it and use the technology. Yeah, I mean, I say this and it's not, it's not simple at all, but it is a simple formula that this assessment really, and use the resource that already exists, you know, most departments, organisations or whatever within HG would have some sort of career facing events, traditional careers for a lot of the moves online, you know, kind of tapping into those employers that are already engaging on campus and then combining that with the tech, with the technology as well. But I think that's what most people have been interested in, that kind of the ability to embed that authenticity. I say with ease, but Jason's the one that does all the hard work in the background with all the technology, so it's probably not true, no key for me to say that, but you can say that, it's fine. Thanks. Excuse me. But yeah, to kind of, to have that employer-connected element of it as well. As Jason said before, the employer-employed connection scene, we're so lucky to have some team and careers that goes out and builds those relationships with employers for us. But yeah, they were able to build that again in their own time, as increasingly it provides us that content, so it's good. Well, that's been so interesting. It's really, really good to hear about such a brilliant example of innovation and the way that you guys are supporting our students authentically is really exciting. So thanks so much for sharing your work with us. This podcast is called The Developing Practice Podcast, and we really like to finish each podcast with a couple of take-home tips that the listeners can reflect on in terms of their own personal practice. So if you could share a couple of tips with us for us to reflect on what would they be? Jen, if we start with you. Thanks, Alex. Alex, I think I'd say two tips from me. The first will be Collaborate. So I think Jemma started this podcast talking today about how this whole thing was driven between the collaboration from our two teams and kind of sharing the knowledge that we've all got. You know, Jemma's understanding of her module and her students and how things work and all is coming in kind of talking about what is needed in terms of employability development and embedding that into the curriculum. And then Jase, with the kind of understanding of that technology, just without that collaboration with us, just no way. We could have confidently embedded this assessment in any way. So I would encourage other people to reach out to whoever, you know, and work together and get other people's opinions when you're building something new. And then the other thing when I think about assessment is, you know, we're driven here in careers and employability to be driven by the data, of course, but to be completely employed connected wherever we can. And that would be one of my tips. If you can, be a connect what you do to an employer voice, to, you know, to their understanding of what they need from graduates. You know, we didn't mention it earlier, but one of the, one of the great things about the flexibility in building this assessment, Alex, you were talking about how students were marked on their video interview and how it could be standardized across so many students. The questions that we had were the same for the first part of the interview and really focused on that commercial awareness and their understanding of that. But the last question we gave to employers, and we asked them to identify a skill that they think is important with their, in their organisation. And our students provide an example of how they develop that skill. And we were able to standardize that quite easily because we, we asked, we, we marked them on the structure of their answer and, and how they structured a typical answer in that way using the star method. But yeah, it was just really good to, to be able to kind of have that employer stakeholder position in the assessment as well and, and give them that element of design. So small, but, but, but, but effective as well. So there might be. Brilliant. Thanks, Jen. Gemma, how about yourself? So, Jen's kind of really touched on the, the car elaboration, which I think is a big thing. I don't think I would have done this if I didn't have the support of Jen and Jason, and particularly as the academic, I'm always thinking about the risk and how students are going to really evaluate these assessments and the process. And, you know, Jason was really confident that the technology would work. And Jen was positive, you know, we were doing the right thing, we were looking at the right skill. And so, yeah, I think the collaboration was key. And being willing to take that risk and change your assessment, you know, the previous assessment worked okay. And, and was what happens in the real world as well in terms of you do have face-to-face interviews. But I think, you know, moving to the video interview has been much more beneficial for the students, you know, they're likely to have had a face interview in the past somewhere anyway. So it's much better now. And I guess the other thing is to think about the process as not being kind of fixed. And obviously, we ran the assessment once one way we've made some tweaks and gone back and made further changes. And so, you know, it's kind of a knitter of processes and that so we'll just keep making changes and improvements to try and make things even better, I guess, for the future. Brilliant. Thanks, Gemma. And, Jason, how about yourself? Thanks, Alex. Well, I can only echo what both Gemma and Jen says. Really, obviously, my main thing would be to not be afraid to try new things, otherwise we wouldn't be in a position where we are to date an example without doing that. But that's obviously led by collaboration and being aware of each other's skills and expert knowledge and say at the areas and stuff like that. So obviously, I'd echo that. Plan a lot for anything that's new, such as this. And we planned a lot for kind of various different combinations and scenarios. And I think that stood as a good stead when we actually came to the point of actually rolling this out to our students. And there's also just echo in terms of maintaining a level of flexibility within some of our projects, which is just to be able to adapt to any possibly possible and for seems to circumstances that might be, that you may come across. Wonderful. Thanks so much, everyone. It's been really good chatting to you. Yeah, thanks for your time. I was trying to unmute you, thank you. Thank you. Thank you. Well, I loved chatting to Jen, Gemma and Jason. They spoke about how commercial awareness had been identified as a skills gap in the graduate labour market across their sector. And so as a result, they'd explicitly embedded this within their assessment. To do this, they'd even brought in employers that their graduates are likely to work for post-graduation, to be panellists as part of the assessment. And I really liked that aspect of the work that they'd done. We talk a lot at the university about designing authentic assessment. And for me, this is truly authentic. Yeah, the authenticity of the assessment was extremely clear, wasn't it? They spoke about how market information has shown that post-pandemic 71% of employers in their sector are going to keep digital interviews as part of their recruitment processes. And so once again, this assessment is authentically supporting students to develop the necessary skills. Well, there's lots to think about there. If you'd like to take your thinking further, we've added some resources to the website on a specific podcast reading list that you can access at Liverpool.ac.uk/theehyphenacademy/podcast. Also, we'd love to hear what you thought about the episode, so please do tweet us @LiveUniaCademy. And you can also find us @elernamatt or @Alexandra_oin on Twitter. And we're really grateful for those who have taken the time to either rate or review our show and your podcast providers up. So if you haven't done so already, please do take the time to review our show. Or even better, simply share the episode with friends and colleagues on your social media. Bye for now.