244 — The value of benchmarking

Hello and welcome to the Good Practice Podcast from Emerald Works, a weekly show about work, performance and learning. I'm Ross Garner and this week we're asking how you can actually measure the effectiveness of your learning and development function. Can you provide clear evidence that your team are delivering value for your organization and how do you compare to other L&D teams? If your answer to any of these questions is "I have no idea", then benchmarking might be a good place to start. For years, the Tours Maturity Learning Health Check has been the industry leader for benchmarking your L&D function. Now part of the Emerald family, we're relaunching the health check this month as the Learning Performance Benchmark. Here to tell us more is Gant Amitai, Head of Research. Hello Gant. Hello Mr Ross. Good to have you back. And we're joined by Kari Horvath, Product Manager for the Learning Performance Benchmark. Hello Kari. Hi Ross. Welcome to the show for the first time. Kari, what's your point of being here? And of course we're joined by Mr Mussel himself, no stranger to Bench's Owen Ferguson. Hello Owen. Oh, I am going to regret that what I learned this week for quite some time. We'll be talking about your Ripper board for a month to come. Brilliant. All right, let's get into it. Again, why is it important for L&D teams to benchmark? Very broad question and yet very specific at the same time. I guess I'll start with this notion of why is it first important to even have data, collect data and then ultimately analyze it. And knowing what the pandemic has done to organisations to work like balance and generally the introduction of digital tools are the pace that probably has never been seen before. The ability to explore, understand and gain new insights from a variety of sources is not just critical in terms of measuring impact but also increases the chances of survival and competitive advantage for organisations. And so this is where the idea of Benchmark in comes from is trying to understand where you are, where you are in comparison to everyone else and how you can move forward from there. We looked at our own data set and actually only about 15% of organisations are Benchmark on a regular basis internally and externally, which is quite low and quite surprising in itself. Well, we thought then we said, well, let's split this data set and let's look at organisations that Benchmark on a regular basis and organisations that don't benchmark. And what is the major differences between these two groups and what we found is the ones that Benchmark usually are a bit more digesting mature. So they have almost like a sound balance in terms of delivery mode, mixing blends, understanding how to deliver online but also using trust and as well when needed. The next one, which is probably the most critical of all, is they're just simply are more proactive, behaviorally and strategically and as to do with things like well by collecting data they understand a bit more about their situation and therefore they understand how to move forward from it. And last but not least and I guess this is a logical kind of conclusion, they tend to measure more things in more depth and for longer, which is a major aspect of why Benchmark is important. You simply get legacy data sets and you also have longitudinal data to understand a bit more about how you are moving forward. So that's in a very quick way why Benchmark is important. This is a good research question I think you'll like this get. Are these organisations benchmarking because they're already digitally savvy or they digitally savvy because they're benchmarking? Very good question, cause and effect, the perennial challenge of research. I mean, I thought I'd bring it up early. Yeah nice, I mean, I like it. We wouldn't know that to be fair, it's very hard to know what comes first. I mean, what we do know is these are highly correlated, so we do know they go hand in hand. Whether digitalisation comes in first and then Benchmark comes in second or data analysis comes in second, we're not sure but what matters is that these two go hand in hand and I think that's probably the important bit. Whenever we're introducing whether that's digital tools or whether we're trying to benchmark, what comes first, I guess that's really matter at this point. It's more about them being correlated. Yeah, another way of looking at it, there is no downside to benchmarking and there is potentially a significant upside to doing it. So we want to see what benchmarking is, even are we, is that a term everyone would be familiar with? I think no. I mean, common definition is just the ability to compare yourself to other people and a baseline. Usually it's an average of a group, a collective, a community potentially. It's essentially a way of you knowing how, where you stand against the community UI, the industry UI and the average UI. All right, cool. So when how can Benchmarking inform an organization's L&D strategy? I think it's an incredibly useful input into the strategy. I mean, you have lots of other inputs that will inform the strategy. Like for example, what's the overall organizational strategy, where the priorities are being worked on? But Benchmarking gives you three main benefits that I can think of. As Gaines mentioned, the first is that it helps you to answer the question, how do we compare to other L&D functions across a range of different metrics? And it's important to recognize that there's no right answer to that question, but there are likely lessons you can learn from any variances that you have from a typical L&D function or the best performing L&D functions. So some of the measures, for example, made it different because you've made some deliberate choices. But by and large, it's an incredibly useful lens to look at your L&D activity through. The second benefit is that the actual process of going through the Benchmarking exercise forces you to answer questions that you may not have asked yourself. It can uncover hidden blind spots. And we hear from people all the time that even the process of doing the Benchmark has helped them in a number of different ways and has helped them from their strategy from that perspective. The third is that it helps you to track progress over time. So doing the Benchmark on a regular cadence allows you to track your progress on the dimensions that you have decided are important. And as well as identifying new ideas that you want to improve it. So basically it's a really good check on what you're doing and how you're doing it comparing yourself against other organizations. And provided that you're not chasing after the answer to life, the universe and everything, it's a it's a continuous improvement process, a sense checking tool, more than anything else. Well, for that final point, the life of the universe and everything, what you mean is if you're looking to give yourself a part on the back, that's a dangerous mindset to go in with. Is that what you mean? No, I mean, it is an incredibly useful and insightful input. It is not going to create your learning and development strategy for you. Okay, great. And one of the key aspects of Benchmarking is that you're doing it, but also loads of other people and similar roles to yourself across the L&D community are doing it as well. So again, what's the value of community here? Again, you're here with a big question, you're asking easy questions. You're the head of research, you're the big question guy. silly questions that asked the word. So, I mean, apart from being a sounding board to test ideas, approaches and really ask pertinent questions, and you would expect this answer from me, Ross, I know this. It's a really good place to collect more data. You know, just by doing it, obviously you're giving data and you're receiving it, but again, the process as Owen said is in itself super valuable. Now, why the community aspect is important? Well, again, building on Owen's point, Benchmark is in an iterative process. It draws very heavily on the active participation of a collective group and a community. Without it, really, it wouldn't be possible. I mean, the Benchmark is the average of the group, ultimately. That's that's the idea. It is also a reflective process. Again, building on what Owen said, it can help the community find entrenched views. It can really help beat part dependency and by part dependency, I mean, going, making decisions and by making those decisions, you are narrowing your options and as a result of narrowing the options, that becomes your only option as a, basically, the part you're on. By doing the Benchmark, it in a way helps you reflect on that part and say, am I still going in the right direction, asking, changing my community, and do I need to ask new and new and different questions? And so ultimately, it also helps surface new questions as a result and it helps the community build on that. So, yeah, I mean, Benchmark is a tool that is part of the community. It builds on the community and it's sponsored by the community ultimately. Just to add to that, really, one of the major reasons why we created the tools, maturity, Benchmark was to help the LNOD community reflect and continuously track their progress. And so it remains that way. I guess the other thing that I would add to that is unless you are charting your own course and you're right on the bleeding edge of what of what's possible. It's unlikely that the challenges that you're facing or the things that you're trying to improve haven't been tackled by someone else at some point of time. So the benchmark acts as a common language, a common way of framing those kinds of problems that you can then enter into conversations with other people who have been through the process. Perhaps they've been through it earlier, perhaps they've tackled challenges earlier in the process than you are. And don't forget, it's not a case of everyone's perfect at all things. Some people will have acted on some aspects of their L&D approach earlier than others, whereas you might have decided to focus on other areas. So it gives you an opportunity to reach out and speak with other L&D professionals about how they've tackled certain challenges or how they have improved certain practices. Okay, I have two thoughts on this. So one is, does benchmarking act as almost like a source of gravity towards the majority? Does it pull ideas away from the sort of breeding edge thinking towards what is the master people doing? And then the other one is by computing yourself to other organizations, you pitched it as collaborative, but it sounds competitive. In a way, it is both. I mean, you can be collaborative and competitive at the same time. I think, again, when we look at the benchmark, all is completely right. The majority of organizations will, we'll, we'll almost be segmented in a particular way. So for example, within the benchmark, when we do the analysis, we have four distinctive groups that we, we cluster organizations and really 60% of organizations fit in the lower end of the group, where they just simply start into things strategically about learning and development, how that, how the function fits within the business, how strategically they can push the business, etc. Only the really mark even I think like six to seven percent of organizations fit into top performance. And even within the top performance, the funny thing is when we think about top performers, we think of them as achieving everything. Even they struggle with really key aspects like driving innovation and, and, and, and, and exploration, only about 40% of top performers achieve that consistency. I mean, when you think about that, it's four in ten organizations, you know, for the average, it's, it's, I think it's like two in ten or one in ten that really do it consistently. So it shows you that there is still a need for LND teams and LND leaders to really understand what they're doing with their function. And however not to doing it, and then what are the top performers doing? And that again, that's benchmarking allows you to do that. So I guess, and your first point, Ross, I don't think you could be answered well. I mean, you're basically saying, is it a rush to the average? Yeah. And I don't think that's the case because people's natural inclination is not to look at what's behind them, but to look at what's ahead of them. So I would suggest that benchmarking access are catalysts for people to, to strive onward. And I don't think any organizations that come out as high performers are complacent about that. I think people are always looking to improve their practice. I can't remember what your second question was. And then you were talking about the benchmark as a, as a space for collaboration, for cooperating with other businesses, but it sounds, I mean, even the way that you've discussed it there when it sounds competitive. So even if you're a high performer, when the motivational factor is, are the people nipping at your heels in the areas that you know, again, was talking about that maybe you're not a high performer. Yeah, but I don't necessarily see the two as mutually exclusive. So you're lappering on that? I can't say it's a generalisation, but I would like you to defend it. I think it's, I think it's partially possible for people to be competitive against from where they are just now. I think most people want to not necessarily be the best in the world, but want to start, or if the starting point for getting there is striving to improve. And I also think in general, the Elendine community in my experience is, is positive, encouraging, wants the best for the profession. I haven't come across very many Elendine professionals that have turned around and said, well, I'm not sharing my secret sauce because we've got a competitive advantage because of it. That might just be a lack of commercial cynicism on their part, but I think in the mean. Or a lack of opportunistic secret stealing on your part. Yeah, absolutely. But I see it in lots of other domains as well. I mean, you know, one of the things that I try and do, I haven't been so great at recently is speaking to other leaders and digital businesses to find out what they're doing, what they're working on, how do they approach certain things. That's being part of a community. And I don't think that any people that I'm speaking to are keeping secrets because they think that there's a competition between me and them. Even if you're even, even when you've got organizations, even if you wouldn't necessarily want to speak to a direct competitor, there are lots of other Elendine functions in similar areas that you can learn from. So you don't have to have someone from BP speaking to Shell. It's perfectly okay for them to go and speak to price for ice guppers, for example. Katty, you are our product manager for the benchmark. So you've been working out for a year? No, something like that. Just over a year. It's flying by, I don't know, a lot of times. Do you want to maybe explain how it works in practice and how it's evolved over the past 12 months? Yeah, so I think it's a bit of a real first-go-together. The platform itself would probably be a little bit for a while. We've not taken away from the heart and so on, all of what it is around that community driven piece. What we have looked at is listening to our users and understanding their pain points and looking to improve that user journey for them. One of the main things we've tackled is it's no longer just a survey and a dashboard. It's an all-encompassing portal so that whole journey is connected and they don't get lost on the way. We know that obviously work days are further distractions in competing priorities. So, for example, we users can now complete their survey at different stages. There's an instance save. So if anything does crop up, there's clear breaks where they can sort of park it and come back to it. But I think one of the key things is we've looked at how we can provide clear insights of how to sustain progress for them, for that reach for that high impact learning culture. So we've added a lot of additional insights and now, for example, they can filter against their industry but their regional location as well and their organisation size because, as you know, you can be in the same industry but an LNDT might be a size of five whereas a competitor might have 20. So what they can achieve is realistically very different. So we're looking to cover as much of that as we can so everyone can get as much value from it as possible. Great. When we talk about COVID because we've made it in 20 minutes into this episode and we haven't said it once, this has been a weird year where everyone's strategy was thrown out of the window and they had to come up with something overnight. The strategic probably wasn't the word that would spring to mind for a lot of organisations. Is this a strange year to start doing the benchmark? Our comparisons with previous years still valid. We've already collected data during COVID from around 203 organisations. So, I mean, we've already seen large changes happen as a result, especially in terms of obviously learning modality has been hit differently. But actually, overall, organisations during COVID have done better than pre-COVID, which I think is it for a number of reasons that we will be discussing during the annual report. But one of the key reasons that we find is what do you mean by doing better? Well, better, better. So I'll put it in a different way, behaviorally and strategically better, but also in terms of impact, i.e. profitability, productivity, performance, these areas have been achieved at a higher scale. The reason I think why, and again, this is a hunch because we're still writing the report and we're still working on the data analysis over time. It has to do a lot with purpose and collective purpose on the organisation. So, whereas before that used to be hindrances or blockages between L&D, IT, marketing, loads of departments working on several different projects simultaneously, they've all been put in almost like in a place where they all have to work in the right direction, in the similar direction. And as a result, it's been just a smoother collaboration space and everyone has had almost like an authority to go forward in a way from a business perspective, from a strategic perspective. And so, yeah, it has has an impact. What we do find is that from an industry perspective, there's large fluctuations, large differences. And some industries budgets have actually gone up and spending on learning a development has gone up and in some it has gone markedly down. So I mean, that's again expected. But, yeah, at an initial stage, a lot has changed and we're hoping to go a bit more in depth during Daniel report. Yeah, I would argue that there's no better time to benchmark than at this particular point. And the reason is because we've probably seen the most significant shift that we'll see in our lifetimes in terms of working practices. I hope so. Yeah. So, and the benchmark gives you kind of two areas of comparison. One is it allows you to compare over time. So you're able to look at it and say how much have things shift towards digital compared to this time two years ago, three years ago, etc. What's the magnitude of the shift? And then from you're reflecting on that, how much have we as an organization made that shift? Because we're looking at all these other organizations and they've made this dramatic shift and we've not really made that much of a change. There's a danger that we're starting to fall behind the curve a bit. And the other thing that you can compare to is what other organizations are doing right now. What is the mix of learning technologies, for example, what are the, you know, the learning impact approaches that they're taking or adopting? Just to get a sense of how have we fallen behind or how have we coped particularly well in this situation or is everyone, you know, just in a state of I don't know what to do next. But certainly the the take that I'm picking up from again, is that actually organizations are starting to get a handle on this and they're starting to think much more around what is this mean for the future ways that we're going to deploy things. So no better time than to do it now, I would suggest. As a persuasive pitch, I'll slip briefly into my before we go into like where can you do the benchmark? This is a free tool that people can do. So I'll play my Greek cardic role and go, but you enrolled works, people, what's in it for you? Why are you providing this service? Heart hitting questions, Ross, you know, heart. I mean, I would say it's a given take the potential for this reason why this is also free is because in a way it provides us insights, but also we provide back insights to the community. All the data that we collect is completely anonymous. It's completely confidential. We actually don't share it internally with the business. Only at a very aggregate level and those data points actually shared with the public as well through the annual benchmark report, but also through other reports like the leadership one that we just did, the COVID one day is coming up. So while organizations and individuals that do it do give a lot of data away. They also receive a portal they go and compare themselves and also insights via the reports. Whereas we do not share it with the business internally, with ourselves or marketing. Yeah, I mean, ultimately it's in our business interest for the LND community to succeed and continuously improve. So we're healthy. So yeah, I would say that there are two altruistic and one cynical reason for for doing it or potentially cynical. Emerald works is passionate about learning and development in general. We see this tool as a way of helping organizations to improve what they're doing. We want to raise the profession up and we'll work with any partners that will help us to do that. I mean, a lot like the reasons why we do the podcast. I was just thinking I saved it because we make no money from this. Yeah. And whose fault is that? The second thing is the better and more forward thinking organizations are the easier they are for us to work with in reality. So we find it easier to work with organizations that are forward thinking, moving the profession on that makes it easier for us to find and work with high quality organizations. The slightly more cynical reason is it's really good for brand building and brand awareness. I'm not going to lie that you know that that's just the natural truth of it. That is a an outcome of the first two as opposed to the number one reason why we're doing it. And we have put me in kind of the team have done a brilliant job. They've gone right back to basics, taking a look at the benchmark from right from the very start, spoken to existing users, people who haven't used it for a number of years gathered feedback. We've gone through a really thorough product development process. I'm really excited about the new version of the benchmark that's coming out that will benefit everyone that has used it in the past. They'll be able to go in and look at the data that they've collected in the past and see it in a much more visually exciting, informative way. All right. I'm convinced. Sold. Carrie, working people access the benchmark and how long will it take? So you can access it at mwwx.com at the moment. It takes about an hour to complete. They will be given the learning health check currently as we haven't launched the new one yet. But the new portal is coming this summer. We still encourage everyone to do it now because they'll get into the insights and they'll get even more insights when the new portal launches. All right. Lovely. Let's move on to our regular feature of Wilder in this week where we share something we picked up over the past seven days. Oh, and do you want to go first? Sure thing. I wish I could remember where I got this from. But I sort of tweet. It was a 90 second clip of Andre Agassade talking about returning Boris Becker's serve. And he had been beaten by Becker in a number of high profile matches. And so he was determined to get on top of the serve, which is notoriously difficult to return. And what he did was he just watched hundreds and hundreds and hundreds of hours of Becker serving. And he noticed over a period of time that he had this this tick where right before he served, he would stick out his tongue. And if he stuck out his tongue to one side of his mouth, he would serve in one direction. And if he stuck his tongue out in the center, he would serve in another direction. And so he used this in the game. And he said he had to be really careful because you didn't want to give away that he knew what Becker was about to do. So he had to strategically pick the service that he was going to return on crucial points, etc. And apparently he spoke to Boris Becker a number of years later and told him by the way, did you know that you had this tick? And he said, I thought you were reading my mind. So I just thought it was a remarkable story of the attention to detail that you have to have at that level of performance, that the smallest thing like that can aid well. It can give you a way, but it can be picked up by another professional. Well, Carrie, what have you learned this week? Mine is no, and there are interesting things that I learned on an average workday, that I've had this fingers travel 12.6 miles. That's a day. So I'm thinking I don't actually need to join a gym because I do a lot of typing every single day. It feels good. I feel good about myself now as well. You know, I've been a lot of exercise, I think today. I'm all of very musly fingers except Owen who's musly all over. Ross, that was a good one. Thank you. Again, to what have you learned this week? Oh my god. I've learned that it's super hard to enter a country, want to leave it during the pandemic. Basically, I'm traveling from Russia this week to the UK. And I, yeah, the amount of documentation you have to read and sign up to. Yeah, just a warning to anyone looking to travel. Just, yeah, make sure that you read three or four websites, including the government itself. Just as well, you're the research guy then. I know, I know, I go my spreadsheet open. Having the time of your life. Yeah, no, I'll probably start. Well, mine is that way back in episode 2008, workplace culture in the age of COVID, I shared the thoughts of Harvard's Michael Meena, who argued that the route out of lockdown was for rapid low quality at home testing. Low quality is fine because it's cheap. And as long as everyone is testing themselves all the time and isolating where required, there is also be accurate enough to drive down transmission rates. It was really a, you know, perfection is the enemy of good enough argument. This is particularly important because one third of people with COVID are asymptomatic, so you might have it and not know about it. Well, such tests are now available for free to anyone living in the UK. I ordered my pack of seven this week and I took one this afternoon. And it is unpleasant, quite difficult, involved an awful lot of gagging, but did come back negative. So my reflection was, I think it's good that it's free and widely available, but it's really hard to see how anywhere near enough people at this point, otherwise healthy people are going to gag themselves multiple times a week. And it takes like 30 minutes as well to do the test. So why 30 minutes? You have to wait for the solution to reveal the there is a, oh, you have to wait. Yeah, you get your answer there and then, but it takes 30 minutes for the little strips to light up. So I almost feel like we're working for the government, Ross pushing the like pushing like, you know, testing, testing kit. No, obviously the opposite of a nice tribe. I don't think it's going to work. It's still too hard. Yeah, I didn't even know that was available. I wonder what the take up is like of something like that. You would imagine it would need to be fairly significant for that to have an impact. Yeah, for sure. And also you'd, because we have reasonably wide vaccine roll out at this point are people going to bother. And once it's in at once, I think we're going to do it again, because I don't think I am. What's the rate of like four punters and four negative? That would be in itself. Yeah, anyway, does it matter? You can write a paper on like Anten come back to us next. I will write a paper on your thinking. But you're right. I think that's the, that's the point of it is, eh, it is not that accurate. There's a reasonable chance if it says you're positive that you're not, eh, or if you're negative, that you do have it. But what matters is, in the aggregate, eh, is right sufficiently frequently to drive down transmission rates. You can write a paper on that one. It could be cool. I think so. I don't think so. Probably someone has. Let's not leave it up to us. And that's it. That's all from us. If you'd like to get in touch with us, we end thing we've said on the show. I'm on Twitter at Rosskarner EW. The win is at Owen Ferguson. Gent is at get off me, I. Well, we spell that in the show. Don't find me. Don't worry. Yeah. And Carrie is at EW and school. We say great. You can also tweet at a road underscore works and we'll put links to all of those details in the show notes. For more from M road works, including details of our mind tools toolkit or custom e-learning or our benchmarking service, you can find that at emeraldworks.com. The new platform is launching this summer, but start now to make sure you get early access. We'll be by next week. Until then, bye for now. (upbeat music)