Measuring Return on Investment - Best Practice

[Music] We all know that coaching works, but how do we specifically measure the return on investment? This week we continue our discussion about best practice in the implementation of a coaching culture in your organisation. We look at the ways that we can measure the impact of coaching on individuals, teams and the whole organisation. Join Natalie as she introduces you to a key framework for measuring return on investment with current examples from clients that we've worked with. [Music] Well, hello everyone and welcome to our coaching cafe. My name is Natalie Ashton from OpenDoll Coaching. It's great to have you all. Join me for this webinar on best practice in building a coaching culture. And today we're looking at return on investment. And for those of you that are listening to this recording, just letting you know that we actually did this webinar last Friday, but I forgot to record. So I'm re-recording this for you, but I won't have everybody's comments. So hopefully you'll still get heat to the information out of the webinar. I know you will. It's just that we won't have all that additional shared knowledge that we already have. Before I begin, I'd like to acknowledge your traditional owners, the custodians on the lands on which we're all listening to this webinar today. And a continuing connection to the land, waters and communities of Australia. We pay our respects to them and to their elders past, present and emerging. So the agenda for today, yes, we are live in this recording. We're going to continue our discussion around coaching culture. The last two weeks, we looked at the coaching culture framework and Bridget last week introduced you to the concepts of organizational artifacts. And what are we thinking about when we think about organizational artifacts? So great our webinar for you to catch up with as well. All of those things that give us evidence that coaching is actually part of the organization and we're rolling this out as a culture. Today we're going to be talking about return on investment now. I could talk about return on investment forever because I'm really passionate about the topic and we need to be very confident and clear about how we measure return on investment. So that's what we're going to be focusing on. I want to talk about best practice, give you some best practice examples of return on investment. And as you know, if you are joining us for the first time, we are all about creating a community. We come together every Friday, 12 noon at Melbourne time to share ideas, to explore topics. Open dog is your heaps of extra knowledge and extra tools that you can use in your coaching practice as well. We want to share our learning experience, have thought, provoking conversations. And of course, if you are doing this because you also want your ICFCC ease, your continuing coach education units, then you've come to the right place as well. All righty. So as I mentioned, two webinars ago, we actually introduced to the coaching culture framework designed by OpenDore, informed by all the coaching practice we're doing and by some of our amazing top clients. And we talked about the strategy and setup of coaching and how we have to have a set of enablers, a set of factors that enable the implementation of coaching within the workplace. And we are definitely going to get to the point where we need to look at evaluation and feedback loops as part of that enabling strategy. And today, that's really what we're focusing on. How do we enable coaching within the workplace? How do we support it through evaluation feedback loops? But most importantly, how do we measure the return on investment that we're getting from the investment in coaching? Now, some of you might be doing this as a business case, so you need to put the business case forward for coaching. Some of you may actually be doing this because you're evaluating the programs. And we've been involved throughout the last significant number of years in both those scenarios. So I want to share some of that experience with you today. For those of you that you don't know, everything that we're sharing over the last few weeks is featured in our diploma of organizational coaching. I absolutely love this program, not just because I designed it, but we really go into the detail of coaching culture and the implementation, the strategy of implementation. And we also talk about advanced coaching tools as well. So this information is featured in our diploma. All right. So there are many very different ways of measuring return on investment. And when we think about it and we were getting feedback from the people on the line when we did the coaching cafe, there's cultural measurements we can do. So anything where there's a cultural survey, there's transaction-based measurements that we can call upon and we can think about. There are customer-based measurements. So anything in terms of customer satisfaction, increases in customer awareness, all of those kind of metrics come in here. There's find natural measurements that we can actually look at as well. That's what we're focusing a lot on financial measurements. And we can also measure those results by individual team department, the whole organization, or perhaps even the region. Now, when it comes to measuring return on investment, people ask me, well, how do you measure? How are we going to measure the return on investment from the program that we've implemented? And the question I always ask when I'm doing that initial needs analysis is, well, tell me now what you have in place, what are the base lines that you have in place that we can actually use to measure right now today. So when you think about it, if an organization is doing pulse surveys or cultural surveys or engagement surveys, we have a series of metrics out of those surveys. We have a series of measurements that we can use from those surveys as the baseline. That's very important because quite often, we can't measure what we can't measure. You know, we're not quite offered. We can't measure what we can't measure. We can't measure if we don't have some form of baseline. So we need to be clear about what it is we're actually trying to measure and exactly how are we going to do that. And as I said, the best thing to do is go back to what are you currently measuring now? We worked with an organization in major roads and they were measuring the number of transactions in terms of the number of transactions fees and tolls they were processing. But also the number of things like number of calls, the turnaround time of calls. So if you think about call centres, they have a million customer-based measurements and they have transaction measurements as well. So when we've been working like in the call centre environment from the coaching point of you, we're going lay out the metrics now. Let's talk about what kind of change we're actually aiming for. And then we can insert the coaching intervention. And then we can measure at the end of the coaching intervention. We can come back six months later and say, hey, what happened in terms of those measurements, those indicators that we were focusing on. Now, if your organization or your team or your department that you make your don't have those measurements, then we need to agree as a team, what are we going to measure? So can we in some way do a survey or can we do some sort of pulse or can we do some engagement survey to get some measurements so that then we can move on. Now most organizations and teams will have their measurements. Maybe it's a safety measurement. Maybe it's a sick leave. Maybe it's about absenteeism. Maybe it's about some of our productivity. We want to capture all of those. And as I said, what we're thinking about is what is the baseline now? Then we insert coaching. And what's the difference? Say in three months time or six months time or in a year's time. Now, I always get asked this question on any program or anything that I'm doing, where I'm talking about returning on investment. I always get asked the question, well, you can't possibly say that coaching did all of that. So is that what you're saying that coaching achieved all of those outcomes? And of course not. We're not saying that coaching achieved all of those outcomes. We could never claim that because at any one point in time, there's a number of different things that are happening within the organization, within the teams, within the individuals that can make a difference to the change in results. So we're not going to claim at all, but we have to think about, well, how much of it could be claimed, maybe 50% of it? Was it the majority? Was it was was coaching really the only main thing that we did? Or was it part of a change program? So we have to be very clear about how much are we claiming so that people don't go, oh, you're trying to say that, you know, you that coaching attributed that, whatever it might be. Now, in many cases, in many cases, it is actually true. Coaching has been the only thing that they've done differently. So they get a very big change in results and they haven't really done anything differently. You could say coaching attributed, you know, was attributed, you could attribute the results to coaching. But we never do that. We discount it anyway because otherwise it gets a bit fanciful and people start not to believe us when we talk about results. So if I'm talking about there, what are we measuring currently? What can we use in terms of current measurements? How much you're coaching attribute to it? So there are some of the makies we think about in terms of different types of measurements. But I want to move on and actually just talk to about one, one way of measuring coaching programs and the coaching culture within your organization, which we use extensively. And I'll just give you a bit of an insight into this model. And I'm talking about the Kirkpatrick evaluation model. Now it was originally designed back in 1959 and you know me, all of those people that have worked with me. I am all about taking models that have stood the test of time, updating them with our thinking and our coaching. And they're thinking about how we can actually apply those models, which are standards, stood the test of time, how we can apply them to our today, to our modern coaching practice. Because we've wanted them to do the coaching, we might, everything that we do it to be practical, we're not trying to reinvent the wheel either. So let's go back to 1959 and Donald Kirkpatrick invented the Kirkpatrick evaluation model. They also wrote a couple books back in 2008 and 2010. So the references are there for you, a 2008 and 2009. So the references are there for you as well. Kirkpatrick came up with four levels of evaluation. Now, why we like this is because it's four levels of evaluation on training programs, on development programs. So it was specifically designed for how do we measure the return on investment, how do we measure training interventions, development interventions, training program interventions. So specific to what we're doing in terms of coaching, we can specifically use this model for our executive coaching, our one-on-one coaching for coaching teams, and particularly as we're using it to measure the returns on coach training that's actually happening within the organization. And we've done a couple of very significant studies using this model. So here they are, it's level one, four different levels that you can measure the returns and measure the impact of training programs. Level one, the reaction level, level two is a learning level, the behavioural level is level three, and then we have the results level which is level four. Now, let me explain these to you. So here they are. Level one, the reaction level. This is the extent to which participants reacted favourably to the training program. So what was their reaction to the training program? And quite often this is we catch a level one, we catch a level one out of feedback forms. So this is once the training program, once the coaching has been completed, we do a feedback form and we're finding out how did they find it. Did they react favourably to the training program? And of course, we've got lots of data that we can collect and we can use. So it's interesting to me that quite often we're like, "Oh, what was the result?" You know, et cetera, level five, that's extremely important. But what we know is that if they didn't react favourably to the training program, nothing is going to happen after that anyway. They went on a training program, they didn't get what they needed to get out of it, it was a waste. So you've been on training programs where that's happened, you're like, "Oh, how long is it going to be?" "Book to I get myself out of here." So we think about the big stint, which the participants reacted favourably to that training program, get it out of your feedback forms. Now it's one thing to react favourably to the training program, but did learning occur. So have they learned themselves something? Level two is defined as the extent to which learning occurred, which I'm very interested in. Did their attitudes change? Did their knowledge increase? Has their skills improved? Again, you can get this out of the feedback forms, but your feedback forms need to be asking very specific questions around learning to gather this information and our feedback forms that open door do exactly this. To what extent did you own knowledge increase? What's changed during the program? So we're looking for attitude, knowledge, skills increase, and that's signaling that learning has actually occurred. So level three is a level of behaviour. Now it's great that they've had a good time, they've reacted favourably, they've learned themselves something, but what are they going to do? What are they going to do with it? And you know me if you've worked with me that it's all about implementation back into the workplace. I want to see, we want to make a difference back into the workplace, we want to give people the practical tools that they can then go and use back into the workplace. So to what extent did their behaviour change? And to what degree did they apply the learning back into the workplace? This is what we're interested in. To do that you're going to have to catch up with them three months later, six months later and ask them the questions. Now in our feedback forms for our certificate for in workplace and business coaching and our diploma of organisational coaching, we ask people at the end of their residential program what happened and what changed. But then once they've actually completed their whole program like maybe six months later, they're re-surveying them and we are trying to understand this level three behaviour change and it's very powerful and it's fantastic to read about the degree which they applied learning back into the workplace. Now what's important about this is that most people are focusing on level four and they forget that these level one, level two and level three indicators are extremely important. So maybe you need to think about how are you measuring return on investment not just in six months time but how are you measuring it at each of these indicator points because you might have different groups going through your coach training, summer only at level two but others are at level three or others market level four where we can measure even more specific results. So thinking about return on investment not just in a point in time like let's come back for six months time but actually thinking about it more as a journey across the learning journey that you might get a pick up measurements of evaluation measurements of return on investment. So level four is often what everyone focuses on and that is the degree to which the targeted outcomes for a cheat. Now this is very important. This is why we need to ask at the very beginning of the actual program we need to ask what are the targets, what are the target outcomes that you're expecting to achieve from this program. If we don't know that then how can we measure it in the end whether we've achieved it or not. So we need to be clear in those briefs that we're doing in those proposals that we're writing in the conversations that we're having with stakeholders what are we trying to achieve what are the targeted outcomes and that's when you come back here six months time ago to what extent we actually achieve from the issue I've seen is quite often programs implemented without a real clear link through to the strategic direction. So it's super important that we have the top part of our coaching culture map, excuse me, in place so that it is linked through to strategic. It is linked through to the workforce strategy. And with that in mind, then we go, well, we're trying to achieve that part of the workforce strategy. And if that's the case, then let's measure against what we set out to achieve. If our target outcomes were specific metrics like around customer calls or customer satisfaction or engagement scores or whatever it might be, we worked on programs where the outcome was an increase where the target outcome was increase engagement across the middle management cohort. So we want to know what's the level of engagement now, what's our target. We're introducing the coaching intervention and then we're going from there. So we're thinking about, okay, we're very clear about what we're actually going for and what the training and the coaching is actually going to do. What we want it to do. So again, go back to those different metrics that we share ahead on the previous slide. We have a really good book at it, set the targets from there and then we're introducing the coaching. So I just wanted to give you a couple of level two and level three examples. Now, you know me. This is not too brag about our results. This is just to demonstrate what we're looking for when we're actually gathering that feedback so that we can be hitting the different levels of evaluation. So learning to, level two learning. The person said, my effective list skills have matured, my knowledge of coaching and techniques and so my effective list skills have matured, knowledge of coaching techniques and their practical application is what they feel like they've learned. So that's indicating that if you see that there's their knowledge of coaching techniques has changed and their practical and their knowledge of how to implement has changed and their listing skills have changed. They've obviously learned something. Another person says, we've completed the course and changed my focus from mentoring to coaching in many cases, which is certainly a change. I also did not understand the difference until I completed this course. Now that's great. We've really signaled learning through that comment. But previously they thought they've had this massive change for me talking to Coaching and they didn't understand the difference. Now they do. That's a great signal of learning. Signaling behavior. Let's talk about this one. So this person said, "I'm applying my skills." Yes, that's what we're talking about when we're talking about signaling behavior, behavioral implementation back into the workplace. "I'm applying my skills through practice during the one-on-one monthly conversations with my team members, conversations with my children at home, as well as my team leader at work." So we've got three examples there where they're implementing, they're taking those behaviors back into the workplace that is signaling level three return on investment. Practicing during my one-on-ones with team members, conversations with my children and a team leader. The other one here is, "I now routinely default to using Coaching conversations." Right. So we're talking about that, that's real signal of behavioral change, implementation back into the workplace. These conversations have become more natural over time, learning and implementation. And I find myself choosing more appropriate tools based on the conversation. So we're gathering this information because if we're talking about return on investment, these are the kind of quotes, these are the kind of statements, these are the kind of feedback that we're actually trying to gather. So that we can demonstrate the levels one, two, three levels of measurement. We can turn these into actual numbers as well. And again, it goes back to tell me what numbers you are looking at now. But it's not all about the numbers. What has to happen before we see changes in numbers is we have to see changes in learning. We have to see changes in behavior, then we get the results. If you don't get level one, two and three, you don't get those results. So that's why we need to track it across the different levels. But some of you may want level four. Let's go to Squadron Leader Anita Greene, who heads up part of adaptive culture with our Air Force. She did a presentation to OpenDore's International Coach Week in 2021 and presented at the conference, the impact and the changes and the coaching program, the Air Force has implemented. They implemented, they have been implementing for a number of years now, the certificate for in workplace and business coaching. And then coaches who have gone through that program have then rolled out a coaching 101 program. But you believe at the time that I'm actually recording this, over 9,000 people have been exposed to Air Force's coaching 101 program. And the tools that they're using are like the top five tools out of the coaching assert four, our program. But what's really important is that we've enabled those coaches to go on and make a fucking difference to roll that program out to over 9,000 people. So again, you can see that the different levels, do they get a lot out of the program? Yes. Did they learn themselves something? Yes. But had they gone and implemented back into the workplace and one of the measurements were how many coaching 101 programs have they been delivered? What difference are they making in terms of the roll out of coaching and the coaching culture within the organization? Now I love these statistics, I'll share these with you as well. From these programs. Air Force surveyed the commanding officers of the units and said of Air Force, the commanding officers and said, "Tell us to what extent you've seen behavioral change in these key indicators." So the targets that they were looking at was collaboration, communication, innovation and overall behavioral change. And this is unbelievable these statistics. In collaboration, 53% of the commanding officers said that they saw a moderate improvement, 84% of commanding officers said they saw a significant improvement in collaboration. Think about that. People are working better together. Imagine what they can do if there's been a significant improvement in the way that people are working together. Communication. 38% thought they'd seen a moderate improvement in communication. 63% of the commanding officers said they saw a significant improvement in communication. And do you know me? Communication is the center of everything. If we communicate better, that's going to have an impact across our results. These are very significant statistics that measure the difference that coaching is actually making. Innovation. 63% said they saw a moderate improvement in innovation. 25% said they saw a significant improvement in innovation, a quarter of the workforce. That is significant improvement in innovation. And overall behavioral change, a moderate improvement, 63% said they saw a moderate improvement and nearly 20% said they saw a significant improvement in overall behavioral change. So these are just wonderful statistics. They're real statistics, they're practical statistics. They give us the evidence that we're looking for that coaching is making a difference in these key indicators. One of the key indicators was behavioral change. So for those of you that want the numbers, these are the numbers. Then we go, well, how does that translate into dollar terms? Well, you can actually do those calculations as well. It's part of our deployment program. I don't have time to take you into turning these numbers into physical dollars, but I can tell you that it's really, really significant when we start to do that. We get significant dollar returns on the investment or dollar savings as well. So there you go. You can pick up a new presentation if you like. Just go to our website and look for International Coach Week 2021. And so presentations called how coaching enables Air Force and how coaching has fundamentally transformed Air Force. So pick that webinar up. It's very inspirational and it includes a lot of statistics that they've actually seen as well. So there you go. If you are listening to this, it's good to know that we are actually having a slash sale. So we have had a significant change in our diaries for the next three weeks because of COVID and travel restrictions. So we're having a slash sale. If you're interested in doing our certificate for in workplace and business coaching, a once of opportunity to do this course. It's coming up next week and we're offering it for $3,950. So please call us if you've got time and a bit of a professional budget. Come and join me on the program. I'm delivering it. It's going to be great because it's like let's do this. We've had three major courses move. So we're just going, let's see what else we can do in terms. I would rather be coaching than delivering and doing compliance work, which I'll have to do if I'm not coaching. So anyway, help me, help me procrastinate on that and come and do coaching. So there you go. I hope you have gotten a lot of thoughts out of that. I'm sure you will have. You start thinking that return on investment across those different levels, specifically designed for training programs. Level one, level two, level three, level four. You've got some good examples there that you can use as well. Please keep us in touch. I'm very keen to hear what you've been achieving, what you've been achieving from a return on your business point of view. So please keep us in touch and I look forward to catching up with you at our next webinar. I will say goodbye for now. [BLANK_AUDIO] [BLANK_AUDIO] [BLANK_AUDIO] [BLANK_AUDIO] ''Hются" Angy They're big You have to give me a try I'm not a big fan of you