Hello and welcome to the Experience Perspective. This is an Ipsos podcast which is for anyone interested in improving experiences for your customers and colleagues across all sectors. Across this series we will bring you insights and inspiration from around the world. We'll post me a chat with experts within the field. Now let's get started. Hi everyone, thank you for listening to the podcast. Another interesting episode ahead. Before we get there I just wanted to say a quick reminder that please do like and follow, wherever you listen to your podcast. That way you'll get to know first every time that we release new episodes. As I said today another interesting conversation on the way and today's guest is Andy Firth. Andy you're a colleague of mine at Ipsos with a very particular area of expertise and specialism. So to start it'd be great if you could introduce yourself and tell us a bit about what it is you do. Thanks Rob. Hi everyone. My name is Andy Firth. I am head of advisory, the advisory and analytics team at Ipsos channel performance UK. I see the role of my team comes in really at the very end when all the kind of hard work's been done, the operations and we've got this great treasure trove of data. And the role of my team is to go and analyze that along with client data and basically tell our client what it means, what's the so what, what's the commercial impact as well as the now what to help them kind of action our insights and drive their business forward. That is amazing. And actually it's it's kind of mirror image of that topic that we're going to be focusing on today because like you say you come to this data so often at the end and you're looking at this data on mass and finding these themes. But the purposes of today's podcast we're going to rely on your expertise to talk about if that's what we can get out at the end of a project. What are the considerations that you should have at the start of a project to ultimately design and build a mystery shot program that can give you the greatest amount of use and actionable data at the end. So yeah, really looking forward to getting your advice and steer and insight on that. But before we start, maybe we'll start a little bit broader because I think the first question that would be great share with us is when it comes to mystery shopping as a methodology, where do you think it particularly excels or where do you think the best uses of mystery shopping tend to be in your experience? Well, the division mystery shopping sits in with an Ipsar, who's called channel performance and there's a reason for that because I think it's the best methodology for objectively assessing the performance of a channel. So that might be telephone, online email, but predominantly it is face-to-face interactions and that is often or mostly in stores, but we do plenty of work on transport for example and also in delivery. So the real strength of it for me, the client is knowing what is happening on the ground, that's saying a store environment that gives you a real detailed objective assessment of what is happening, not based on recall and predominantly focusing on what the employees are doing and how the operations are working. You know, we can ask mystery shoppers to provide feedback on their own feelings, that's more the remit of customer experience research, but for me, this is how your employees performing, how your operations are working and we know that's important because when we put it together with commercial KPIs, for example, we tend to see really strong relationships and I don't think there's another methodology that can get to those insights, get to those kind of moments of truth, those absolute moments of interaction between the business, the entity and the customer. Yeah, absolutely. So when you think about, again, obviously we are thinking about what can come out at the end of this and the many insights that can be drawn from this type of data, but if you go right back to the start and if we're advising maybe an imagined client or even a group to think about what needs to go in initially, what are the ingredients that make the most impactful mystery shopping programmes? What do you think of the key things that clients, organisations individuals should consider at that stage? Yeah, well, I mean, in many ways, the mystery shopping is a research method. So you need the same pillars as any strong research programme. So at the core of that, of course, is the questioner. I mean, the questionnaire or survey, whatever you want to call it, has to be well designed and it has to be designed with the business objectives in mind, of course, and from an analytics point of view, there are elements for a questionnaire that really help us in analysing the findings and understanding the why. So we have methods and structures around how we create really good questionnaires. And actually, mystery shopping questionnaires can be quite long. No questionnaire should be too long, but they tend to be more detailed than say a research questionnaire because the shoppers know in advance what they're observing and what they're answering. Of course, you have to have a really good sample, you know, for a lot of our clients who own their own network, you would tend to advise that we need to go and assess all of your stores. But a lot of the time we use smart sampling and say, actually, if we're going to focus a bit more resource, let's do it on the maybe the really big or quote unquote important stores and possibly those stores that are under performing. Now, if you're reclined with the biggest states, you'd probably want to focus on the ones with the highest football and the under performance because that's where the most potential is, for example. Obviously, we do do a lot of mystery shopping where we are going into third party retailers. You can imagine a manufacturer wanting to know how their products being recommended in another partner retailer. So again, we have to design the sample around that. Of course, samples really important. The third bit that really isn't using other research methods is of course the scenario or the mission you're giving the shopper and their instructions around that. You know, most other research is a survey with a customer. We are instructing shoppers to go in and follow a mission so we can assess that interaction. So again, that's really important to get right. And of course, as I was always say, the analysis of all that data is really important, but we can't do a good job with the analysis unless we got the data itself. And that relies on the survey and a good robust sample. So when you say good robust sample, what are, and I know these vary based on the client based on their estate, sometimes based even on their industry and setup, but is the kind of any kind of broad rule of thumb about the size of a data set you would need to be able to do some of that analytics, some of that deep dive work on it. What kind of sample sizes broadly would we be thinking of? Well, I mean, for analytics, the more the more the merrier. Yeah, obviously, you know, we're doing quite a lot of work with our US colleagues who have big programs. I mean, obviously, the US is this kind of the size of Europe, thousands and thousands of surveys, which allow us to just do so much good work. I think we would always say that maybe a hundred is a kind of minimum for some sort of top line analytics, but we really prefer it against the realms of three and five hundred if we're going to do some of the analytics modeling, including combining MS State with commercial KBI's that really gives us confidence that what we're saying is true. If you like, and we can put our kind of statistical confidence levels on that, well, that's an issue because already we're starting to move from one theme here into something that feels like it's got a real impact. So because on the one hand, a mystery shopping assessment at an individual level is at the heart of the interaction often between clients and customer, but also is a moment in time, right? It's a single snapshot in a single location, but once you start building those numbers up, like you say, that robustness, that statistical confidence that builds with those additional data points, you can absolutely see how that can, you know, be incredibly useful when it gets into the hands of you and your your very expert team. So sample sizing, I think that's that's really useful and really handy. I want to bring us back really quickly to a point you made at the very start. So you've talked about the importance of making sure that your questionnaire, times with and reflects kind of key business objectives. Is there any particular themes that match very well, all ways that maybe clients can step back and do that type of checking, that type of checking and balancing? What types of things are you talking about there? Well, the first question my team always asks, and that maybe at the start, it maybe wanted to programs running for us to be able to, you know, provide the client with insights that they can use and are meaningful is why are you doing this mystery shopping? And I think everyone in our team from from operations to project management of allodators, allocators, analysts, really, they try and understand that. I think it helps all of us. And we tend to bucket the reasons into four groups of this helps. One of them is just what we call compliance. And these will tend to be run by compliance teams in inoclients. There is you take age verification for example, age verification teams in a client that sells over 18 products for example, they need to know when that product is being sold either in store or delivered that the person buying it is being asked for their ID. It's almost as simple as that. But that has such an impact.
if you get that wrong. And really the only way you can measure that objectively and not based on recall is mystery shopping. But yeah, that's compliance. They don't really care a lot of these teams about whether the person's been friendly or did this or did that. It did day after their ID. And there's a lot to compliance whether we do financial services. That's just one example. The second one I'd say is sales conversion and that tends to happen in retailers who own their own network or luxury or auto. Where really when you think about it to the primary objective is to sell that product at the point of purchase. If someone comes in and cries about product X, the role of that employee or that store is when you think about a store that expense or a dealership that are expensive things to run. So really they need to be driving revenue. Plus keeping their overheads low. So what we're measuring is the likelihood of that sale being converted. So we call it sales conversion. I mean, sales is sometimes a dirty word. It shouldn't be. If you don't make the sale, you don't have a customer. The third one very quickly that's related to that is what we call product recommendation. And that's where manufacturers are looking to assess employees in their third party retailers who sell their product. So think electronics, bed mattresses, phones. How else do they know how their product is being spoken about in their third parties again, objectively in detail without recall to that mystery shopping. So the scenario is the customer goes into a third party electronics store and says, I'm looking to buy a TV. What happens? What's recommended first and the manufacturer of those TVs is really important information for them. Yeah. And the last one is what I would call kind of custom retention. And that I guess when we use shoppers who are already customers of somewhere like a bank or they are already buying something like a hospitality like a meal or petrol in oil and gas. They've already bought it. So we're not trying to convert them. But what that location is doing in the employees and maybe trying to get them to buy a bit more, which is an upsell. But also make sure that those customers are going to come back again and the loyal and they've had a good kind of experience. So that makes sense. We kind of bucket it into four. I'm always happy to hear if anyone thinks their mystery shopping program doesn't fit into those. But that's quite a neat way of covering things out because that allows you to focus on the business objectives and the insights that are important. Nice. And I think you mentioned there. So the takeaway for me there is you've obviously got call measurements that are important to be included objectively. You know that your real core goals are there. But also there's an element there that you're describing that it also needs to reflect in a way. The customers journey or the real world experience. And yes, there will be a number of other questions that will add some color and add some detail to what those interactions look like. But those core measures are all important. So the counterbalance of that then I suppose you've got these kind of two contrasting elements. You've got one thing that you want a mystery shopping program that absolutely aligns to your goals and matches your real world kind of customer journey to capture all that important data when that data starts to come back in. And I know we've talked about there's an analytics element that can certainly be added to this. But what are the best ways to use a mystery shopping program. So if I'm a big business and that data is coming in, I'm imagining there's quite a lot of that data that can be actionable that can be informative for clients. What do you feel some of the best uses of that can be? But if you take a classic mystery shopping program where mystery shoppers are assessing the stores of that client, they say they got a thousand stores, I don't know, Fashion Retailer. Are we going to every store? There are two key ways that I would say they should use it. There is the tactical level. So every store would receive a really clear report on what happened at the assessment in that store. And the store manager as well as their regional manager should be taking those insights and kind of actioning things and saying this is where we fell down. This is important. We need to do something about it. Yeah. That's tactical. The other end is strategic, which is really where my team come in. And this is where we are talking generally to head office people. You know, these might be head of ops, retail people, whoever's got a stake in the retail network. And we are telling them these are the big themes we are seeing. We can see there is a relationship between these types of behaviors or processes and your commercial KPIs. So this is where you need to be focusing at a strategic and the tactical level on improving those. And if we can identify the kind of levers, investment levers, you need to build to improve those behaviors. Now that could be training better, resourcing, stronger employee wellbeing, whatever it is. Yeah. And we can help you improve those, which means you will see stronger commercial KPIs, which at the end of the day is what senior people in businesses ultimately care about. It is that commercial background really and improving those KPIs. Well, that actually was going to be my next question. You've started to answer it actually really or in fact, have answered it really well because I was going to also ask about how these programs tend to evolve and what people can learn, you know, how it can help shape your business and kind of shape your actions. But I think the two examples you've given there are great because I'm assuming that, you know, we identify this area of focus. We're talking to the client about, like you say, maybe training or a level of investment in resources. But then I suppose what you're looking at in the next wave around a mystery shopping is the impact of that, right? Whether that's moved certain needles, whether certain metrics have changed, whether the feedback from mystery shoppers is kind of reflecting those changes. So it does end up being a very much an evolutionary process, but a continually relevant process. I suppose because there is that connection between, you know, ourselves and the clients that you are working with them as you make these evolutions and forever to slightly change these focuses to measure these key changes and key steps. That's the ideal. You have the work closely with your client. After quarter one, you said, these are what we might call the key behaviors. This is what we think you can do to improve them. It's kind of incumbent then on the clients to try and do something about it. We can help them activate it, but then the beauty of it, if they have done something about it and we see those behaviors improving and we see the commercial impact, you can track that over time and once you prove that, you know, they might have sorted this one bit out really well. Okay, then there's something else now that you need to do because you're never going to reach absolute perfection, but you should be improving, you know, wave on wave almost if you can iterative approach as you kind of describe. And like so, some of the analytics elements, I mean, even with some of the key measures, you know, the core questions we were talking about, those are all things that hopefully you're seeing have an impact not only within your mystery shopping program, so an improved score or improved performance in a certain area. We would argue that a lot of the time we're seeing that in the broader business as well, you know, that those interactions and those performances at a store level are actually having a far bigger impact further afield and been seen across a whole host of metrics in the business. In terms of my next question, you've talked to little about the objectivity and the fact that mystery shopping as a methodology is perfectly suited some of these things. How would you say we would ensure that the observations of mystery shoppers are captured in such a way where it always is usable. It's always reliable, accurate, unbiased. What are the kind of steps that need to be considered when building a mystery shopping program in relation to those do you think? It's a great question and what am I kind of favorite questions if you like. I mean, the first underlying fundamental principle is that the shoppers are anonymous. That's the whole point of mystery shopping. The staff member can never know that they're a bit sound obvious, but the way you do that is the shopper has to be as real as possible. Employee has to think that they are a real customer and we think we're the best in the business at doing that. At the same time, while the customer is doing that, and you know, the closer they are to reality, a lot of them are customers. This is what they are doing, the better, but at the same time, they are observing the things that we need them to observe. So that's the first thing. The second thing and I think in all the analytics we do, we tend to find that it is the objective metrics that we are measuring, tend to come out as the things where we can find a strongest relationship with business outcomes, such as commercial KPIs, and also they tend to be the things that are most easy to action because they are objective. So one example of that might be friendliness. You know, we would never say don't be friendly, but the difference between a question that says how friendly was the member of staff on a scale of one to five. We'd probably include a question like that. Yeah, but the elements that kind of characterized friendliness like did they say hello? Was it a warm greeting? Did they ask you your needs? Did you get friendly farewell? They're the objective metrics that we would score whilst maybe having a how friendly were on a scale of one to five in there, but not scored because it's too subjective. Yeah. And what do you do with that anyway? Yeah. Okay, go away. You've got to make that store a bit friendlier or only 60% of your stores actually gave this greeting. You need to work at which stores they were and how we can help improve that metric. Yeah. And does that make sense? That's the kind of, and that's just based on a lot of analysis. It's not just me, you know, hypothesizing that. No, absolutely a new. You talked about the very start just to bring us back in of a slide. You talked about, well, I suppose there's two things here. One of the things is we've already talked about creating mystery shopping scenario that follows the customer journey that has these really key objective kind of touch points that we want to measure in the middle. But you also talked about the fact that one of the things we pride ourselves on within Ipsos is how well those mystery shoppers are prepared for those assessments and how well they know those scenarios and the things to look for. What is it about that that's so important? Yeah, I mean, as I said, my view is that the job of the shoppers is to assess the employee and the operational processes around it. There's no other method that does that, unless you can tell me one. I don't think there is. And that's what sets the USP. That's what sets mystery shopping apart. So, it's a really important job for these shoppers and for any shoppers that are listening, what you do is absolute foundation of everything we do. We couldn't do our analytics without good data. So they have to be that real person. As I said, a lot of the time they are. And I think that really helps if they are a real customer. But they have to be because it helps them just, they're just doing it as naturally as possible. But they are also observing very specific things and the sooner they can kind of relay those observations in the survey after they finished the better, I think. Because one, again, one of the USP's is that this isn't recall. They are looking at it in the moment, which is, you know, voice of customer customers is a really important. Of course you need to know what your customer thinks in their experience. But asking them an very objective question, I don't know how long did you queue for afterwards. They won't be accurate. Whereas a shopper will have timed that queue to perfection. And we can say the clients look, this is the, this is the you need to get queues under X, Y, Z to improve your commercial capabilities or whatever it is. And I think that's really important. So highly trained shoppers, these really, Maconnall Wellam, the stud scenarios and some of these measures requiring absolute accuracy. You know, this isn't about, sorry, I was just going to say I'm backed up by, you know, photographic evidence or in some cases, the video recording. That's as objective as you can possibly get. Compare that to other methodologies or other things people might be used to in these same retail settings, like customer satisfaction surveys that the general public might fill in. In other aspects, it's not that there isn't a place for those, but this is a specific purpose that mystery shopping is able to deliver that sits outside of the remit of those other things. And I think that's, that's such an important aspect, isn't it? So much of the time people might look at this data in combination or be comparing this data. But certainly what mystery shopping is doing is following a very distinct methodology for a very distinct purpose and delivering on a very specific mission. So yeah, thank you. That was, that was really, really useful. And if we think about age verification and ID checks, couldn't contact a customer after they've been, they've gone and bought, I don't know, alcohol or something. And that's, did it was your ID check? I mean, in A, you'd have to find customers of a certain age rather likely to get it. And they might remember you have to do that objective, really, and without recall to be as accurate as possible. Yeah, you know, that that's really helpful and really clear in it. And I know we've kind of hutsched on a few of these already, because you've mentioned certain industries, where it's well used or certain industries, where there's engagement with clients at the moment. But thinking about some of these, do you have any, I don't know, favorite examples of where you found projects to work really well or been really impactful or, I suppose, the reverse, where we've had to make adjustments to the way people might be assessing their processes. What are the standout stories for you when you look across mystery shopping at the moment? I remember the first time that we were given sales data from a client, I think was sales conversion data. It was a luxury chain. And the first time we, we put that together with our mystery shopping data and you just saw a beautiful pattern. I almost described it as a work of art, which is quite sad. And you could see that their low performing stores are mystery shopping and actually see their sales decline. The medium performing stores have seen a small growth and that top performing stores had seen a really nice growth of X percent or whatever it was. We were just like, that's what more do you need? I mean, there's a lot more to do after that. Don't get me right. But what more do you need? And you can put nice, you know, pound, pound signs on that by saying, look, if all your stores had done this, we think the gap is 10 million pounds or whatever it is. That's really nice. And the role of the employee is so important in that sales conversion. And that's why we see strong relationships. At the other end, we're doing loads of work, oil and gas in the US at the moment. And the metrics that are influencing sales are just completely different. Obviously, it goes on saying they're very infrastructure driven, very processed driven, you know, did you consistently have fuel available? That is pretty important. So, and so the difference in the diversity in what we're measuring here, not just the sector, but actually in the metrics that matter, I think it's just really interesting and shows the massive depth and spread of what we actually do in mystery shopping. It must be such a delightful thing because, like you say, to have the confidence, you know, when you're talking about, you know, for a number of our clients, I would imagine large levels of investment, you know, to move some of these needles to change some of these areas of performance. To have that confidence, you know, that statistical robustness that says, you know, based on your data, here is where the investment might be needed, but here is the like the outcome if this lands well and can be achieved. So, you know, that's fantastic. Back to the earlier point, you know, a lot of the time we need quite a lot of data to have the confidence. We will only show data if we are sufficiently confident in it. And I think that's really for Ripsos, an integrity that is so important. And I'm kind of commoditized markets where things like price and geography are important reasons for customer to buy something. These operational processes, you know, they do shift the dial on sales, but you kind of need a lot of data to prove it confidently. So, for people who have listened to this podcast and who are interested in keen to find out more, knowing you as I do, I know one of the ways they can find out more is they can come to you. They can absolutely find money. But what would you advise as a first step? If someone's listening to this and says, you know what, this is a conversation I'd love to have. This is something I'd love to find out more about. Where could they go? Well, yeah, come directly to me. If you're setting up a new program, again, we would like a chat. But Andy, thank you so much. Really fascinating. Walk through the reports of design and the impact of mystery shopping. I'm sure that I've been a really interesting listen for our audience and I know I've certainly loved it. Thank you ever so much for sharing those stories and for taking the time. Thank you. Thank you. Thank you for listening to the experience perspective and Ipsos podcast. Make sure to subscribe to us on all your favorite podcast apps to have new episodes send straight to you as soon as they're published. We're also available on Spotify. If you'd like to reach out to us, please send an email to
[email protected].
PodcastTranscript AI