How Experimentation Drives Innovation.

So I'm the VP of the economics team and within economics we have the economics and experimentation group and as part of that we have the experimentation platform. We want to get to a place where we're running not hundreds of experiments a year, but tens of thousands of experiments a year trying to be an innovation engine for the entire company. Hi, my name is Jenny Matos and I'm the host of the Peace of Me podcast by Zalando. Career Development Advice, industry insights and stories from leaders at Zalando to supercharger career and fashion and lifestyle e-commerce. It's a packed show. So let's get started by meeting my guest for today Gregory Crawford, Chief Economist. Hi, Greg. Hi, how are you, Jenny? I'm very excited to have you here. We talked at the summer party about you being my guest at the podcast at some point and I'm really looking forward and finding actually out was a chief economist actually does. It's a very nice title. It's an excellent title. I agree with this totally. I mean, what do you have to do to get this title and what do you actually do to keep the title? And you're not alone. I think there's many Zalando's that go, we have a chief economist. What does he do? Right? So that's why I'm here. I mean, we will go into that. You also have a little side hustle, which actually is a big one. So also looking forward to knowing more about you being professor Gregory Crawford, but you actually brought three pieces as asked. So what is your close to the hard piece? Let me just say I love the whole idea of this podcast and one of the things I love about Zalando is it's culture and we'll get to that later. So this is one piece of the culture that I very much appreciate. So my first personal item is something quite old and this is I'm going to show it to you. It's a newspaper carrier bag from when I was a kid. So this is from I would say 1980 to 84 something like this. I delivered the Wisconsin State Journal. So this bag says Wisconsin State Journal. I grew up in Madison, Wisconsin in the US. That's the daily morning newspaper. There wasn't afternoon one in Madison and I was the paper boy. So that's object number one. Let's move on. What is your career piece? Okay, my career piece I actually struggled with a fair bit because you know as a professor I really work with ideas a lot. Right. And so I needed an object to represent sort of how I approach like research. And then I thought, okay, well, there's actually one object that kind of works and it also works because when you think of an economist, you think dork. Right. And so here we have. Yeah, many people do it. And maybe you don't, but trust me, trust me. This is a Rubik's cube. But here with the Rubik's cube is the original book that I have from my childhood, which is called the simple solution to the Rubik's cube. And then the third one, my Zellando piece, I also struggled with a little bit, but what I settled on, I would call it a finger spinner, but it's technically called a fidget spinner. Okay. And it's in Zellando orange. Okay, before we go into three really great pieces, let's get started by chief economist. You did say you're a dork. I am. I am. Those were your words. What do you do as a chief economist? So I try to help Zellando make better business decisions with trustworthy data driven insights. That's what I try to do for 25 years. I've been a professor. Right. It's only I started at Zellando over two years ago. And so look, I'm a professor. I go to conferences, I do research, I present papers, blah, blah, blah, what professors do, right. All of a sudden I met one conference and everyone goes, do you know what? Pat Byering was just hired by Amazon to be their chief economist. And we're like, what? What the hell is he going to do for them? Pat's like a friend of mine. He was just like another guy. He was like another professor of economics that was just going to conferences and he just jumped to Amazon doing basically what I said is trying to learn from data to provide trustworthy insight to decision makers to business decision makers. So that they can make better decisions. So new products, new, really, everything. Exactly. So if you want a better demand forecast, if you want to understand whether a new feature is actually moving the needle for your customer. If we want to think about how do we design a quality assortment? Like what does a quality assortment mean? And how do we know if we add a brand or some collection from a brand? How do we know that that's actually quality in the eyes of our customers? So economists actually have this mix of skills. So I study consumers and competition and competition means firms. So an economist with my kind of background were used to thinking through the structure of the behavior that ultimately yields the outcomes that we see. Like how do we get GMV? Well, we buy assortment and we offer it to consumers. So economists are used to thinking about well, how do consumers decide which platform to go to? How do consumers decide whether they value the quality more or the price more or the delivery convenience more, the payment option more? Right. So we're used to thinking about sort of the structure of how consumers and firms make decisions. And then on top of that, we're used to analyzing data. The key thing we often work on is something called causal inference. And so it's the idea of if Zalanda we at Zalanda, we want to roll out a new feature. We want to know its impact and we want to measure that impact. But we can't look to the past. We've never had that feature. We want to measure what's the causal effect on say consumer and commercial outcomes of this new feature. So by doing user tests, forgetting new data and also real life data, let's say. Yeah, yeah. So and basically what we do is we run an experiment and we say, okay, let's take some beta consumers that are interested in testing this or whatever. And we make it available to half of them. We hide it from the other half. We can then compare basically it's like pharmaceutical companies do when they're testing a new drug, right? And that's a core skill that economists bring, but then there's other sort of related skills. I won't go into the details. But but basically we try to learn from data. That's sort of what I try to bring to Zalanda. Yeah. So if now in the beauty section, they want to add a new feature to maybe support or improve the customers shopping experience. They would come to you and your team to test this and make user experience like user testing to see what what what what do the customers actually need exactly. Well, I would say yes and no. So yes, at the moment, but ideally no in the target state in the following sense. So still yes, but not by my team. So I'm the VP of the economics team and within economics, we have the economics and experimentation group. And as part of that, we have the experimentation platform that allows us to run experiments, right? And the goal of that team is to democratize this capability across the company so that anybody like Janine, if suppose you wanted to say which of these three ads for this podcast is going to get people to listen. The idea would be, okay, let's just set up a quick experiment. The punchline here is that we really want to democratize the ability to learn from data, right? So we don't want you to need an economist. We don't want you to need an applied scientist. We want to make it like super light touch. You come in. There's a super easy interface. You set it up. It tells you you need this many users. It analyzes the data for you after the fact. That's how you innovate. The leading companies in the world all have decision making cultures based on learning from data where the key tool when you're learning from data is experimentation. So you did say you have been now 20 plus years of professor and you are a professor of economics at the University of Zurich. Is this still true? It is still true. So I'm, I mean my main job is the land. So I'm 80% at Zalendo and I'm 20% at the University. How do you combine those two positions? It's actually fabulous, right? So as I said, for 25 years, I've tried to learn from data. Now that I've been at Zalendo, I just have such a wealth of experience of how a big company that has lots of data tries to use that data to make decisions. So it's totally improved the quality of my teaching. So the thing that I teach is what's called empirical methods. So it's, it's basically trying to learn from data. All the examples I had for that course previous to working at Zalendo were policy examples, you know, or consulting work or whatever. But now I can bring these corporate examples and it just has totally changed and improved the quality of the teaching that I give these students. And this is a big course. I have 280 students. It's like a first semester required course in the Masters of Economics program. And the students love it, right? Because I have, I have academic experience. I have policy experience and now I have business experience. And these are sort of the three big domains for economics, right? Look at the time again. We have the three pieces and I'm so looking forward to the stories behind him, why you chose to bring them so starting by the newspaper bag from the Wisconsin State Journal. So you were a paper boy. And of course, the first thing that comes to my mind is these movies where you see a boy on the bike and throwing. I was on the bike. Our newspapers weren't wrapped with a rubber band. So I normally walked them up to the door, but I might walk halfway up to the door and then give it a toss, you know, and just landed on the on the porch. So Wisconsin is sort of in the Midwest of the US. It's cold in the winter every morning. I did this for either four or five years from like 11 age 11 to late 15 16 something like this. It was so cold that I was riding my bike. My eyes would tear and a little icicles. Oh my god, I'm on my eyelids. No, this wasn't every day, but you know, I understand, but like and deep winter when it was cold, the newspapers to be out. I mean, this is all before internet or I mean the very. This is 1980, 85 something like so people were relying on those news people like if Greg does not come by with the with the newspaper people to complain they would totally call the newspaper company and I would hear about it. So this was kind of your first job. Yeah, and you know, it really shaped who I became as a person in a few ways. First is it helped reinforce a love of reading. Right. So I would just read the newspaper all the time, but also not just reading it just gave me a window onto the world. Look, Madison, Wisconsin. Yeah, it's a university town. It's like a city of 200,000 people at that time. Right. It's a medium big US city, but it's in a pretty rural part of the country. And by reading the newspaper, I really got a sense of the broader world and especially about other places, other people. But also, by the way, it taught me independence because here I was 11 every morning going out being responsible. I made good money. I think I made $50 a month, which like when you're 12, that's a lot of money. And in the 80s, I mean $50 into 80s was a lot of money. I see in myself now young teenager, whatever delivering his newspapers. So with that money you were earning on the newspaper, did you also buy your first Rubik's Cube? Rubik's Cube, sorry. No, I think I like every other kid in America in Christmas, 1980 got one for Christmas. I'll come to the book in a minute. And the reason I chose it as my professional object, I mean, it was something I really enjoyed as a kid. Also, don't get me wrong. I was able to solve sort of two thirds of it before I had to go to the book. So I'm not like, I'm not some brilliant genius or anything, right. I'm like, you know, I'm good at math, but I'm not, you know, I'm not like one of these really outlier characters. I needed the book, but the reason I brought it is that it actually symbolizes quite well. Not only the work I did as an academic, but even the work I do at Zillendo, in the sense that I try to learn from data. And to do that well, there's many different pieces that have to work together. And these different pieces, I think of as like the sides of the cube. So first, you need data. Second, you need methods, you know, you need to be familiar. Here, like, is this prediction problem? Is this a causal inference problem? Whatever. You need familiarity with methods. Then you need to combine those two things and be able to do it in a computationally feasible way. Actually, the most important thing I should have said I should have started with is the question you're working on. Is it interesting? Is it important? Is it impactful? Right. So at some level, that's the cube. This is I have an important impactful question that I want to work on. How do I try to answer that question? But there's literally uncountable business problems that fit this idea. Okay, so we have data methods. We have computation. Then there's this art element to it, which is how do you combine the data and the methods and the computation in the right way in the simplest possible way to get at the question you're trying to answer. And that's really like solving a Rubik's cube because if ever you try to get one face, it always messes up the other side. Right. The other sides. So you're really trying to work things together as an academic, that's sort of where the story ends. But at Zalando, there's another even more important piece, which is how do you take whatever insights you've learned from data? And how do you incorporate it into our business decisions in an effective way? For me, solving the problem of helping people make decisions and as an academic policy decisions at Zalando business decisions is really about combining all these different elements in the most effective way. And when you do it right, then the cube is the right color on on every side. Now, your Zalando piece, which is the finger spinner. So for the audience, this is one of these things you hold between your thumb and forefinger, and it spins around a little bit like a top and it has three arms. But for me, it actually also works in terms of how I try to help Zalando succeed because we always want to put our customers first. So one of these arms is our customers. A second of the arms is us as Zalando, sort of our commercial performance. So if we put our customers first and we can provide them with assortment that meets their needs and that is inspirational to them that really connects with who they are as a person and how they want to express themselves as a person, then we're going to be successful as a company. But we also have another customer, and that's our partners. Many of our customers come to Zalando to access the leading brands, these top brands that we have on our platform, right? We have strong brand relationships with many of the best brands of the world. So if we meet our customers needs, we're going to be more attractive to our partners. And if we meet our partners needs, we're going to be more attractive to our customers and then you get a flywheel. Look at you again, come on. And the thing spins and we grow. I have one last question for you Greg. Before we wrap it up, I ask you to choose what do you think our listeners should dare to do more? What did you choose? I think people should think big. I don't know if it's whether I'm American or if it's just, you know, it has nothing to do with nationality, but I love the idea of possibility, right? In life, in career, in everything. And one should never feel that you can't do something. Everyone is different, but everyone has something that makes them special. And you just have to, as I say, find your spark. Everyone has to figure out what that spark is, right? And it could be totally different for different people. So maybe instead of think big, maybe it's find your spark. I really like it and it actually reminds me that on my situation, I really like interacting with people and talking to people and my job is not actually moderation is not doing this podcast, but actually having the opportunity to be doing this podcast is letting me actually use what sparks joy. I see your spark when you when you do these. Thank you. Thank you. Thank you so much for being my guest today, Greg. It was amazing. It was a really good conversation. Ginny, thank you for having me. This was really so much fun. Thank you all so much for listening. If you'd like to know more about careers at Zalando, go to jobs dot Zalando dot to eat. You'll find that link in the show notes or check out our Instagram page inside Zalando. Our next episode is coming in two weeks and I'll be talking to another guest from Zalando about life inside the fashion and tech retail industry. And of course, there are three pieces of me. [BLANK_AUDIO]