May Contain Lies: Stories, Stats, and Bias

Today on Episode #574 of the Teaching in Higher Ed Podcast, may contain lies, house stories, statistics, and studies exploit our biases and what we can do about it with Alex Edmonds. Produced by Innovate Learning, Maximizing Human Potential. Welcome to this episode of Teaching in Higher Ed. I'm Bonnie Stahovjak and this is the space where we explore the art and science of being more effective at facilitating learning. We also share ways to improve our productivity approaches so we can have more peace in our lives and be even more present for our students. I'm excited to be welcoming to the show Alex Edmonds. He's a professor of finance at London Business School and an expert in the use and misuse of data and evidence. He's given the TED Talk, "What to Trust in a Post-Truth World with 2 Million Views Spoken at the World Economic Forum in Davos" and testified in the UK Parliament. Alex served as managing editor of the Review of Finance, the leading academic finance journal in Europe. He has written for the Wall Street Journal Financial Times and Harvard Business Review and has been interviewed by Bloomberg, BBC, CNBC, CNN, ESPN, Fox ITV, NPR, Reuters, Sky News, and Sky Sports. He was previously a tenured professor at Wharton and investment banker at Morgan Stanley. Alex has a BA from Oxford and a PhD from MIT as a full bright scholar and is a fellow of the Academy of Social Sciences. Alex Edmonds, welcome to teaching in higher ed. Thanks so much, Bonnie, for having me on. Could we start with something about lies, since that's of course in the title of your book, would you tell us about a lie that you taught your students? I will do so, I was an assistant professor of finance at Wharton and so most of the course I would teach finance, but the final lecture I would spend the final half an hour giving a last lecture which is some life advice for students in particular, I teach the core at the start of their MBA, so this might be forward looking not just to be on Wharton but also to the rest of their MBA. Can I talk to them about the famous 10,000 hours rule by Malcolm Gladwell, which is the idea that you can succeed in anything if you just devote 10,000 hours of practice to this? I said, well, you should push yourself outside your comfort zone at Wharton, it may be that you're new to public speaking, but if you were to join the Toast Masters public speaking club and really work at this, you could become a master public speaker, really there are no limits to what you can do. That would sound inspiring and when I would say this, the students would nod their heads, some of them had heard of the 10,000 hours rule and so what I said was consistent with what people think, but when I later had to give a talk on that rule for a more general audience and there it was not going to occupy just half an hour, but the entire lecture, I looked at the science behind the rule in much more detail and I found that the science was extremely weak. Number one is the original study looked at just violin playing, had nothing to do with public speaking or anything else that you might want to develop, but Malcolm Gladwell extrapolated from that one single study to claim a universal rule. And also the study never measured the violinist performance, it never measured 10,000 hours, so this rule which apparently 10,000 hours leased success was based on a study which didn't either measure success and it didn't even quote 10,000 hours. Well, let's dig a bit more into the word lie. How do humans typically limit or otherwise misinterpret the meaning of what it means to lie? So we think a lie is basically the opposite of truth, so something is a lie if you can disprove it factually. For example, if somebody was to claim that Barack Obama is a not a natural born US citizen, that can be exposed as a lie if you produce his birth certificate. And that certainly is a form of lie, but that's something that you knew of probably 10 years ago at least. So in 2016 post truth was the Oxford History's word of the year, we know to check the facts. But what I focused on in my book is a more subtle form of a lie where something could be 100% accurate, but the inferences that we draw from them might be misleading. For example, you could say, well, as his story of how I got my kid into Harvard and you might say, well, I did all of these things, I made them study the violin after school, I made sure that they only socialize with friends that I approved and so on and my kid got into Harvard. Now, it could be absolutely true that your kid got into Harvard and it could be absolutely true that you made your kid do all of those things, but the idea that one caused the other is not correct. And so why is that so important to recognize is that we think, well, yes, we have people telling lies, but we can easily prosecute them, we can easily stop this because if somebody spreads misinformation, then we can do something about that. But these other forms of lies are really subtle. Nobody would be able to prosecute you or for writing an article how I got my kid into Harvard because you did do those things and your kid got into Harvard, but the idea that one caused the other is not correct and so what I want to do in this book is two number one ways of awareness of this other form of misinformation, which is actually, I think, even more dangerous because it's harder to spot. And number two, tell the reader, well, what you can do about it. You started our conversation with a lie you've told your students, I'd like to now come fess up to a lie that I've told mine and it involves marshmallows. Please tell us about the marshmallow research and why would I told my students all those years ago just might have been a lie. Absolutely. This is one that I don't think I've taught to students myself, but I have believed myself, so I'm just as guilty. So the famous Stanford Marshmallow study, what it did is it took some kids, young kids and then it gave them a marshmallow and they said, you can either eat this marshmallow now or you can wait 15 minutes and if you wait 15 minutes, you'll give a second marshmallow. And so what it did is it took those kids and it tracked them and saw how they did later in life and the kids who were willing to wait for the second marshmallow outperformed in many areas in terms of graduating college, in terms of their salary. I think they also looked at relationship outcomes so they were happily married and so on. And so what this highlights is the power of having, of being able to control your impulses of not being short termists and myopic. And just like the 10,000 hours rule, that is something which plays into what we think to be true. And so this is why lies can spread even among intelligent people like college professors is we think these are true. We tell our kids practice makes perfect. So that's why we would believe in the 10,000 hours rule. And we tell everybody about the importance of not being impulsive, the danger of say, smoking or eating unhealthy food, the importance of doing things with long term benefits, like practising music or learning your French verb endings. And so yeah, we think that to be true. But what is the lie there? Is it's non-actually that being non-impulsive causes you to be successful later, but there's other factors that play. So who are the ones who are willing to wait for the second marshmallow? Those are the ones from absolute family backgrounds. Because if you're an affluent family background, there's lots of food to go around and you don't need to eat now because you're going to have still that food later. But in a less affluent family background, you eat what's there on the table because if not somebody else might have eaten that. So let's say you're in a family with five kids and not much food there, you are just wired to eating what's in front of you otherwise it disappears. And so it's not necessarily this patient switches causing the future benefits, but actually the decision to wait for the marshmallow is in an indication of your family affluence and that family affluence is behind the future benefits. And so why does that matter? Because interestingly, we've taught the findings or the conclusions of the marshmallow study in various contexts, even on Sesame Street. There's an episode where the cookie monster tries to highlight the importance of not eating cookies immediately. But actually sometimes a short term focus can be helpful. So cramming for an exam can sometimes be helpful. It can be useful to if I want to run a marathon in a year's time, I could set myself short term targets. I want to run maybe 20 miles by the end of the week. So the idea that always waiting for the future, that's not always going to be the best strategy. And I'm going to say, well, let me never get married because I mean, I'm going to wait, I'm going to find the perfect person or let me not take a job because I'm going to find the perfect job, which is going to pay me a massive salary, but sometimes you actually, it is actually prudent for you to take what is available. Mm. Both of these illustrations are something that I think many of us would want to believe. And I think there would be those who might be listening who say, nope, that's not even an appealing premise for me. I'm not, you know, I'm not all interested. I think it's important and you of course stress how important it is to recognize these are forms of confirmation bias, but also tell us about confirmation bias as being an inherent part of being human. Yeah. So you might think, well, why do people make these mistakes when they interpret information? It's not that they're bad people. It's that they're people, they're humans. And if we're a person, we have biases and biases need not be strong biases like about gun control or immigration or abortion, biases could just be small hunches. Yeah, I do believe that practice makes perfect. And that's not a bad thing to believe, or I do believe that we should delay gratification. And again, that's not a bad thing to believe. That's a product of how I was brought up. But if we have these biases, even if they are quite small, this leads to misinterpretations of information in two ways. First is what I call naive acceptance. If you see something that is consistent with your worldview, you lap it up uncritically. So I, despite being a university professor, when I read about that 10,000 hours rule in the book Outliers, I didn't bother initially to read the underlying research and to find that it was actually an violent playing and it never actually measured success. But notice that if the outliers had said something I didn't want to be true, I wouldn't believe it. I would have read the underlying paper and I would have read it trying to begin apart. And so that's the second mistake that we might make is blinked skepticism. That is the opposite of naive acceptance. If you see something that contradicts your worldview, you would either not read it to begin with or you would read it but not with an open mind, you will at every juncture try to tear it down. And so what I'm trying to highlight is the importance of being discerned. And we want to be somewhere in the middle. We don't want to accept everything, but we don't want to be a conspiracy theorist and disbelieve everything. We want to have healthy skepticism, but we want to have the same healthy skepticism to something that we do like as something that we don't rather than deploying our discernment only selectively. What kinds of questions or what kinds of symptoms might we be looking for within ourselves to know that we're susceptible to this form of bias? I think it's just to think about how something makes you feel. So if you are willing, if you're just eager to shout something from the rooftops or chomping at the bit to reshared on social media, then it is probably something that you might want to just sit on your hands first and study, is this actually true? If something makes your blood boil, well, actually you might be better off reading it. And I know this is far easier said than done. But in 2016, when there was the Brexit referendum, I was a very strong remainder. I thought it's nuts trying to leave the European Union. Anybody who believes this was xenophobic. But I went to Brexit talks and I realised, well, actually, there is logic behind the other side. Now, even though I did not agree with the conclusion that the speaker drew, I could at least see the logic that led to his conclusion. Similarly recently, I think Trump's tariffs are complete madness. I cannot see any logical justification in them, but actually the last podcast that I listen to had somebody who said, well, actually tariffs can be beneficial in certain circumstances. Now, he said that the way that Trump has implemented them is incorrect, but he certainly opened to the idea that correctly implemented there is a case for tariffs. This is a Harvard professor at the Kennedy School very learned and so I learnt something by listening to him, even though I thought, well, let me not listen to somebody who's pro-tariffs because they cannot be another side. Very helpful. There is a researcher here in the United States might co-field who researches among other things, fact checking and misinformation. From his work, I so often draw, coupling with what you described, Alex, asking myself the question, what's my aim in this in the first place? So if I go onto social media and I have this visceral reaction to something that I see, whether it confirms my biases like you talked about because I was so much want to agree with it and pass it on or quite the opposite. I so vehemently disagree. I should be taking the time to go and check the 10,000 hour rule or the marshmallow, but perhaps it doesn't align with my goals. Perhaps because there's only so much time in our days, so perhaps I need to be patient and disciplined about those things which I would like to share to my students or otherwise. And then of course spend the time or even invite students to spend the time to thinking critically together about how to break down some of the arguments being made and to discuss those things. The other thing that was resonating with me so much as you described this is just the idea that you talked about not wanting us to be in either of those two extremes, but yes, of course we're going to hold opinions. But we might be in a good place if we hold a really strong opinion, but we could understand some of the logic and the other person who we so disagree with in their perspectives too. Yeah, so and you're from thinking listening to this other viewpoint does not align with our goals, but actually might align with your goals because it is much better for you to hear the other side than this to be brought to you by a student in a classroom and then you are you're on the spot you're not able to respond on your feet. So the reason why I tried to force myself to read viewpoints I disagree with is not so much being open minded as being selfish. I want to know the weaknesses in my arguments so that if I'm to give a lecture to students or a talk at a conference and get some Q&A, I will not be blindsided. Similarly when I was writing this book, I had about eight agents suffer me representation and the one that I chose to go with was the one who was most critical about my proposal because I thought he has the best potential to improve it. There were some agencies that said, oh, this is a great proposal. You don't need to do anything. I'm happily going to send it to publishers. This guy made me work hard, but I think that was much better because it then meant it came out as a stronger proposal. So when I see a different viewpoint, even if I think 90% of what they say is wrong or says not, maybe 10% of what they say might actually be right and I come away learning more than I knew previously. Sometimes the biggest praise we give a person is, oh, we would think that you said, but if so, well, why did you even bother listening to it because you didn't learn anything new from it? And also to sharpen your arguments too, right? So you're hearing from another perspective and your arguments, your critical thinking becomes even sharper as well. Yeah, and maybe this is an analogy I'm pushing too much, but if you are to wait train, why you wait train is is you train against resistance? So if you are able to conquer resistance, then that makes you stronger. And actually, how does wait training help you? Wait train doesn't actually grow your muscles. It tears them and it causes tears in the muscles. And then when those tears are healed, then the muscle grows back stronger. And so I think the idea of listening to different viewpoints and challenging arguments, then means when it comes game time and game time could be giving a lecture to students, it could be giving a graduation speech or speaking of a larger conference, then you are going to have your best possible stance. You tell us the importance of examining the evidence for a statement. Can you talk us through a little bit of what that looks like and maybe even specifically how we might help students be able to do this more effectively as well as ourselves? Yeah, so I've given one example from the book which is the 10,000 hours rule. So let me give you one example from outside the book. And this is on something which might be even more important than developing habits, which is the opioid epidemic, which has been really serious. It's killed 650,000 people in the US, 2 million people around the world. So you might think well, how can opioids be prescribed so liberally when they are highly addictive? Well, it's because many people thought they were not addictive at all. There was a study published in the New England Journal of Medicine, so highly respected, where the title was "Addiction Rare in Patients Treated with Narcotics". And we love to quote statements all the time. Right, we say 10,000 hours is the secret to success. "Addiction is rare in Patients Treated with Narcotics", and we accept them uncritically. Yeah, it was published in Outliers by Malcolm Gladwell, a best-selling book. It was published in the New England Journal of Medicine. But you ask, well, how do we check statements and why do we check? Well, first, let's just pull up the article, "Addiction Rare in Patients Treated with Narcotics". If you bother to do this, you realise that it's actually not a scientific study at all. It is a letter to the editor. It takes up less than one six. of a single page of the new England Journal of Medicine. So it hasn't been peer reviewed or checked by anybody, it's just a five sentence letter. Yet this letter has been cited 1700 times as evidence that opioids don't lead to addiction. People didn't even look at art, they thought. The headline, "New England Journal of Medicine, they don't even bother to look at it." And let's say, everything in it was completely true. Well, what they did, as they looked at people who'd had at least one dose of narcotics. Now, one dose of narcotics could actually be okay, but maybe four or five opioids might make you addicted. And they also looked at hospitalised patients, so maybe if you're in a hospital and you're given this as a controlled substance, then you're not going to get addicted, but if you're given this as an outpatient, then it could be quite deadly. So this is what's really important to go to the underlying source, rather than just to rely on the statement, we need to understand the context in which that statement was gathered, or hospitalised patients versus outpatients, just one opioid versus multiple doses. And then this is the same. Let me just then circle back to the 10,000 hours rule, is that only once I read the underlying research, so I find this was on violin students when he claimed that the rule was about everything. Now the listener might have been quite perturbed by my early comment that this study did not even measure success. Well, how can bad will I have quoted this? So what this measured was what they thought was success, but it was actually perceived success. And so what I mean by this, what they did is they took students who were in the Berlin Academy of Music, and they had them divided into groups according to their teachers. There were some people who the teachers thought were good and would make it to the top orchestras. Then there were the middling students who would make it to a minor orchestra, and there were the lower quality students who would perhaps make it to become music teachers themselves. And so there was no objective measure of success here. It could well be that if the student is practising all the time after class, then you might say, "Well, this person is going to be successful, because unlike, say, running 100-metre dash, there's no objective measure here, so it's perceived success." Also, in terms of practice, you might think, "Well, as bizarre how can this have not measured 10,000 hours?" What they did is they asked the students who were now around 20 years old to remember how much they had practised from age 5. Age 5, that's 15 years ago. I've not even remember what I did last last week or even yesterday. And so again, it could be that if you know that you're good right now, you will say, "Yeah, I did practise a lot." Whereas if you're not good right now, you're not going to claim I practised all of those years because that would admit to yourself that all that practise was futile. So if you're to run a marathon and not get a good time, you typically say to your friends, "Hey, I just didn't have time to train." And so again, this study which has said to be gospel, neither practise was measured objectively. It was your recollection of something which happened 15 years ago, nor is success measured objectively. And without looking at that, then we might actually take this at a pace of mine. I could talk to you all day, but I know we're going to run out of time. We also live in quite different times. So I want to be respectful and finished by just asking you to reflect with us about the transition you'd like us to make between thinking about creating individual critical thinking and what it looks like to create smart thinking organisations. And I think what's really important here is to create a culture where people are able to disagree with each other and express different viewpoints. And so this is why I think academic freedom is so important and why the challenges to academic freedom that seem to be happening, I take it really seriously, I think this is a serious concern. So there may be viewpoints that people may not like, but as long as they are expressed in a respectful way and backed up to the extent possible with either evidence or logic, then we should be willing to hear them. And indeed some of the most groundbreaking ideas were seen as a revolution at the time, be this the world is as round, rather than flat, or the sun is the centre of the universe, not the earth. And in higher education institutions, we often don't have political diversity. So the fact is, and one might lament this fact, but it is still a fact that 52% of Americans voted for Trump, yet there's very few college professors who might identify as being right-wing. And it is important to understand the concerns of other people. So I'm somebody who thinks that baritur trade is completely nuts, but I could quote to them, David Riccardo's law of comparative advantage all day and night, but that's not going to hold sway if I'm not hearing the concerns of say, the 55-year-old manufacturing worker who might be out of a job because of Chinese imports. So I do think it's important to hear the other sides. And even if you end up disagreeing with them, you might at least understand where they're coming from. And I think this makes us a more educated person, than just listen to people whose views that we feel comfortable with. And any, you said you were maybe were overdoing the muscle building analogy, so I'm going to just let you do it one more time. I mean, are there muscles we should be building up as organizations to help cultivate what you're describing? Yes, it's to be willing to take knocks. Just not just like in sport is that you want to, when you take a knock, you stand up. And I think, why is it that sometimes we are willing, when we want to cancel or not even listen to different viewpoints, is that we might just no deep down ourselves. We don't actually have a strong argument ourselves. So we don't want to hear the other side, because we no deep down that our instinct is a little bit fragile. And actually, the stronger the stance is, the more willing you are to hear a question. So after every talk again, I always happy to have Q&A. And for every podcast like this one here, I don't ask for the questions in advance. I'm happy for you to ask me anything. And I'm happy to respond to anything, whereas there's others where, oh, I'm only going to participate in the interview if you give me the questions in advance and you allow me to edit it afterwards. You're reminding me of the Harvard researchers who have written a number of books about negotiations and influence who talk about separating the intent from the position. Because that's, I mean, talk about introducing a lot of bias when I'm making up stories in my head that may or may not be your intent as you're criticizing my work. That's just going to make it that much more messy versus being. And that, to me, at least, has created in how I feel about criticisms of my work. If I've worked with many students where their writing feels like them, because of course it does, too right, is to express, when's real personal views. So it makes sense why we as humans would feel hurt if someone criticized it, but to teach ourselves to separate that writing isn't me. They're helping me make that writing better or that research proposal better or whatever it is. I'm not going to apply intent that hasn't been communicated. In fact, I try to train my brain. I'm not saying I'm perfect at it, but to actually have quite the opposite feeling. What if I just assumed logical or not that you criticize in it? You might always have a positive intent in doing that. And again, it's just a way to play tricks with your mind. I'm not going to write a research study on it, but I need to sometimes train my brain to be able to receive that in more nourishing ways to help me build those muscles. This is an excellent point. I think it's really important because we often conflate these things. I often use the phrase, the terms goal and the approach. So one topic I do research on is diversity, equity and inclusion. So there are many studies out there by the likes of McKinsey and Blackrock claiming that more gender or ethnic diversity magically improves financial performance. They've been widely circulated and shared because people want them to be true. But actually, if you look at the data, it's got really serious, basic errors. And I'm an ethnic minority myself. I would love to believe this, but I have pointed out there are some flaws in these studies. And you might think, oh, then might I be cancelled for being racist or sexist and opposing diversity? No, because what I'm trying to be clear about is my intent here is absolutely to build and promote diverse and inclusive organizations. But the approach must be more than just looking at gender and ethnicity. So these studies only look at gender and ethnicity. That gives the impression that if you're a white male, you can never add to the diversity of an organization. Why? Because even if you've got a different social background, even if your background is humanities and everybody else is engineering. So by me highlighting the flaws in these existing studies, this is not saying that DEI is bad. It's saying DEI could absolutely really good. But actually, we need to look just beyond gender and ethnicity in a box-taking way, but consider the whole person. And so I share the goal of everybody to build diverse and inclusive organizations. I share the intent of everybody. But my approach to doing this is to actually have a more nuanced view and a more holistic view as to what are the diversity dimensions we want we need to look at. This is the time in the show where we each get to share our recommendations and boy, mine comes from a very polarizing subject and that is artificial intelligence. In a past episode I had Chris Ostro on to tell us about the literature involving AI detection tools and in that conversation we had probably a five minute discussion about how he handles it when he identifies that students may have used artificial intelligence in ways that don't support the learning goals and I heard from so many of you he jokes with me that you know I'm totally mischaracterizing but we'll be playful with each other and he'll say I made him famous please don't think I'm actually believing that but we just tease each other that way because many of you have also reached out to him. So Chris has recorded a video part which was part of an invitation for him to share with the community but it's available called taking a mosaic approach to AI in the writing classroom and in this video Chris expands on many of the things that you got curious about and wrote to me or wrote to him and I just think it would be a wonderful thing for you to watch if you listened to that episode with Chris and want to hear more practical ways of both minimizing the likelihood that someone may in a way that's less helpful or less relevant for a given learning goal use AI or use it in helpful ways and then how to address it when there's misalignment between the use and the intent for any type of an assignment. It's a really wonderful video and I'm glad to be connected with Chris and that we continue to have this communication since he's been on. So Alex I'm going to pass it over to you for whatever you'd like to recommend. I'd be delighted to recommend two things. So I mentioned this podcast episode I listened to earlier on how tariffs can in certain circumstances be beneficial. This is part of an excellent educational podcast called All Else Equal by two Stanford Finance Professors Jonathan Burke and Jules Van Binsberg and why I really like this is it takes really topical issues but presents a slightly contrary spin which is not to be deliberately contrary but just to give a different way of thinking about an important topic. So in terms of education they were asking well is the idea of legacy emissions always bad and to me I think legacy measure terrible right shouldn't the only thing that matters be your academic potential and your ability to contribute to the community and they had some reason arguments and again it might not be that you agree with the overall conclusion but I always come away from this learning more about this. The second I think I'm going to recommend is going to be completely unrelated to everything that we've discussed which is the favourite book that I read last year. It's a novel. It's a novel called A Little Life which is an extremely like emotionally really complex novel it covers some really difficult subject matter which is so and so intense that it completely blew me away. So I think it was a finalist, a shortlisted for the Pulitzer Prize. I myself heard about it because one of my friends posted his best recommendations of the prior year so given I heard about it through a recommendation I'd like to pay it forward and recommend to others and indeed when I did my final LinkedIn post of last year on the best things that I'd read or seen or watched I put this on there there were loans of comments saying oh I've read this book as well it's extremely powerful. Oh my gosh and now you're making me want to go check out that LinkedIn post too so I'll find it and put a link to that in the show notes as well because many of our listeners would probably really enjoy browsing it just like so many of your followers. Before we close our time I know people have gotten curious from our conversation today Alex could you tell them just a little bit about may contain lies how stories statistics and studies exploit our biases and what we can do about it so they can get even more curious and go get their hands on it. Absolutely so this book is about misinformation but with a twist so we often think that misinformation is just about checking the facts and if the facts are accurate then we're home and dry but what I want to highlight is there's more subtle forms of misinformation even if something is 100% accurate it can be misleading and so what I want to do in a simple way and to make it accessible as possible to highlight all of the different forms of misinformation so you might think okay this is a big bad statistics I find statistics either boring or technical but I try to lay it out in as engaging way as possible without dumbing it down so to have all the substance without the jargon and so I had various reviews I'm really grateful to have had mainstream reviews not just by university press it says likes of the times or the Wall Street Journal or the Guardian and one of my favourite ones is it's a really timely book and despite the nerdy statistical theories it is funny and it's about what I wanted to convey in this book is I want to make it accessible and of interest so even if a listener is a professor of a field completely different from statistics or economics hopefully he or she will find it interesting. Well it's fun I agree that it is funny and that's such a perfect encapsulation of it and despite having watched your videos before so I already knew kind of what I would expect getting to talk to you it's been such a delight to get to have a conversation with you today I'm grateful for you investing your time in the teaching and higher ed audience and best wishes on getting this hand getting this book into many hands and into many minds out there thank you again for your time thanks so much Monique for inviting me on thanks once again to Alex Edmonds for joining me for today's episode of teaching in higher ed today's episode was produced by me Bonnie Stahovjak it was edited by the ever talented Andrew Kroger podcast production support was provided by the amazing Sierra Priest if you've been listening for a while and have yet to sign up for the weekly update now is your moment head over to teaching in higher ed.com/subscribe you'll receive the most recent episodes show notes as well as some other resources up of and beyond that thanks so much for listening and I'll see you next time on teaching in higher ed. [Music]