Episode 15: Valuing the Literature Review

[music] Welcome to Doc's Student 101, a podcast where we discuss the necessary skills and unique challenges of doctoral education. With your hosts, Dr. Linnea Raidamaker, Dr. Peter Williams, and Dr. Scott Self. Hey everybody. Good morning. Good morning. I'm so glad to talk with you again. We're joined today by Larry Mocky. Larry is joining us to talk about literature reviews. So Peter, I thought I'd let you introduce Larry and kind of help us think about our topic for today when we talk about literature reviews. Yeah, I'm happy to welcome Larry, a friend and colleague about 11 years ago, 11, 12 years ago. We started teaching together in the EDD University of LeVern and he's joining us to help us sort through some of the motivations and reasons why we do liter reviews because he's been thinking about this for a long time and helping people do it and wrote one of the more widely used books on literature reviews. So Larry, welcome. Glad you could be with us. Good morning Peter. Good morning, Linnea. Good morning, Scott. It is good to be with you all. Thank you. We appreciate you. So one of the things that we've noticed, I think all of us in our teaching in various doctoral programs is that students often get to the proposal stage, perspective, proposal or wherever they have to write the first big version of chapter 2, their lit review and they stall. They stall for a number of reasons and sometimes it's because I think this because I've had several students tell me or imply that they don't really see a reason for a lit review outside of the very requirement in the proposal and later in the dissertation. So I think there may be some misunderstanding and I want to talk about that for a bit. The first question I guess I want to start with is like, why don't we even bother to do a lit review? I had a professor tell me when I was a master student, he said, basically, there's nothing new under the sun. There's not a problem or an issue that you're going to bring forward that we probably haven't already thought about in the case of a doctoral dissertation that is really, as my old friend Bill Beerle used to say, that's the book, the dissertation. What the book is about is the research you did and in the case of a doctoral program, the original research you did to extend the knowledge of the field, well, to extend the knowledge of the field, you've got to know what the knowledge of the field is. Consequently, if you have a potential problem or a particular research question that's tentative, you've got to find out what we already know about it. So you have to do the science of that, go to where we know about that, that happens to be in a library because books record the history of what we already know. I really like where you're coming from on this. There's a kind of virtue here of, I don't know, if you want to call it epistemic humility or some form of humility, but recognizing that you're not the first person to look at this phenomenon, you're not the first person to explore this cave, you're exploring it in a new way, hopefully, but that kind of humility, that virtue is probably dispositionally a good place to start. I have a failed book on critical thinking that the publisher said, basically, there's no real market for critical thinking. People won't buy the book because basically they already know how to think critically. But in that research, what I found was that in order to be able to think critically, you literally have to trigger your emotions and flip over to the rational side. So the idea of disposition is really a mindset in that research. What I found was, and you'll find it in most literature review materials, that there's a certain mindset that the student has to come to in order to do a good literature review. And those dispositions are being inquisitive, skeptical, independent, being honest with themselves, persistent patient, having a sense of collegiality, like Linda Nia said, you've got to be able to express yourself with others. And you've got to use reasoned and logical thinking. Those are things that you literally have to trigger in yourself to be able to do a good lit review. That's not. To think critically too. It's the same thing in decision. So that kind of makes me think of how students often encounter a lit review, and I've used the phrase before, like they see it and I will tell them, well, typically a lit review for a dissertation. It's anywhere between 30 and 60 or 80 pages long, just to kind of give them a wide range. And they get wide eyed and it's like a big bear is running at them. And so I think that there are a lot of parallels in professional life where you're faced with big, bad, hairy situations that you have no idea how to deal with. So even that encountering the big project or the big bear of a lit review, it has parallels outside of this little academic process we're going through of a capstone dissertation. I really appreciate what you said, Peter, because I just recently bought a house. I mean, recently, like yesterday, but my, my realtor, yeah, my realtor has a PhD. My realtor has a PhD and so, and I told him how much I appreciated him working with me because I felt like he understood that I don't know everything. And the reason I bring this up is a lot of folks, when they find out you have a doctorate, they'll say something like, oh, you know so much or you must know a lot of things or they believe that you have in your head a whole bunch of facts. And that's what being, having a doctorate means. And I always want to tell people, no, that just means I know how to figure stuff out. It doesn't mean I know what, I don't know everything. I know how to find things. I know how to, whether it's by my own observation or whether it's by listening to other people's conversations, I know how to find information. I don't know things and, and the reason I was happy with my realtor is because he knew I didn't know anything, which I needed somebody to help. But, but I do think that, you know, that exercise of going through the literature review is a great, a great exercise in experiencing that process of discovering what other people know about something, which is really what expertise is about. The thing that you, I agree with you, Scott, the thing that the literature too does is it actually does the same parallel steps as research would do, as we're attempting to do research itself. It's still applying the same steps, the differences, rather than looking at a test tube or actually going into the field to be able to observe and analyze something. That basically you're going into a library and you're doing the same thing. You're going to go out and grab data and you're going to be able to organize it and you're going to be able to develop findings from it and make some conclusions based upon some logic. It is the same process, the differences we have to stuff in the book versus that that we're going out in the field to be able to change. It's the same thing that you're suggesting, you know, as you said it, we tell students that they're going to engage in original research. If you take a look at the word, please take a look again. That's interesting. No, it's the fact that you know how to look again. I like that. I like that a lot. Talk a little bit about some of the other contexts in which this skill is applicable beyond doing. Hopefully, if you do a dissertation, you only do it once. I mean, I can tell you how to do it wrong and do it more than once, but, you know, I did three literature reviews for three different dissertation proposals, but if you think about the process of critical thinking, the process of critical listening and the process of critical organizing of conversations that you do in the literature review, what are some ways in which that is antecedent of other forms of critical thinking in other contexts? If, in fact, you are engaged in a lit review that is applying critical thinking, then those processes are fundamental to all of the applications that critical thinking would have. As we talked about it before, a lit review is a critical or rather an applied process of critical thinking decision making and problem solving used the same skills at and the same process, they're just different applications of the same thing, critical thinking. And so the idea that you have the ability to observe the idea that you have the ability to organize data, to be able to determine its reliability, to be able to organize in such a way that you can make findings about something and you can analyze in a logical fashion. Those things you should learn in the literature review are directly applicable to decision-making and problem solving in the everyday world. Well, you know, I like that you've said before, it's entering and engaging in the conversation about the topic, about the problem that, like Larry said, there's nothing that's been not studied before, whether it's from a different perspective or aspect or a certain construct of something, everything has been studied before in some way. And so when you are conducting a literature review, when you are gathering that information and you're analyzing it and you're putting it together in a coherent way so that the reader can understand what's been out there, you're engaging in that conversation and you're helping explaining that conversation to the reader. And it's really important that you not only have literature that supports what you want to do, but that you also have the conversation from people who disagree with you, because that's the key to argument, is that you are giving those arguments, you're explaining what they said, but you're showing where it doesn't apply to what you want to do. So I think that gets us back to a particular type of disposition, right? This ability to see oneself and to see one's preferences, proclivities, tendencies, favorite ideas or solutions and hold them lightly, recognize what they are, and then look at alternatives. And that's it. I think it's sort of a professional, and you could say a virtue, but a professional disposition of, yes, this is what I think, but I'm going to listen to you because you have a totally different perspective and it could be a productive listening session. Peter, that's so important to what we teach about leadership and what we teach about being the kind of leader who realizes they don't know everything and that they need to engage in these other conversations and that other knowledge that people have. Larry, I would love to hear you talk about that. Well, we have certain proclivities about leadership and the ones that I studied for a long time, both as it as it school administrative field and then later on as university. The idea of collaboration, the idea of being able to understand that you don't know everything is critical. Not just from the vantage point of being able to get the whole elephant in the room because we all have different perspectives looking at the same thing. But more than that, to be able to engage others to see the same thing so you can move forward. As I work on talking about leadership, a little Peter Drucker had the definition of the universal definition of a leadership that's accepted around the Western world at this point is the leader as a person who has followers, which I thought to myself, well, if you have followers than you probably are going to go in a certain direction. Well, if you don't have a sense of us to say it, an interaction where people can come to understand each other, they can understand you, you understand them, how can they follow or why would they? And so the idea of, for instance, in change theory, the research is absolutely replete with the idea that in order to move a vision forward, you need to know how to do it in order to do that. You have to have a critical mass of the individuals who are working with and without the organization to make it happen. You have to do that, Linda. You have to bring people together and you're going to have to have that conversation. And it better be a good one because you have to basically convince them to do something that they didn't want to do in the first place. That was an awful lot. I was like, seven years of pent up, pent up, I haven't said this in a while. No, that was perfect, Larry. That was perfect. I think, you know, a lot of my, when I'm teaching research classes, we tie, I tie both of the research classes I teach evaluation and qualitative and extra research. I tie them into change leadership because you, you can't just tell people you're going to do this, which seems to be a prevalent thing right now. I won't get political, but it seems to be a prevalent thing. You have to give them a reason. This goes right into adult learning theory and you could just tie in a lot of things that they have to understand a reason why. And so I think the literature review really is the first demonstration of that that we have in a doc program and it's really critical. I want to play off of what you just said because the literature review is really taking from the idea of making a convincing argument, the one that goes way back to Aristotle and the organite. I mean, basically, you're making that argument to convince somebody to agree with you. That's what you're doing. And that's why the argument's so very important and it's a critical argument that one has to make a leadership as well. And I find it so very interesting that people haven't picked up on that. There's an interesting aspect of that that I think happens as the doc student moves through the lit review and the leader goes through the listening processes or in other professions that's similar and that's a, I want to call it right now, a personal transformation, kind of a mindset shift of here is my argument that the doc student might say, this is where I want to get to. I have this little pet study I want to do and now I'm going to go write a lit review to show that I need to do this. So it's a foregone conclusion that they've already arrived at and then they go find evidence to support their foregone conclusion. A lot of leaders lead like that too, unfortunately, but I think a lit review well engaged is like in an organizational listening process well engaged. It's transformative to those who do the lit review on one hand or who are listening on the other hand. It shifts my mindset, it helps me see that a broader picture, more depth, whereas I've previously saw surface level things. Now I'm understanding the mechanics of what's going on that produced, for example, the felt need that brought me to this particular juncture. So what is that, and that relates to disposition at mindset shift, but I see it as a possibly when it's well engaged, the lit review has a transformative journey. Yeah, I would agree with you, Peter, that basically a lit review, as Scott said, you're going out to find out what other people know about it. It has to, in fact, in many cases fundamentally change what you think about something because basically you're finding out what we know, not what you just know. And many times you'll find out, in some cases, that we already know how to solve the problem you are defining, which can be disconcerting to many students. I had a student do that, and I said, what am I going to do now? They already solved the problem. What am I going to do? I said, well, find out what they didn't solve and then go from there. But I think you're hitting on a very important point, which is that this is not only, you start with a question, you start investigating, you end up with a different question, you investigate some more. There's an iteration that's involved here. You don't devise a well-devised research question, go to the literature review and end up with the same research question. Rarely do you do that. Usually what you end up discovering is that your questions are refined, and those refined questions cause you to look at new literature, and that new literature causes you to refine your research question. It's a cyclical process. Hey, Larry, let me ask you a question. One of the things that I see students struggling with a lot, I think I mentioned this in our last episode, is kind of this disconnect between reporting on what other people say, particularly what other people have said in relationship to my preconceived notion of the way things are. But that's a very different position than reporting on the conversation, and the synthesis that goes on between just saying, "Brown found this, Smith found this, Jones found this, and start to think about synthesizing that into a description of the scholarly conversation as it surrounds a point of inquiry." Can you talk a little bit about how we can think about breaking that process from just reporting on what other people say into that moment of synthesis, how that works? I'll try. I think there's a natural inclination on the part of most students to say, "This person said this, this person said this, this person said this," because they're used to doing book reports. The problem with the book report is we shouldn't do book reports. We should probably ask them what that they learn from the book report and how does it relate to other things. Fundamentally, it's Linia said earlier, the fundamental difference between a book report, this is what this person said and this person said, is the argumentation it went to what is it mean? It's fundamental, what's happening when you do the book report lip review is you're not doing any analysis. The skill set is the analytical set and the analytical set is, "Look, given what I have, how does this stuff organize and fit together and what is it point to?" I think you've nailed down one of the major problems that I saw all of a way through my career with folks who were trying to move things along, they failed at not producing the finding because they produced the findings, this person said this, but what they weren't able to do is the analytical aspect of this, how do these things fit together? It's that organizational aspect that starts to put things together and makes the puzzle start to paint the picture, unless you can paint the picture of what that sells by saying, you know, this person, this person, and this person said this, and in essence, this is what they meant, while these people said this, and this is what they meant, now we have two issues, so there you go, you got your synthesis now, and your synthesis is now starting to compare itself as you're starting to look at the various viewpoints, now I'm going into a lip review where I can make an argument that advocates for one direction or another. I don't know if that explains it, but that's very nice. What's happened? That's good Larry. Let's say you're a school leader, I'm always tracking this back, the parallel of the skill set that you develop besides the product of the lip review, the skill set you develop along the way, the process you engage in, the critical thinking process and argumentation, and then the communication of that, in doing the chapter two, and then finally the chapter two, I'm looking at the parallel of that process to let's say a school leader, it could be in any industry, but we'll say a school leader for this case, and you're told there's a pandemic, you have to stop doing things the way you have been doing and figure out what to do next for the next three months. Well, the parallel is directly related to the lip review because basically the critical thinking process is the basis for the rational decision making models that you may be teaching in university programs are pretty much the ones that would be taught in rhetoric or some other areas in business. Basically, rational decision making breaks down either five, six, or seven positions, but you're identifying problems, you're choosing a decision style, you develop alternatives, you choose the best solution, you implement the solution, and then you evaluate your decision, which goes back to what Leninia, Leninia said about action research, or action learning, which are the same steps, so they're directly related. We were talking before we started recording, I just recently had to do some work for my department on kind of what an interdisciplinary learning outcome would be for a capstone project, and what I discovered as I got started was I guess I'm going to have to write a literature review. Literature reviews are a part of, even if it's not a formal product, that process is an important part of decision making, regardless of whether you've learned to write a chapter two for a dissertation or not, that's something that most of us, if we're thinking carefully and critically about a problem, or about a point of inquiry, that we are kind of engaging in that, the literature review is really just formalized critical thinking, isn't it? Yeah, we had a similar situation now that I think about it. I was on a committee work group really to look at the online graduate writing center and decide, after three years, what do we do now, so to evaluate what we've done to some degree, but also to decide what to do next, how to change, how to improve, and we did a lit review, I mean it was a quick lit review, and then we interviewed key informants from various other university writing centers, and that's how we conducted it. It was like a miniature research project, right? Because that's what we were trying to figure out, we said we entered that task thinking, we're not the first ones to do this, let's find out what's been written down about it, and we're not the first ones to be doing this, let's talk to people who've been doing it longer, and so that's what we did, it was, and then we looked at what we had learned and tried to make sense of it and synthesize it all. It's very much a practical skill, I see. Oh man, I mean, listen, we were looking at whether to buy a TV yesterday, and Beverly was reading reviews. I mean, you know where they did tests and things, and we didn't go to, we didn't go to Kardashian's Instagram to find out which TV she has, you know, you, I think people do this on a regular basis, and they just don't recognize that, that regularly we, we engage in evaluating points of, or sources, or testimony of others observations, and weighing those against one another to try and make a decision, even about buying a television. Yeah, sometimes we spend longer trying to decide how to buy a television, Scott, than how to buy a house. Yeah, if you think it's a critical thinking, you find out that basically we can't navigate our day without doing it. Yeah, we'll have to do it in out of seconds, but we don't always do it well. Sometimes we, sometimes we gather the evidence that supports, I really want this and give it to our spouse or our significant other. And I really want this, and here's why these are the four things that say I should get this. And if you have a savvy partner who says, yeah, but here's the evidence against what you want to do. That's a great point. But and also, Lennia, I think one of the things that is true about doing something like a literature review, I play guitar, and so, you know, I'll learn in arpeggio, and I can, once I've got it down, I don't have to think about it anymore. But one of the problems is, when I do have to think about it, it's hard to do. When you really try and unpack something that you've begun to do naturally, it really becomes a laborious and difficult and you, you know, you stammer and struggle and and I think that I see that happen with a lot of students where they're good at thinking about the testimony of others and working that together to kind of the vibe. But when you start to do that in a critical way and you're looking at your fingers as they're making their move through the arpeggio, then all of a sudden, paying attention to the process really becomes another, another layer or maybe more, more taxing on the your CPU. I don't know what it is, but it makes it, makes it more difficult. One of the things you just wrote up is very important that a colleague of ours at the Vernebach Parker used to talk about thinking about your thinking. And the idea of metacognition is fundamental in a lit review. It's also a fundamental decision-making process. And so the idea of the reflexivity that one needs to apply and the re-application of our understanding of our dispositions are critical through all of those processes. Otherwise, we do what Linia said and we move to our emotions and our needs in one trial and then to the rational side. We move to that confirmatory. Well, I mean, recognizing our own tendency to our own confirmation bias that we all have is part of the dispositional nature of self-leadership through a lit review and through a complex decision-making process when you're faced with a wicked problem in the world. And so being able to say, yeah, I know what I think I want, but now I'm going to listen. And that takes a good bit of thinking about her thinking and thinking about her feeling. Yeah, I think the difference is, look, my two-year-old granddaughter engages in inquiry. She sits in the high chair and she puts food, she drops food, you know, onto the floor to see if it'll splat on the floor and every time it does, she giggles and she does it again to see if it'll happen again. That's research. She's doing inquiry, which she's not doing is thinking about thinking about the inquiry and that's the hard part. That's why that's the difference between being two and in a doc program.