S10 E5: Reimagining comprehension assessment, with Gina Biancarosa, Ed.D.

Hey listeners, it's Susan. If you've been with me on this 10th season journey, then I have the perfect event to tell you about. The Science of Comprehension Symposium will take place on Friday, February 6th. And as our listeners know well, reading comprehension is incredibly complex. And at this all-day virtual event, you'll develop a richer understanding of its multiple processes, products, and outcomes. Plus, you'll hear from some familiar experts, including keynote speaker Doug Lemov, whose recent episode was an instant classic. Seriously, if you haven't listened yet, you're missing out. I hope to see you at February 6th Symposium. Go save your spot now at amplify.com/comprehensionsymposium. Other reading comprehension assessments I was talking about can tell you if someone's not a very good comprehension, but they don't tell you what to do about it. This is Susan Lambert and welcome to Science of Reading the Podcast from Amplify. So far on this comprehension-focused season, we've explored why comprehension is so key, and we've unpacked some of comprehension's foundational components. Today, we're turning to a new angle, Comprehension Assessment. Our guest has done groundbreaking research in this field, and she's joining us to explain why it's critical to build on traditional methods of comprehension assessment. Dr. Gina Biancarosa is the Anne Swindells chair and a full professor in the Department of Special Education and Clinical Sciences in the College of Education at the University of Oregon. She's also the director of the Center on Teaching and Learning. In this episode, she'll detail her work to develop more precise and informative measurement of reading comprehension. This new assessment called MOCA is something you will definitely be hearing more about in the coming months. Quick reminder, we want to tackle your comprehension questions, too. Please share what you'd like to learn about at amplify.com/sormailback. And now, here's Dr. Gina Biancarosa. Well, Dr. Gina Biancarosa, it is so amazing to have you here on the podcast. Thanks for joining us today. Thank you. Thank you for having me. You know, listeners can't hear all the things that we do before we hit the record button, but I was saying how amazing it is that we are finally getting you on and how surprising it is that we haven't had you on the podcast before because you know a lot about both early reading development and how that relates to assessment. So I love it, though, if you could introduce yourself to our listeners and tell us a little bit about your background and how you ended up in this world of literacy. Yeah, yeah. Well, I grew up in Abbott Reader myself and really was passionate about sharing that with others. And so, you know, even as a child, I was involved in tutoring other students who were younger than me, you know, volunteered as a tutor for young children. And took a little side trip into hospital administration. Yeah, yeah. And realized, you know, after a few years of that, that this was not my passion and that I wanted to get back to reading. And so I pursued higher degrees in language and literacy at Harvard Graduate School of Education. And was intending to be a literacy specialist, actually, but was seduced by research. I had found that I had a knack for it. So, and became really particularly obsessed with the notion of improving our assessments of reading and particularly of reading comprehension. That's an interesting story. I think that's the first former hospital administrator turned researcher. So that's really cool. It's encouraging, though, because that, you know, I often talk about the fact that when we have a passion for something, sometimes we don't understand the range of opportunities that's available to us. And I'm sure nobody thinks, well, when I grow up, I'm going to be a reading researcher around assessment. So, that's really fun. You are at the University of Oregon. Yes. Yeah, I'm a professor in the Anne Swindells Chair in Education and also the Director of the Center on Teaching and Learning. Oh, congratulations. That's an amazing, an amazing role. So, you actually do have a lot of influence in terms of preparing our teachers to go into the world along with this work that you're doing in reading comprehension research. So, you know, this season of our podcast is season 10 is all about reading comprehension. And I really would love, since our focus is on reading comprehension, if you could provide your sort of definition of that and maybe some of the challenges teachers encounter when they're faced with, you know, trying to teach comprehension. Yeah, yeah. So, reading comprehension, a lot of times people talk about it as though it's only the end product sort of what you have after you're done reading. But I really think of reading comprehension as a process, as what happens as you're reading, as you're sort of constructing that coherent mental model of what it is that you're reading, the message or the story. And so, when you focus on reading comprehension as a process, it really attunes you more to the cognitive strategies and processes that we engage in in order to create that coherent mental model as we read. That really, for me, is reading comprehension. And it emphasizes the interactive nature of reading comprehension that the reader is actually an active agent in creating meaning, which that it's not a static message coming from the text, but rather that it's the reader creating that coherent mental model. That was a big ah-ha to me too. I don't think I understood the reading comprehension as a process when I was actually a teacher in the classroom, but always, just like you said, saw it as a product as an outcome and didn't always understand that, well, I say it now that I think this is right, you're the expert so correct me if I'm wrong, but you don't just, even when you think about comprehension as an end product, it's not that you comprehended it or you didn't comprehend it. Comprehension is a range in a little more complex than that. Do I have that right? Yeah, absolutely. Yeah, there's not a singular message to be obtained and once you have it, you're done. There's really a spectrum and a lot of how much we comprehend and the depth with which we comprehend what we read depends on our goals for reading, right? We read a news article differently than how we read a novel for pleasure, differently than how we read something that's been assigned for a class, right? And even when something's been assigned for a class, oftentimes, you know, there are different purposes within that in terms of, you know, paying more attention to the details versus the gist. A lot depends on what we're being asked to do with what we've read. That's a really good reminder. You know, we spent a little bit of time talking about word recognition as the gateway into reading comprehension, right? If we can't get word level reading accurate and automatic, it really impedes reading comprehension. And we've talked a lot about, you know, how do we assess that? And so through that area of reading, that word level is a little more straightforward in terms of assessment. But can you tell us a little bit about how difficult and how complex assessing reading comprehension is? Especially if we're talking about it as a process, right? It's not a product. Wow, that's even more complicated. Exactly, exactly. And historically, you know, for over a hundred years, the way that we've assessed reading comprehension is by asking questions after someone's done reading, right? And the problem with that, it's twofold. One, you're not getting at that process. You're only kind of getting at the end state. But even more importantly, once you ask a question, you can change what it is that someone has understood from what they read. So tell me more. Well, basically, we could ask questions about inferences that one might have made while they were reading. But you could have a reader who didn't make an inference while they were reading. And it's only instigated by the question itself, you know, when we ask sort of, you know, why did Harry hide under the visibility cloak, right? They might not have been thinking about what was motivating Harry, sorry, a little bit of a Harry Potter fan. But once we ask that question, suddenly, you can think, hmm, I wonder why? So you may not have been wondering why while you were reading, but the question itself instigates that thought process. And so that's really the problem with asking questions, period, you know, even if you were to ask them during reading, which folks like Kristen McMaster have done has done a lot of research on, do we improve comprehension by asking questions during reading versus after reading? And the issue with asking those questions is again, you're, oh, you're changing that process, potentially changing that process. For some readers, they may have made those inferences naturally. But you have no way of knowing whether they were made naturally or not when you ask questions after the fact. There are a lot of other ways that people have tried to get a reading comprehension. One of the stranger tasks that folks have engaged in is having students break apart words. I'm trying to remember what that test is called. It's like the torque, the test of reading comprehension. I'm not sure, but what they do is you have a text where all the words are pushed together, all the spaces are gone. The task of the reader, yeah, is to break up the the stream of letters into words that create sentences that create meaning, which is a really strange task. And, but it also taps into one's ability to recognize a meaningful message when they see one. That's interesting. Never heard of that one before. There's another task called the test of silent reading efficiency and comprehension where I think of it as an assessment of basic reading comprehension where you read a sentence and you read it as true or false. The idea being you have to understand the message of the sentence in order to understand whether it's a true statement or a false statement. And the statements are very sort of common knowledge that every woman know. This guy is green, false, right? The grass is blue, false. Fish live in the water, true. Things like that. Another approach that has been developed is something called a sentence verification technique where individuals read a passage and then the passage is taken away and they have to rate sentences that are related to that passage. Some of them are verbatim from the passage. Some of them are paraphrases. Some of them are paraphrases but with meaning changes. And again, you're rating things as yes or no, this was part of the passage, was part of the meaning of the passage. So there are a lot of different ways that we get at this. In dibbles, we use a maze approach where we're reading in every seventh or or nth word is deleted and we have to choose among three options what best fits into the passage in the place where that word has been deleted. They really only get a sentence level comprehension. They don't get it that deeper comprehension that we're generally trying to build as we read. So it's a really challenging task and there's a reason that we've been assessing it in the same way for over 100 years by asking questions after reading because people have had a really hard time improving upon that. As you know, I'm one of the lead authors behind a more innovative measure called MOCA that we developed to try and get at that process of reading. And this one is really based on think-aloud research. So a lot of what we know about reading comprehension comes from think-alouds where you ask someone to tell you what they're thinking about as they read. So they kind of pause after each sentence and give you insight into, oh okay, I think Harry is hiding under the invisibility cloak because of this that and the other thing. So that's generated by the reader himself and not like sort of prompted by a teacher. Exactly. Exactly. So they're trained to, after each sentence, tell me what you're thinking. And so it's freely generated by the reader. And then you code what they're saying in order to understand what processes are they engaging in? Are they paraphrasing? Are they making inferences? Are they making evaluations? Are they connecting to their personal experience? There's tons of codes for sort of what are the things that they're doing? And so that's how we've learned about what good readers do when they're reading. And that's where so many of the reading comprehension strategies that we teach children come from is from the think-aloud research. How far back does that think-aloud research go? Think-aloud research goes back, gosh, at least to the 50s. It's probably older than that actually. Yeah. Yeah. Yeah. And you know, there was a lot of work done specifically around reading comprehension with think-alouds in the 80s. Okay. That's when sort of the reading comprehension strategy instruction sort of was born more in sort of the 90s as based on that think-aloud research. That is really interesting because I can imagine as a former classroom teacher that I would be able to action that in my classroom to be able to get inside a, that's like a heavy one-to-one sort of process. Exactly. Was it ever actioned like that in the classroom or was this more of literally a research project? So it's primarily a research approach. There are individuals like Isabelle Beck who have promoted think-aloud as a way of modeling reading comprehension for children. Okay. Yeah. But that's also really difficult. Yeah. It's really difficult to model reading comprehension thinking-aloud in front of a classroom of students in a way that is instructive for them, but also authentic to the reading process. So it takes quite a bit of training, but it is one way that you can teach reading comprehension strategies is by sort of modeling that through think-alouds. So there definitely this think-aloud strategy has more potential benefits than what we were doing with more traditional assessment. Is that what I'm getting? Yeah. Yeah. I mean, it's been one of the primary ways that we get at the process itself. Okay. There's also been sort of retrospective interviews where you ask people, "What were you thinking when you read that sentence?" But think-alouds are preferred because they're kind of natural in the moment spontaneous thought. So really, then, the think-aloud is it's the only mechanism so far to get at this process, that comprehension process, or what the reader's doing in the process of reading. Well, this assessment that we developed called MOCA is an attempt to kind of get around having to do think-alouds. So one of the other products that has come out of think-aloud research is we've learned that students who are struggling with comprehension tend to do one of two things when they're thinking aloud. They either paraphrase a whole lot like more than 80% of the time. And I like to think of those readers. It's very text-bound, you know, they're reading the text and they're receiving the message, but they're not doing anything with it. They're not getting interactive with the text in the way that good comprehenders do. And then you have a second group of these students who are struggling with comprehension who tend to make inferences. Some of them are valid, some of them are invalid, but they're just, they're making inferences all the time, not all the time. That's a little bit overstating the case, but they're making inferences a lot. What we tend to think of about these readers is that they're not being selective with their inferences. So they're making all kinds of these inferences that may or may not be helpful for building what we call causal coherence in terms of the story that they're reading. And that really, when we're trying to build a coherent understanding of what we're reading, we're really sticking to the central message of the text. So we're not getting waylaid by details and sort of, you know, saying, you know, I'm reading this story about a character and doesn't say anything about how the character looks, but I'm making lots of inferences about I bet this character has blonde hair and blue eyes and is, you know, average height and but a little overweight. What have you like these inferences that are nice to have, right? It kind of gives you a visual image, but aren't necessarily central to the meaning of the text. And so the second group of readers we tend to refer to Mocha as elaborators because they're elaborating the text, which is great. This is something that good comprehenders do, but they're doing it to, they're not being selective with the inferences that they're making. So what Mocha does is using a sort of modified maze approach, we instead of deleting a word, we delete one sentence from a passage. And it's always the next to last sentence. And what that does is it creates a causal gap in the story because you've got the story going along, then there's this missing sentence and then there's a conclusion. And so there's like something missing that gets you from how the story evolved up until the missing sentence to the conclusion. And students are asked to select the sentence that best completes the story. So we don't ask them any questions, we just say what best completes the story and that's up to them. And so what we find is that our poor comprehenders tend to pick paraphrases for those sentences or elaborations that don't fill that causal gap, whereas our good comprehenders recognize, no wait, something has to happen to motivate this ending to make it make sense. And so they're picking the causally coherent inference, the one that really ties the whole story together, whereas your elaborators are picking something that sort of, you know, goes off in left field. Interesting. Yeah, yeah. Can we just take a quick step back and you mentioned mocha, it must be an acronym for something. So can you let us know what it stands for? Yes, I'm sorry. We speak about it in shorthand because the name is a mouthful. It's the multiple choice online causal comprehension assessment. It makes me think of really good coffee though, right? So mocha and rice. I mean, now I have all kind of questions of how, how did you get to this place where you are like, oh, we're just going to remove a sentence from these two, I'm assuming that they're shorter, they must be shorter passages that you put in front of kids. Okay, how did you get to the place where you're like, we just, if we remove that sentence, it's going to do this thing and we're going to be able to see kind of the process that that readers are using to get there. That's that's a great question. My colleagues, Dr. Sarah Carlson and Dr. Ben Cyple really came up with the brain child of mocha when they were still doc students at University of Minnesota. And the idea is that, you know, when you're deleting a word, it's very constrained. It's constrained both by the passage, but more specifically by the sentence that it's in, whereas when you delete an entire sentence, you're deleting a thought, right? Something that is an occurrence in the story. And so their idea was to create a gap in a story such that it forced the reader to make an inference in order for the story to make sense. They really drew on a lot of the work of Paul Vandenbruck and his colleagues who have done a lot of work around this idea of causal coherence in terms of the role that that plays in reading comprehension. The good comprehenders are really prioritizing that, the causal coherence of what they're reading in order to make meaning. And that that's what really distinguishes them from poorer comprehenders or comprehenders who are really struggling more with establishing that coherence. It draws on the landscape theory of reading comprehension, which sort of, I won't get into here because it's actually pretty heavy stuff. But it's this idea that the way that we draw coherence from what we read is by paying attention to reinstantiated concepts and understanding that our role as a reader is to keep those ideas active as we're trying to make meaning. That makes sense. I'm going to actually pull out the articles. So we have a couple of articles that I read in preparation for this super fascinating, this idea of causal coherence. In this one article, you have three short sentences. It says, "Consider the following very short story." So I'm going to read these to you and I want you to talk a little bit about this idea of inferencing for coherence. Tyrese decided to bake a pumpkin pie for dessert. He looked in the pantry. Tyrese was disappointed. Yeah. So what does a good reader have to do to comprehend those three short sentences? Yeah. So most of your listeners, probably all of them immediately inferred that Tyrese was missing some ingredients. And so he's not able to make the pie and that's why he's disappointed. None of that is explicit in the passage. And that is very typical of the things that we read where there's this reading between the lines that we have to do in order for the story to make sense. Because without that inference, it's a string of non-sequiturs. It's like Tyrese wants this thing. Tyrese does this thing. Now Tyrese is feeling something. Like what's the connection? The connection is made by the reader, right? So even going to look in the pantry is a motivated action by the fact that we're inferring, "Oh, okay." So he's looking to check for the ingredients. And so that's what good readers are doing when they're reading oftentimes completely unbeknownst to themselves because it's such an automatic process for a proficient reader. For our struggling readers, you've got your paraphraser for whom this really is a string of non-sequiturs. They can tell you what the story said, but as to why Tyrese looked in the pantry, why he was disappointed, they're not making those connections. And your elaborator, the one who's making inferences, but not like selective inferences, they might have imagined what Tyrese looked like or where the pantry was in his kitchen. They might even imagine what's in the pantry, right? Or basically details that again, do enrich the story, but don't necessarily make it make sense, but don't make it cohere. Yeah, that makes sense. I love that example because it's three short sentences and a couple of things what you said about inferencing. I think it took me a while to recognize the fact that you can make inferences automatically without ever knowing it, right? So there's some level that it can be an automatic response. I think as a teacher, I always thought, oh, you got to teach that process, right? But at some level, it gets to be automatic. The other thing is that the elaborators, right? They bring all probably this rich background knowledge, they understand what a pantry is, they understand what's in a pantry, they understand, but yet they're not making the connection to what the passage actually means or the purpose of that passage. And to me, that is actionable then. And I think this is where you're going with some of why Mocha is really helpful because if you say, well, hey, Susan is a poor comprehender. What do we do next? I don't know. Right? Right. Yeah. Yeah. No, I mean, that's that's exactly the issue with all of the reading comprehension, other reading comprehension assessments I was talking about is like they can tell you if someone's not a very good comprehender, but they don't tell you what to do about it. And if the problem is word reading, we've got lots of assessments that will identify that and we know what to do, right? We've we've got the interventions that are kind of all set up there. But what Mocha is trying to do is to reveal what is going on in the mind of the reader, what is their process? And then what's the next step, right? And the next step is really informed again by our colleague, Kristen McMaster over at University of Minnesota, both Sarah Carlson and Ben Cypill worked with her as doc students and, you know, they've developed these questioning interventions that really prompt our struggling comprehenders to make the connections that they need to make in order to improve their comprehension. And so for your paraphrasers, you're really just trying to get them to interact with the text at all. So a lot of the questions that you ask are really asking them, how does that connect to what we already know? How does that connect to the sentence just before it so that they're trying to just get them to realize you've got a role in connecting ideas across the text versus the students who are elaborating, they're making inferences and that's great. But we want to get them more directive or more selective with those inferences. And so we use a more specific questioning technique called causal questioning where we ask them why did Tyrese go to the pantry? Why was Tyrese disappointed? And by constantly asking those why questions, we're really prompting them to attend to really the causal mechanisms in what they're reading. And I should emphasize, we always kind of illustrate this with narrative texts and with stories and Mocha relies very heavily on stories. But this causal structure also exists in expository texts, particularly science and historical texts. That's very interesting. Just thinking about this idea of the dimensions of comprehension, I'm wondering, I don't know if that's the right way to say it or not, but I'm wondering how understanding what paraphrasers and elaborators might need to have to support their comprehension. Is that also beneficial for someone like me who is actually a good comprehender? Can I learn from some of the strategies that you engage paraphrasers and elaborators with? Yeah, you're not going to be harmed by being asked these questions, right? I mean, in fact, you probably already know the answer when I ask you these why questions, you've already inferred that. So it reinforces your attention to the causal mechanisms in the story. When you were talking about the dimensions of comprehension there, it brought to mind situation model theory, Kinch and Van Dyke talk about, they're the authors of that theory and the idea is that we have at least sort of three dimensions to the meaning that we're making as we read. And the first is sort of the text base. And so that's the first level, it's kind of, do you get the literal message of the text? And that's where I sort of see those paraphrasers stuck at, is that text base level. The next level up is what we call the propositional level, where you're sort of extracting the most central ideas, maybe letting some details go. So really getting at the gist. But the level above that is that situation model, that coherent situation model. And that's where we start to make connections to our background knowledge to potentially even learning, right? And changing our background knowledge as the situation model is built. So that's another way to think about the dimensionality of reading comprehension. I love that you brought that up. I often use a sentence, a couple of sentences when I do trainings or anything like that about reading comprehension or presentations. And I think this is the way it goes. I should have it memorized by now. Carla forgot her umbrella and got wet on the way to school. And the question is, why did Carla get wet? And if you say, well, because she forgot her umbrella, that would be one that's just text base. It's right there in the text. And other people can say, oh, because it was raining, but the passage doesn't say it's raining. So that would be that deeper level of connection and comprehension. Exactly. And that's her causally coherent comprehender right there, the one who's like, it goes beyond what the text says. There's more to this story that makes it make sense. And that, to me, is depth of comprehension, just like you outlined with Kench's model is that, well, it's not wrong to say she got wet because she forgot her umbrella. That's absolutely right, right? But there's another there's another layer. Wow. Yeah. I just had a good aha. So thank you for that. So you've been working on this research for for some time now. Is there anything that really surprised you in terms of what you learned in the process of doing this research? Yeah. I think what surprises me is, you know, we've been working on this for more than a decade now. And we have we started working in grades three to five because that's seen as like this transition to, you know, reading reading to learn, right? And so the importance of readers becoming active agents in their own comprehension. And what has surprised me is that there's been a real hunger for this at the middle grade level. And we have expanded up to grade six, seven, eight, and even nine. And Ben and Sarah have even developed a sort of college version of of Mocha. And what surprises me is that you find these paraphrasers and elaborators throughout that developmental arc. So one of the things we had sort of hypothesized was that maybe paraphrasing is sort of an earlier developmental stage, right? Then you're an elaborator and then finally you're a causal comprehender. And that that may or may not be true. I don't think we have enough evidence to say whether that's how an individual develops in their reading comprehension. But what was surprising is that you can get to eighth grade and still find that you've got like 10 or 20% of your students who are really stuck at that text-based level who are paraphrasers. That for me was a real aha. So it's been a real learning experience working with middle grade teachers and understanding that these same issues are being experienced by their readers. Is there any, have you done any work with the same kind of idea in the earlier grades where it's related to listening comprehension? Like can we sort of look at K2 and some read-aloud environment or something and sort of say, oh, this group of students is exhibiting behaviors that might be paraphrasing? Yeah, yeah. We have actually expanded it down to grade two. I think one of the issues is that decoding starts to be this bottleneck, right? So if they're not decoding well enough, Mocha is overly challenging for them and you end up with what we call an inconclusive categorization where it's just like they're guessing because it's too hard to task. We have thought about putting it into a listening comprehension sort of mode, but we haven't gone there yet. I think that's one of the sort of next steps for the work that we do. I will say I've been a little bit less motivated to do that because I keep mentioning Kristen McMaster. She's a good colleague and does a lot of related work. She and Paniota, Kendo, also at the University of Minnesota, have developed the Minnesota Inference Assessment or Mia, which is for grades K to two. It actually uses video, which I think is potentially a little advantageous to just relying on listening comprehension. So students watch a little narrative and then are asked to make an inference about what they've seen. Now their assessment isn't focused solely on causal inferences, so they're also assessing more sort of lower level just connective inferences. So to go back to this tiry story, when you say goes to the pantry, you need to infer these in this kitchen, right? Pantries are in the kitchen, things like that that do lead you towards coherence, but aren't solely focused on coherence. But yeah, I think the challenge with this is when you think about doing it for listening alone is the mocha task requires a lot of sort of working memory. And so by having the text in front of you, you can revisit propositions that are sentences that were in the assessment. And one of the details that the experience of the assessment is if you click on a sentence, it appears in the story. So they don't have to do the mental effort of like, okay, I'm reading and then I know this sentence and then back up here, it helps them kind of see where it fits in the story and read it within the context of the story to kind of settle on which of the sentence is best fits. And I worry that when we only do it in a listening context, that it's an overly demanding working memory task. So that's the challenge that I think we'll have to tackle if we want to go to the younger grades is how do we keep it within the working memory capacity of our kindergarten to second graders? Yeah, that makes sense because otherwise then it's just an assessment of working memory as opposed to an assessment. Exactly, exactly. This is, I'm going to underscore this for our listeners, but this is an exciting new breakthrough, if you will, in assessment and an assessment that actually delivers instructional support for teachers. How do you see this assessment actually impacting classrooms? Yeah, I mean, we're seeing an impact classrooms already. The hunger for professional development around these questioning strategies is real and that's sort of one of the next developmental tasks that Sarah and Kristen are actually really focused on is how do we give teachers more support around how they can ask these questions within their instruction. And I talk about it as professional development rather than an intervention because a scripted intervention will only get you so far. What you really want to do is sort of arm teachers with the knowledge to understand where these causal gaps are in texts that they have their students reading. Being able to identify where those causal gaps exist and then knowing the kinds of questions to ask at that moment is really the skill that we want to ingrain in teachers. That's awesome. And can you do one more review of that? So if we have and we notice a student is a paraphraser, in other words, they're just sort of parroting back the things that they see in the text. What kind of questions do we want to ask that student? So we want to ask them connecting questions or it's also just called general questioning. But it's really sort of like as they come across new information asking them how does that connect to what we already know? And that's really if like if I had one question I could teach teachers that would be it for the paraphrasers. How does that connect to what we already know? So getting them to kind of go back to the text and really try to glue together the concepts as they're coming in for the elaborators, right? The questioning technique that we really recommend there are these why questions? Why did the character do this? Why is this important? And so, you know, you've got your how and your why. So that's kind of my shorthand for the two questions. I love that. That's a good sort of, you know, something to remember that being mindful of asking the how questions and the why questions, and not just because they feel like good questions to ask, but they're actually productive in terms of helping students think about those connections within passages. That's super helpful. Thank you for going back and reviewing that. Yeah, yeah. Any closing thoughts or advice for our listeners as we wrap up? Yeah, I, you know, one thing that I really also try to emphasize when I'm talking to teachers is that in some ways comprehension, what's so baffling about it is, you know, it's so hard to move the dial on it to actually enact change in what a student is comprehending as they read. But on the other hand, students are comprehending all the time, right? They're actually making causal inferences in their daily lives when they watch movies and when they're hearing stories, right? And so what we're really trying to do is get them to generalize these behaviors that they engage in outside of the task of reading during reading. So we're really trying to get them to sort of raise their standards for how much they should understand of what they read when they read. That's such great such great words of wisdom to think about. There are things that students are already doing outside of the process of reading that we can actually bring in and help support them while they're reading. So that's awesome. Well, Gina, it's been a pleasure. Thank you for the work that you're doing. We can't wait to watch it all unfold and move forward even more. We'll link our listeners in the show notes to an article or two. But thank you so much for joining us. And again, it's been a pleasure. Thanks for the work that you do. Yeah, it's been an absolute pleasure. I always love talking to you, Susan. Thank you. That was Dr. Gina Bianca Rosa. She's the Anne Swendell's chair in education and a full professor in the Department of Special Education and Clinical Sciences in the College of Education at the University of Oregon. A special thanks to Gina for helping make an article we discussed open access. Check out the show notes for a link to diagnostic and instructionally relevant measurement of reading comprehension. Keep your eyes peeled for more news about the mocha assessment in the coming months. Next time, we're continuing to explore this fascinating area of comprehension assessment. We'll be joined by Dr. Melissa Farrell, author of Reading Assessment, linking language literacy and cognition. I think we ultimately really have to ask ourselves what it is that we want to learn from this assessment. That's next time. You can also access a new collection of free comprehension resources, including eBooks and on-demand professional learning by visiting at dot amplify dot com slash comprehension 101. Also, don't forget to submit your questions about comprehension at amplify dot com slash s o r mailbag. On the latest episode of the Beyond My Years podcast, Dr. Tina Bogren discusses research back strategies for improving teacher well-being. When I was an instructional coach and mentor working with new teachers, I would have teachers that they were showing me they were in dissolution, but I didn't know what to do, so I just kept shoving instructional strategies down their throat. And then I learned, oh, let me take a step back. And typically what I, my number one question that I'll ask is, tell me how you're sleeping. Oftentimes, the tears will start flowing. That's available now in the Beyond My Years feed. There's a link in the show notes. Science of rating the podcast is brought to you by Amplify. I'm Susan Lambert. Thank you so much for listening. [BLANK_AUDIO]