ABA Strategies for Safer Playground Behavior: Inside JABA 23

(upbeat music) Welcome to the Behavioral Observations Podcast, stimulating talk for today's Behavior Analysts. Now here's your host, Matt Sequoria. - All right, we're back with the 23rd installment of the Inside Jabba Series Podcast on Behavioral Observations. I'm joined, of course, by editor-in-chief, John Barrero. Joining us also to discuss a really fantastic paper in the spring issue is Jeannie Donaldson and Gabby Gomez. This is the paper called Effects of Differential Re-Enforcement Time Out on the Unsafe Playground Behavior of Young Children. And again, that is in the spring issue. John, I'll put a link to that in the show and I'll get to the other way right as we start here. And there are so many aspects about this paper that are both worthy of discussion and just things that I found really, really interesting at. So I am looking forward to getting into this. Before we do that, though, I'd like the audience to have a little bit of background on you guys before we get going. So Jeannie and Gabby, if you guys can give us a brief introduction, who you are, where you work, what sort of things you're interested in and then we can get into the paper. Sure, I'm Jeannie Donaldson. I'm a professor at LSU in both the school psychology and behavior analysis programs. And I do a lot of school-based research in mainstream educational settings. Hi, I'm Gabby Gomez. I am a second year doc student and Jeannie's lab in the school psychology program. I got my master's in behavior analysis from the University of Miami and I'm originally from Brazil. So, and I've been really interested in school research ever since I joined Jeannie's lab. It was, joining her lab was my first experience with this type of research and it's been really fun. Well, I'm a big fan of Jeannie's work as a long-time listener as well now. And I think you were, was it session 171, I believe. And I'll put a link to that in the show. And some people want to go back and hear the deep dive on the good behavior game. I've done a lot of really great research in that area and then schools in general. So, again, I'm going to try not to fan boy too much during this podcast, but I'll apologize in advance. So, let's start off about the topic in general of unsafe playground behavior. Was there something specific about this topic that led you folks to say, hey, this is an interesting thing that we should try to intervene on and maybe publish some research about what was the motivation to get into this? - Yeah, before I answer that, I just want to mention that I'm really honored that this paper was selected for this because I listen to a lot and behavioral alterations podcasts, but I listen to every inside Java. And so, it's a real honor to have a paper selected for this series. So, we, and the first author, Liz Linton, unfortunately, couldn't be here today, but this is really her work. This was her master's thesis. So, we'll make sure to give her credit throughout. But we got interested in the topic. I think for a few reasons. One is that we spend a lot of time in schools and unfortunately, our local schools here in Baton Rouge, the students in elementary school, starting in kindergarten, have one 20-minute recess period. This is the only time of the day in the school day that the kids have unstructured play at school and time outside. Now, sometimes teachers work in other things, but this is the only kind of scheduled bit of unstructured time. And so, I don't know how you feel about that, but that terrifies me. Kindergarners need to be spending a lot more time running around, engaging in play, and they don't have as many opportunities as they should to do that. So, we kind of developed an interest in recess in part, not necessarily about recess safety, but in protecting kids recess time. Because what happens in schools is that frequently, students lose recess time based on behavior that they engage in in the classroom. Or if they have missed class or are behind on academic work, that might be a time when teachers pull kids to work or finish their work. And also, for behavior that they engage in at recess, they might end up for engaging in a bit of unsafe behavior at recess, which might be, I mean, might be unsafe, might be something that we would consider time out or the behavior. They might lose the rest of their recess for that. And so, in part, we're interested in protecting that recess time. And part of protecting that recess time has to involve ensuring that students are playing safely at recess. And if people haven't spent a lot of time on an elementary or preschool playground, it's total chaos. I mean, you've got lots of kids, relatively few adults, some equipment, and frequently a smaller space than would be desired. And, like I said, there is a huge EO for running and doing all these things because they have very little of this time. So, monitoring and supervising recess is a big stress for teachers and is important in promoting safe play and protecting that free play time that students have. And what do we know about-- I know you did cite some statistics about the likelihood of injuries and things like that. Can you give us some of the highlights of what your scholarship uncovered in this area? Yeah, OK. So, in the chaos that is recess, sometimes kids are injured. And, in fact, between 2019 and 2023, over 460,000 children under the age of six were treated in emergency departments for play-grominated injuries in this country. So, that's-- it's a lot of injuries. And when you think about it, those are severe enough injuries to have children sent to the emergency department. So, that doesn't cover your normal cuts, scrapes, bruises, and those sorts of things. So, it's not uncommon for unsafe play at recess to produce injury and sometimes relatively severe injury. Did you say-- did you say 460,000? 460,000. Now, that's in a four-year period, kids, six and under, in this only in this country. That's considerable. I'm sorry, I didn't mean to interrupt. I just want to make sure I heard that correctly. Holy smokes. Go right ahead. Yeah, it's a lot. And those injuries are much more likely to happen at preschools, daycares, and elementary schools. Then, when families bring their children to the park, that's a whole different scenario. So, we can get into this if we want to. But there's-- I know there's a lot of talk about that we are being overly restrictive on promoting safety and not allowing kids to play in these risky ways that are helpful for development. And I think that's an important conversation to have. But that feels like a very different conversation when you're a parent taking your one to maybe five kids to the park as opposed to you are a teacher responsible for the safety of 20 to 30 students at once. And they are on a playground with maybe 60 to 80 students at once. It's just a whole different scenario for what kind of risky play we can reasonably allow. And the risk of injury for risky play just gets dramatically enhanced in school. I see. So, I know there's been some work in the behavioral analytic literature about this. But not as much as I think we might think, given the magnitude of the problem that you just described, you think it'd be some proportionality role to that. But can you walk us through what we know? What sort of research has been done? What works? What-- what we know-- perhaps what we know doesn't work and so forth? Sure. So, I think it's interesting, too, that this is kind of-- as we've been doing this work, it feels like a bigger problem than anyone recognizes. And it's been really understudied. And also, to be honest, it's somewhat undervalued in schools, too. This is not a time where there is a ton of emphasis on behavioral intervention. And so, it's-- yeah, I think it's important that people take this up. So, what we know about playground safety, most of the playground safety work has been done non-behavior analysis, but in kind of engineering design of playgrounds. So, you know, the playgrounds that we grew up playing on, most of that equipment is no longer present in school playgrounds. You don't see the sea saws and the marigal rounds, because those are high risk of producing serious equipment. Ooh. Yeah, and so things that we feel-- we have some nostalgia about kids no longer get to experience at school. So, there's a bit more in that literature, but in terms of kind of behavioral safety, there's a lot less. There are a couple of studies. There was a study by Heck at all in 2001, published in Java, looking at kind of a-- it was a five-day safety training. So, they did a bit of teaching of the students about safe play in the classroom, and then they arranged some reinforcers at the class wide level for reducing overall rates of particular equipment misuse. And their effects were all right. There was a decrease in equipment misuse, although then when they do the follow-up and they're no longer maintaining those contingencies, we see behavior kind of return to closer to baseline levels. And then there's some other work not published in Java, but published in a pediatric psychology journal in 2006 by Schweibull at all. And they looked at changing teacher behaviors, so essentially promoting enhanced supervision at recess. Now, they didn't measure child behavior, but the assumption is that if we improve where teachers are, so they reduce teachers talking to each other at recess, and then looked at their proximity to students, and then they also had teachers delivering stamps to students, basically on a DRO type schedule. So if students were playing safely, and they essentially caught them playing safely, then they'd deliver a hand stamp. And this worked well to change teacher behavior. We don't know its effect on student behavior, but one concern I have about a procedure like that is I think it's really important that kids have opportunities to play with their peers in this way that doesn't have adult interference. And so I'll talk about this a little later when we're talking about, we use the punishment procedure, but I find when I'm trying to think about using reinforcement based procedures at recess, I end up having concerns about the intrusiveness of teachers delivering reinforcers mid-appropriate play. So if kids are playing appropriately with their peers, I want that to continue uninterrupted. And when we send teachers in to deliver a stamp or some sort of artificial, there are so many reinforcers at recess, and when we start to bring artificial reinforcers to that recess and interrupt that kind of play, I have some concerns about doing that. - That's a really interesting point, and it kind of reminds me of the discussion we had about the good behavior game, I don't know if you recall this, but the kind of like, because I've had a million people when I talk about the good behavior game. So, well, you know, they're reluctant to assign points based off of misbehavior in the classroom and to ask me, well, why doesn't the teacher just provide points when the students are engaged in appropriate behavior? And I think that's, you know, your point there is equally valid, but perhaps more so in the playground context, that's really a point that I hadn't considered previously. - Yeah, I find the same, and now we're getting a little off topic, but with the the copying good game version, when we've implemented that, it feels really awkward to interrupt the class when things are going well, and the teacher is like mid-teaching. As opposed to when disruptive behavior occurs, the lesson's already interrupted. So then delivering the feedback in that moment feels far less disruptive to the learning context than when we interrupt when everything's going smoothly. Not, I mean, their data, that works well too. I just don't prefer it. And students, as we know, from the study we published Jensen Chateau's Study in 2024 in Java, the students like the feedback. So they will choose to receive that feedback over no feedback pretty consistently. Okay, I'm going to take the elastic band and snap it on my wrist for getting us off track. And, you know, so, oh, Jensen's got it, yeah. - If you don't mind, I'd like to jump in there. Something Jenny said really early on in that response that I think is really notable and important is the ways in which this particular topic is important to other disciplines outside behavior analysis. So the observation network is being published in other journals like Pediatrics and that this involves efforts by engineers. I realized that this, you may not be collaborating with engineers because once they built the safe playground, everything is okay and then the kind of behavior side can kick in, but it just strikes me as a problem or a challenge that so many people will be interested in. It really represents that kind of under the dome behavior and what a research. And that coupled with your expertise and the good behavior game just speaks to the fact that you're addressing challenges that a number of people will be concerned about and the solutions for which a number of people will be really grateful for and benefit from. - That's a really good point, Sean. Thank you for mentioning that. - So are there other things in terms of the existing literature in this area that you wanna touch on or do you wanna get into the, if not, if so, let's talk about that. If not, let's transition to the work that you guys have done here. - Yeah, I'll just briefly mention that in my past work in studying timeout procedures, a fair amount of those applications were done at recess in schools. And so we have a fair amount of data demonstrating that when we're targeting individual students who have been identified as engaging in higher levels of unsafe behavior than their peers, then timeout at recess tends to work really well. - I see. All right, so in broad strokes with this paper, what were you guys trying to accomplish? What was the question or set of questions that you were trying to answer? - Okay, so I hope you'll allow me a little bit of an aside because I think this is maybe a venture, because it's an inside job I wanna talk about the review process a little bit. So I mentioned I've done prior timeout research. And so honestly this study is a little bit of, I don't wanna say cutiness, but a little bit of a response to some old reviews I got about a decade ago or over a decade ago. So I think as John was saying this work that we do tends to be kind of this under the dome, it tends to be a little different than the work that many behavior analysts do. And so I think sometimes reviewers who are working in different areas then don't always appreciate the context. And so I had reviews on, I think every one of my earlier timeout studies that suggested that there were ethical concerns with implementing timeout in the absence of a functional analysis and in the absence of exhausting reinforcement based procedures first. And I see laughing that because you work in schools, like, and I mean, you have spoken about this as well. And so it, on the one hand, it, here's the thing, I guess what I wanted to say about reviews in general, in the review process, 100% of the time that I have submitted a paper and received reviews, my paper has been drastically improved as a result of the review process. So pure review is tremendously important, it's tremendously valuable, it makes my work better and it makes me design better studies. Like, I'm really proud of this work that we published and it wouldn't have happened if these reviewers hadn't said the things that they said over a decade ago and left me stewing about it. So even when reviewers say things that I disagree with or that bother me, that tells me I need to do more work or sometimes it tells me I need to improve my writing so that they get the point. And other times it tells me, well, maybe I need to design better studies that can make people feel differently about this kind of work. So in any case, so in part, this is like my response to old reviews. Because what I wanted to do was remind people that a bit about the context. So yes, we're using a punishment-based procedure. But what's the baseline context at recess? Well, it tends to be, and the baseline context in most school situations, unfortunately, is inconsistent averse of contingencies. And so in baseline, what might happen is you might engage in unsafe play and it goes unnoticed, maybe multiple times over. So nothing happens or nothing from the adult supervising happens. Things happen. Reinforcers are delivered. Like, I shove a kid down the slide. I now get to use the slide. But nothing from the teacher happens. Then the teacher catches some instance of unsafe play is reasonably concerned about it. And then perhaps removes that child from the rest of recess. So we might have situations of more severe punishment than what we are arranging. And also, the larger context of recess is it's like a ton of reinforcers, a ton of special reinforcers. It's this special access to equipment, special unstructured time to play with your friends that you don't have at any other time. So you got a ton of reinforcers during time in. Time out, a brief time out, can be a really effective intervention. And as I mentioned, thinking about what the kinds of reinforcement arrangements we might set up, those feel more intrusive to me than the more natural. You've done something really unsafe and you need to take a break. So trying to get at the context, and in part, what we also wanted to do in this study was look at-- so we try to reinforce and face procedure first. That's the DRO part. And we designed it in a way that we thought might not be super effective. It's a whole recessed DRO. And Gabby will talk about what that looks like. So if you were to look at the DRO literature and look at our procedure using, why did you design it like that? Because that's not how I would design a DRO. But we did that for practical reasons. This felt like something if we're going to do a reinforcement face procedure at recess, something that teachers could do, something that's not going to interrupt ongoing appropriate play, not going to be intrusive. But if it doesn't work, we'll add time out, which we had to do for three of the four participants. And then what we cared a lot about is-- well, we as a behavior analyst get very concerned about the addition of aversives. And we should be. As I mentioned, there are too many aversives in school contexts, both at recess and throughout the day, and in the world of children and everyone. So part of our job is to reduce aversive context. But I think also what we want to know is what is this like? What does it feel like to experience these interventions from the perspective of the participants? Are we saying we shouldn't use time out procedures even though they're effective because they're labeled as punishment procedures? And if we're saying that, then I think we get ourselves into a problem. When we say reinforcement procedure is good, punishment procedure is bad. I want to know what does this feel like from the perspective of the kid? How does this affect their recess enjoyment? Are we wrecking recess when we have a time out contingency? And what would they choose if they get to pick what their recess experiences, then how do they want the contingencies arranged? So we asked those questions. And then we also, as an additional measure, looked at the social side effects. And this was because anecdotally, I'd seen with other kids at recess where we implement time out. And then they have to figure something else out. And what they figure out tends to be highly appropriate social interactions with peers. But we'd never captured that before. So the purpose, in part, was to look at all of these potential side effects of time out and capture those. I see. I appreciate that extended context there. So I think that it's perfect given that this is the inside job of series podcast to talk about how the review processes shaped your research behavior. So let's get into the details of the paper. So can you walk us through where was it done? Who were the participants? What specifically falls under the umbrella term unsafe playground behavior-- certainly, I can imagine instances of it, but I'm curious how that was defined and measured and things like that. So if we can get into, maybe, the beginning part of the methods section, that would be a great place to start. Yeah, absolutely. I can talk a little bit about that. So essentially, we had four participants total. It was Brady Austin, Travis, and Crim. Brady was a four-year-old black male. And he attended a public school in Louisiana. So I was in part of the data collection process for Brady. I didn't get to meet him or see his playground area. But from what I understand, Brady's playground area was a pretty big playground area. They had a jungle gym playground structure in the middle. And there was about 50 kids out at playground at the same time, which I think it was three different classes. And then there was three supervising teachers out there at the time. And then, Karim, Travis, and Austin were attended the same preschool-- different preschool from Brady. And Austin was four, Karim was three, and Travis was three, as well. And they were all in the same class. So their playground area was a little bit different than Brady's playground area. They had-- first is that there was not as many kids out at the same time. There was about 15 kids out their playground. And they had two different playground areas that they were out sometimes. One of them was a concrete playground area. They had some play houses. And they had these little tricycles, as well as random toys, like balls, dolls, and whatnot spread out into playground. And then they also had a grass playground area where they would go to sometimes as well. They also had a small playground structure in the middle. They had some swings. They had a merry-go-seical, and a slide. So those are the different settings in which our participants were a part of. So before we began baseline for all of the participants, essentially, what Liz did is that she met with each of these teachers. She talked to them about what was happening during playground, what were their main issues, what they're concerned about, what were rules that they had during their playground time, and what were really common rule violations. So that's how our rules were picked, essentially. So based on reports from the teacher. And so once those rules were picked out, and essentially what those rules looked like was they were related to aggression. So essentially, it was no hitting peers, no pushing peers, no grabbing peers, no throwing things at peers or at their teachers, as well as some of them were related to playground equipment. So unsafe ways to engage with playground equipment. Nose lighting down this light, if another peer is already on this slide, no sitting at the bottom of the slide, or no jumping off the slide. And then for Austin, Travis and Krim, they had these little tricycles on their playground. So we had some rules that were specifically for those. So essentially, they weren't allowed to drive the tricycle into each other, drive the tricycle into the wall, or whatnot. So those are essentially the rules that we had. So what we did after those rules were selected is that we started our baseline sessions. So before baseline, we asked the teachers to tell the participants what the rules were, and then they just conducted their playground as usual, their recess time as usual. They were encouraged to react to all these unsafe things on the way that they typically would, and essentially that consisted of some reprimands. And teachers did say that sometimes they use timeout, but we didn't observe timeout during baseline with any of our participants. So when it comes to our DRO sessions, the way that they were conducted before the session started is similarly to baseline. Teacher went up to the participants, told them the rules, and also described the contingencies. And then they also asked some clarifying questions, to make sure the participants were understanding what was going on. So essentially they asked what will happen if you follow all the rules, and what will happen if you break a rule. So once that was clarified, and they were talked to the students, they were free to go to recess. And then during recess, if they broke a rule at the end of the recess, they wouldn't win a prize. But if no rule was broken, meaning no, they didn't engage in any type of unsafe behavior, they got a prize at the end of the session. And what happened during DRO+ timeout sessions, it was similar to DRO, in which at the beginning of the session, teachers read the rules, described the contingencies, and asked those clarifying questions that we talked about. And the contingencies on the DRO+ timeout included that if they engage in any type of unsafe behavior, they would go to timeout. So important to, I think, also clarify that Brady never experienced a DRO+ timeout condition. It was only Austin, Travis, and Krim. And also, the DRO conditions were implemented by the teacher, but the DRO+ timeout condition was implemented by the experimenter. What happened is that Austin, Travis, and Krim's teacher resigned, and the new teacher, in which they got assigned to, they don't want to be a participant in the setting and implemented intervention. So essentially, the experimenter described the contingencies, the participant, read out the rules, and told them, if you engage in any type of unsafe behavior, you not only won't win the prize at the end of the session, but you also will have to go to timeout. So the way the timeout looked like was that we had two sand timers, a sand timer that was one minute, and then a sand timer that was three minutes. And if a participant engaged in unsafe behavior, we'd go up to them and ask them to go to timeout with us. If the participant didn't engage in any type of challenging behavior upon the request of timeout, they would only sit for one minute, as well if they didn't engage in challenging behavior during timeout. And then if they did engage in challenging behavior, when we asked them to go to timeout or during timeout, then they would sit for three minutes. None of our participants ever had to sit for three minutes. They very, very sweet kids. - Can I comment on that? I want to say that one of the things that I've done pretty repeatedly on the series is to highlight what I think are procedural gyms, and I just want to highlight that one. So it's so cool. If you go to timeout without any objection, you have a shorter experience in timeout. But if there's objection and there's a relatively longer experience in timeout. So it's great that you never had to use the more extended duration, but just procedurally, I think that's a really nice touch because you have less experience in the timeout conditions. If you just go without much objection and then you can get right back to the play after one minute's duration. So I just thought that was super cool. - Yeah, absolutely. I totally agree. And then, essentially, our last phase was our choice phase. So during our choice phase, the conditions baseline, Dero and Dero plus timeout were implemented in the same way that I just explained. But the difference was that the beginning, the participants were able to choose what they wanted to experience that day. So we had a different laminated piece of paper. So we had a white piece of paper that was blank, I had nothing on it, and that piece of paper indicated baseline. And then we had a piece of paper that had a backpack on it, which essentially that indicated that they were going to experience a Dero condition. The reason why there was a backpack was because the prizes were always inside this one specific backpack. And then our Dero place timeout condition was indicated by a piece of paper that had a backpack and a stand timer on it. So at the beginning, we would ask the participants to choose what they wanted to experience for that day. And then we would also explain what each of those met. Once the participants chose, then that's what they would experience for the day. So yeah, that's essentially how our procedures, oh, and I think it's also another piece that's important to mention is that at the end of each session, including baseline sessions, we asked the participants to tell us how they felt about recess. So essentially we had a piece of paper that had a smiley face, a green smiley face, a neutral yellow face, and then a sad red face. And then under those, it said, I liked it a lot. It was okay or it didn't like it at all. We would go up to each participant and we would individually ask them how do you feel about recess today. And then they were able to make that selection, depending on how they felt. Once they made the selection, let's say they picked the green face, we would say, oh, you picked the green face. That means you really enjoyed recess. Is that right? We would just confirm that was their answer. And then we would move on. So yeah, that was a very, very clear description of the overall procedure. So I appreciate that. And again, I'll link the actual paper and the shown onto the episode. So folks want to retrieve that over a behavioral observation. They can kind of read along and go into, you know, refer to it from more detail. But that was really concise and informative. One thing that struck me, you know, I want to get to the results, of course, but I've just got a couple of questions prior to that. And one of them is that you guys took I/OA in procedural fidelity data. And those measures were just from my perspective. And again, I don't do this type of research share, any type of research for that matter. So I don't know if this is on or off base, whatever. But I'm trying to put myself in the mind of, you know, being on a playground with an iPad or a clipboard or some sort of means of data collection and you're collecting information on all sorts of things. And to have both I/OA and procedural fidelity so high, given this chaotic setting of a preschool recess, how did you manage to do that? That's a good question. And I agree, our I/OA was really great, which is awesome. But I think that there's like a couple of reasons why it was, that was possible. So the first thing is that I think it's important to highlight that in our lab, we have this history of collecting data during group contingencies. So we are used to be collecting data on 20 plus students at the time and trying our best to make sure we're observing everything that's happening in a classroom. So because during this study, we had fewer participants, you know, like Brady, we were only watching Brady, and then Austin Travis and Cram was just a three of them. So that was relatively a little bit easier for us, I think. So I think that's part of it. Also, I think that the behaviors that we were collecting data on were very salient. So it usually involved some sort of aggression. It was very clear that those that they were engaging on was unsafe. So, you know, they were like jumping out of like a really elevated surface or they were, you know, hitting each other or they were fighting or with the tricycles, what happened really often is that they were like, we run the tricycles into each other, run the trick, run the tricycles into the wall. So those were things that was very salient to the observer. So I think that was also a reason why it made it easier for our IOA to be high. And another thing that I think also mattered was we, we caught data on tablets and they're handheld and we can essentially walk around the playground as much as we feel like we have to in order for us to accurately capture what's going on. So I think a combination of all those three things really, really helped us when it came to having good IOA. - Okay, so this is doable for mere mortals is what you're saying. (laughing) - I think so. - All right, all right. It didn't seem like it from the description, so I'm glad to know it is indeed possible for the rest of us. All right, all kidding aside. What, how did this pan out? Well, walk us through the results, please. All right, so in terms of the main effect, so on unsafe behavior. For Brady, as Gabby mentioned, we never had to implement or add time out to the DRO. So in baseline, he's engaging in relatively high levels of unsafe play. Most of his behavior was aggressive behavior. I did observe some of Brady's sessions. And so he was a kid that there were lots of kids on the one single piece of equipment. And if a kid was in a location that he wanted to be, he might just yank that kid off. And then we implement the DRO contingency and immediate zero, immediate elimination of unsafe play. We did have one session where he failed to earn and did engage in several instances of unsafe behavior at recess. But other than that, all of the other sessions are at zero. He's consistently earning the reinforcer and consistently maintaining safe play throughout recess. One thing to note about Brady and I guess this kind of adds maybe a little bit to the context. So our student, so Brady was attending a local elementary school. He was a kindergarten student. We had a reversal to baseline that was very brief. It was only two sessions and then there were no more data. And the reason for that was that Brady engaged in unsafe behavior on the bus and lost access to transportation to school. His mother had no other means to get him to school. And so he did not attend school for the rest of the school year. And to me, that's just a horrible tragedy like that shouldn't happen to kids and we need better solutions. But it kind of emphasizes the context where the data are taking place. That these are schools that are serving kids in high-needs areas who not only have behavioral challenges, but there are many community challenges that their families are facing. Anyway, okay, so that's Brady. So DRO worked really well for Brady. Then for Austin, Travis and Karim, DRO did not work so well. And what I mean by that is that they all were successful on at least one session of earning the reinforcer and having zero levels of unsafe play. But then they were consistently unsuccessful, engaged in some unsafe play and did not earn the reinforcer for several consecutive sessions. So then we moved them to the multi-element comparison, comparing all three of those conditions. When we added time out to DRO, we see an immediate decrease for all of those participants actually in the first session that included time out. They engaged in zero instances of unsafe play, meaning that just the threat of time out was sufficient. And but eventually all of them did contact going to time out experiencing the time out contingency. But we see consistently low levels of unsafe play in the DRO plus time out condition and consistently higher levels in the DRO and the baseline sessions. The other thing that Gabby mentioned that I think is really important in this highlights maybe John's, what John caught on to with our time out procedure. So I love this time out procedure that we've kind of developed and refined over time. But none of our participants had to flip the second sand timer and stay in time out for longer than one minute. And this goes back to what I was saying at the beginning of part of what we are trying to accomplish as a larger goal is protecting kids recess time. And so I'm very conscious about how long kids are spending in time out because minutes in time out are minutes of this valuable recess time lost. And when we looked at how long these participants spent in time out collectively, they spent a total of six minutes out of the 332 minutes in that condition in time out. So they're losing very little of their recess for these tremendous effects on safety. And so that is a really important metric to me because if kids are having to go to time out repeatedly, they're spending longer durations in time out and losing, I mentioned that our public schools could get 20 minutes of recess. So if you have to flip that three minute sand timer and you spend three minutes in time out and you do that three times, you've lost half your recess. So that to me would not be a great outcome. Fortunately, we did not see that and kids spent very little time in time out, lost very little recess for what I would consider really important improvements in behavior. Okay, so that's the kind of primary measure, yeah. Let me ask question here, so were any of the students, let's shall we say, we're left in to go to time out when you're implemented it? I have experienced that just as a practitioner, usually with older elementary age, we're talking about preschoolers here who might have a higher level of cooperation with adult directors, but yeah, I'm just curious if you can imagine if someone's engaged in this rockest play of demolition derby via tricycle, which sounds like great fun to lots of little kids going off the time out might be something that one might resist. Particularly if you have a repertoire of aggressive behavior and perhaps not a great repertoire following directions in cooperating with adult directors and things like that. Yeah, I think Gabby might be able to tell you better what it looked like when they were implementing those sessions, but I do remember that there were concerns with Austin prior to us implementing time out because his teacher is the implemented time out. He hated time out, engaged in lots of challenging behavior during time out and teachers implemented it. And so there was some worry about how this was going to go with him. And so, and it's my experience that preschoolers regularly refuse to go to time out, I've had preschoolers run away and hide under playground equipment. And so, but this is consistent with... Gosh, is it the 23rd, Donaldson at all, 2013? I think we published a paper looking at, it was a one or four minute time out contingency and it was only related to compliance with the time out instruction. So as long as you go independently because that was, to me, a bigger challenge than managing challenging behavior during time out was like, don't run away from me, just get 'em to go. And so we had similar effects and so we just extended that to not only do you have to go independently, but you also then have to sit there reasonably, not run away and escape. So there's some other data indicating that that kind of back up contingency of facing a longer duration time out is potentially really helpful in improving compliance with the time out instruction or getting kids to cooperate with going to time out. Yeah, for the most part, they didn't give us any trouble for going to time out. Sometimes what would happen is that they would get upset like some kids would cry that that happened, which was heartbreaking to watch, but that didn't count against them. So that was usually that type of emotional responding is what we would observe, but no true challenging behavior when it comes to like hitting or running away. - So if Austin had a history of reacting poorly to previous attempts of time out and I won't hold you guys to this, I'm just asking you to speculate in one of the things I'm thinking about was that, you know, was it due to perhaps, you know, just kind of like a inconsistent application or, you know, a time out that was too long that made the averse of nature of it, you know, very influential on the resistance repertoire. Do you have any ideas as to why he was the, you know, previous attempts at using time out were unsuccessful with this participant? - I do think that it is true and what you said in the sense of like the time out was inconsistent and it was for really long periods of time. So an example of that is that, for instance, one of our procedural fidelity sessions is a little bit lower for Austin specifically because his teacher did put him in time out for doing something that wasn't necessarily a rule violation. He was laying on the grass so he was put in time out for laying on the grass and he was in time out for the remaining a recess period which was nine minutes and keep in mind that recess was usually 16 to 20 minutes. So nine minutes is a really long time to be in time out out of like, in general is a long time but if you really think about like if they were only out there for 20 minutes max, nine minutes is especially a really long time. So I do think that that might be a big reason why it was so much harder for him to go time out prior. - Okay, got it. So, Jeannie, I think I derailed you from your discussion of the rest of the results. I'd like to talk about the interaction data and maybe, 'cause I wanna have a separate discussion about choice which I think is really important here. So if we can reserve that for maybe the last data set to discuss and again, just for listeners, I, again, I will have, I can't cut past the entire article all the graphs and things like that but I would encourage people to go look this up and I will try to put some visual aids to this on the website where I can. But you guys have a great job verbally walking us through this too. So, you know, go right ahead. - We also have all of the data and our code and everything in open science repository. So we can, the link is in the article but I can share the link and so people can freely see the figures through OSF. - That's another aspect of this that I think is also noteworthy but that's a whole other different, oh, again, I'll stop interrupting you so we can continue with this paper though. - Okay, so we looked at positive and negative social interactions and actually before I get into the data, I'll like Gabby talk a little bit about what those, some examples of what those looked like. - Yeah, so the positive and negative interactions that we usually observed, it was, let's talk about positive first. So the positive interactions that we observed were essentially like, you know, very cute things that kids do, like inviting other kids to play, allowing kids to like join a crowdbole play that they were already playing with. It was in Austin, Travis and Scrims playground, it was they play traffic often, which was really cute 'cause they had the tricycle. So someone would be like the traffic person that he would have to like stop or go and it was sometimes they would either invite other kids that were on the tricycles in a different area to play with them or even just allow the kids to play with them. So that was a type of positive interaction that we observed a lot. And another thing that I saw kids doing is it's asking for a turn. So let's say like a kid is on a tricycle and another kid wants that tricycle. Instead of just like pushing or standing in front of the tricycle or you know, engaging in some type of aggressive behavior, they would ask, can I have a turn when you're done? That was also something that we observed. That was very sweet and it was great to see. Our negative social interactions more looked like the opposite of that. Essentially like screaming at peers when peers would try to join the play, like not allowing peers to play with them. You know, teasing peers or instead of asking for a turn if they wanted something, they would yinke from their peers or you know, stand in front of the tricycle or standing in front of the swing that they wanted to swing. I saw that a lot as well. So that's essentially what we mostly observed when it came to our positive negative interactions that we recorded. - Yeah, so then in terms of the effects on those, the condition that, so for Brady, who only experienced the DRO, negative interactions also drastically decreased. Although I should mention that, so his negative social interactions were primarily aggressive behavior, which were also his targeted unsafe play. So that would get double scored. So the contingency, that's not really a side effect. It's a direct effect for him. For the others though, where it wasn't a fairly double scored and most of their negative social interactions were not counted as unsafe behavior. We saw during the condition that included time out, fairly consistent low levels or most of the time, no negative social interactions during those sessions and sometimes a few instances. And during baseline and the DRO sessions, we saw sometimes low levels but more consistent negative interactions during those conditions. So overall, the condition that included time out or the condition that was most effective for all participants reduced negative social interactions or eliminated them for some participants. And then in terms of positive social interactions, we see the kind of the opposite effect. So either positive social interactions increased during both the DRO and the DRO plus time out conditions or in some cases like for Karim, we saw the highest levels of positive social interactions during sessions that included a time out contingency. And so this was, as I mentioned earlier, something that I had observed anecdotally and from other research, time out research, I conducted a recess that essentially kids when the threat of time out is impending, they figure out other ways to interact with their peers and they figure out those ways really organically. And as Gabby was mentioning, it's really cool to see because kids who get referred to us for this research tend to be kids who are struggling socially at recess. And then once we reduce their challenging behavior at recess, we also see these really lovely improvements in their interactions with their peers. And we don't have to do any sort of like social skills training or teach them scripts for how to include other students. They just figure it out on their own. And so that to me is a really nice bonus of, and not necessarily time out specifically, but I think just a function of I can't do these other things. I want to ride the trike. And if I do this, I'm gonna end up going to time out. I have to figure another way to get the trike. And the way that they figure out tends to be an alternative appropriate response. - It really makes sense to me that you have included these measures. But again, I think that this is one of those procedural gyms. So I think it may be clear, but could you just explain what was the motivation behind taking these measures? Like, and I'm wondering perhaps if this might have been a latent response to peer review that you experienced 10 years ago, but if you could elaborate on what it is that motivated you to get these measures, I think that would be helpful. - Yeah, I think it kind of goes back to that. People had been expressing, and I don't mean to suggest that, I'd also received lots of comments about people liking my time out research. So it's been a mixed bag of feedback, but reviews that suggested ethical concerns are the ones that really, I thought, wow, you just got to see what happens at recess when we do this, and you'd feel so differently about it. And so to me, as a researcher, seeing it means seeing it through the data. Now, I think in future studies, I'd actually like to capture richer data in terms of social interactions. We had these two broad categories. And so we can tell you, Gabby can relay these stories of kids including other kids, but we don't see that in the data. So I think even richer capture of this is important. But yeah, to me, this was really important because it's, when we're considering implementing a punishment-based procedure, then we need to be concerned about what are the potential side effects of doing that. And what I had observed anecdotally had been some really lovely side effects of, I think there's reasonable concern about generalized response suppression. So perhaps you've reduced their unsafe play, but now they're just moping around at recess. Like they leave time out and they just sit there and don't do anything. Which again, we want kids at recess to be running around their friends, playing, engaging in social behavior and physical activity. That's the value of recess. So if timeout produces a kid who is just moping around, like I can't do anything, that would be a problem. And so what I'd seen in the past though was never that. It was kids run right back to play and they figure something else out. And the kid that, you know, in baseline is stomping on the sand castle of his peers, isn't doing that anymore and now has friends. So yeah, so I really wanted to capture that to, and in part to make sure I could trust myself because it's one thing to have these experiences and feel like this is what I think I see happening. And it's another to capture it in the day. I'm so much more convinced now that we have some data. And I'll be even more convinced when we have more data and if other people replicate it. So if other people think this sounds like fun research to do, please, please, I'm happy to help support getting this going somewhere else because replication is essential. I want to be confident that this is a real thing that kids actually improve their social interactions as a function of improving their play at recess. - Both great points. So I want to talk about the choice aspect of this. As I said earlier, I think it's really cool. And it's been a feature of some of your previous work as well. You know, the one that comes to mind is healthier and at all in 2023 where students get to choose between different ways in which, you know, whether or not to play the good behavior game. If so, what variation of scorekeeper did they want? Now I'll put a link to that in the notes if people want to check that out. It's a really fun paper if you're into classroom stuff. But so talk to me about the importance of choice in your research in general and then how it manifested itself in this particular paper. - Yeah, before we get to choice, can we talk about the recess and enjoyment data? - Oh, yes, I'm sorry. - I don't want to lose that. So I think choice is the choice data are more important and weren't longer discussion. I don't want to lose out of it because this, to me, was also a really fun thing that we added. This is the first time we've ever done this. But, you know, as Gabby mentioned, after every single session, we would just ask kids, like, how much fun did you have at recess, basically, on this three-point smiley scale? And in part, we should take these data with a grain of salt. Like recess might have been super fun because my best friend is there and may have been awful because my best friend was absent that day. And may have nothing to do with our contingencies. And these are, you know, preschooler and kindergarten self-report data, so whatever. But, to me, it also was important to know if when we add timeout, are we wrecking recess? Again, like, is the threat of timeout making it so, I can't have a good time. You think I'm having a good time, but I'm not. And the short answer to that is we did not wreck recess. Recess and enjoyment ratings were consistently high or they were more variable in baseline, actually, and consistently high during sessions with timeout. So we didn't wreck recess by adding timeout, at least from a self-reported recess and enjoyment perspective. - Uni, was it difficult to land on this procedure? Did you have others? I mean, the procedure's very cool. So you're indicating by way of three smiley faces, your perception of recess. Was it easy to land on this? Was it a lot of discussion? What contributed to that choice? - Oh, man. I wish Liz was here. She would have a better answer to this because I don't even remember. I think it was just we had discussions about we wanted some sort of data that would speak to this and that it was like, well, what's feasible to do at the end of recess and what might a preschool or be able to accurately respond to? So smiley's and a single selection. We've refined the procedure a little bit. Our Liz has refined the procedure in her dissertation work also related to recess, but where it's group contingency so she's getting these data from everybody. But I'll let we can talk about that at the end when we talk about future directions for this work. But yeah, so I honestly can't remember exactly how we came up with this, but feasibility was essential and we just wanted some sort of data to speak to this 'cause it's also, I mean, we interview kids all the time for social validity stuff too. And I mean, just from a practical perspective, we have a very good time doing that because young children say tremendously wonderful things, sometimes hilarious things that don't make any sense, sometimes insightful things, but oh, I hadn't even considered that perspective. But for this, we really wanted those data every single session and a lot of time to do individual interviews and so this seemed efficient. - Yeah, it reminds me of this tactic that I think is generally described as ecological momentary assessment. So you just have this opportunity at the end of the plate, the recess opportunity to say, how'd you feel about it? And really in clinical psychology and other domains, it mirrors the same procedures that you're using for the preschoolers. So I just thought it was a really nice place, nice way to assess whether or not the children still enjoy their recess experience. - Yeah, that's a good point. I hadn't thought that that was the influence, but I work relatively closely with some of my clinical psych colleagues and some of them use that. And I love that as like this capturing data, even though it's self-report data, it's typically via a smartphone app and it's just how are you feeling right now? And I think that that's such an improvement over the fill out of survey one time and ask over the last two weeks. So there probably has been some influence of my clinical psych colleagues in the development of this assessment. - John, was that called it? Did you say that was an ecological momentary assessment? Do I have that right? - Right, EMA, yeah. And it's used them as Jeanie pointed out in clinical psychology quite frequently and it's just kind of a real quick check-in. And I just think it's so cool that those procedures were applicable in context this day. - Most definitely. - All right, so let's talk about choice making here. So I'll try to reach you the question. But anyway, you've included choice in a lot of the work you've done already. And so I'll ask the question at two levels. Why do you feel that this is an important thing to explore in general and then related to this particular paper? - Yeah, I think there are a few reasons so I think this is important and maybe the first is that the process of having students choose what they experience is not just informative for us at a research level, which I think is tremendously, and I'll talk about that next, but it also gives kids a real ownership over what they experience. And I think of the term shared governance as appropriate for this. So having students included in making decisions about the contingencies governing their behavior is I think there are potentially many benefits to doing that. And so we kind of try to work in opportunities for shared governance whenever possible because students throughout the school day get kind of shuffled around, they mostly don't get to pick what they do and what they experience and they certainly don't usually get to pick what the contingencies governing their behavior are. So aside from just the learning about their preferences, I think having them choose is valuable, a valuable experience for the students. Then the other is that we get really important information from this and information that we can't get otherwise. So when we ask students what they like or which condition is their favorite, sometimes that answer is different than the condition that they select. So we saw this and I'll mention Gents and Chateau's study again and also not one to you. I was a good behavior game study looking at different forms of feedback, asking the students. And so before we had them choose, she had them complete brief interviews. And a bunch of students said that they liked the condition with no feedback the best and they said that it was because they thought it was funny when the other team lost and they didn't know they were gonna lose. These were fifth graders and fifth graders are. - That's on brand, so. - Yeah, brutal. If you spend time in fifth grade classrooms, it's like, oh yes, yes, that's very consistent with fifth graders. But when it came to picking, then very few students ever picked that condition. Like I will actually want, I want the feedback. It's not funny for me to lose. So we get different answers when we ask versus when we have them select. And I think sometimes it's not the case that they're lying to us or it's a different bit of behavior, the what was your favorite condition that you experienced, that kind of historical perspective is a different response than what do I want to happen next? So I think that's important. And it also in this study and in some prior research, it gives us an opportunity to discover whether procedures that we or others as professionals might label as procedures we should not use because they are punishment-based procedures and therefore they are bad. It gives us an opportunity to see do the people who experience them think they are bad and do they work to avoid them? When they have the opportunity to select a different option, do they do that? Do they say no thank you? I don't want the punishment-based procedure or as we found in this study and in other studies, sometimes they pick the procedure that includes punishment. So then we have to rethink what are we applying our own values of what sounds nice or do we care about the values of what our participants or clients are feeling and experiencing? - Yeah, there's a term called, I think it's called a luxury belief, you know, those of us here on the Zoom screen might have the luxury of being able to say, oh, maybe we ought not to use timeout or something like that, whereas it's a very effective and one might even say compassionate way to, you know, in this case maximize enjoyment and participation at recess. So I was gonna ask you what can, you know, someone was concerned about the use of timeout, you know, what would you tell them? And I think you pretty much just answered that question. I don't know if there are other thoughts on that that you'd like to add. - Yeah, well, I have lots of thoughts on that and I have other thoughts on the choice data. Where do you moment go? - Well, let's stay with the choice and we'll return to it that, yeah. - So what we found was that all of the participants selected the DRO plus timeout condition on at least one opportunity. None of them ever selected baseline. Baseline is non-preferred. Then two participants, Austin and Karim selected the DRO plus timeout more often than they selected any other condition. Travis selected the DRO condition more often. And if, for those of you who are looking at the graphs, if you go back and look at the graphs, an interesting thing happened with Travis in that when he selected the DRO condition during the choice phase, he was consistently earning the reinforcer. And that was not true of prior DRO sessions. So, you know, the other participants were selecting DRO plus timeout more consistently and also fairly consistently earning the reinforcer. Travis switched to selecting DRO, but was consistently successful at earning the reinforcer. So I think that's important. I think the likelihood of selecting a particular condition is likely tied to reinforcement opportunities. So I don't think, so in this case, you know, we might say Austin and Karim showed a preference for the condition that included a punishment contingency. But I wanna be really clear that we don't have any data from this or prior studies indicating that anyone has a preference for punishment alone. So in all of the studies where we have data that individuals select a condition that includes punishment, it also includes reinforcement opportunities. And in general, the folks who select the punishment, the contingency that includes a punishment component also attain more reinforcers in that condition. So it's a little more complicated than like people like punishment. I don't think people like punishment, but sometimes we like what punishment affords for us. I'm willing to have this punishment contingency looming because it allows me to function better. It allows me to obtain more reinforcers in the long run. - I appreciate you highlighting Travis's data there. I missed that nuance in my read of the paper. So that is really interesting. - Well, you may have missed it because we didn't highlight it too much because it's really tempting to say that choice did something there, right? That, well, the opportunity to make a choice may have made then the DRO contingency more effective, but we can't say that, we can't conclude that because we don't have an experimental design manipulating DRO no choice, DRO choosing DRO, DRO no choice. So we didn't want to, we didn't want to make too much of that and make people think that choice is doing something special that maybe it is, but we can't. That's for a future study. - Okay, that's very, very cautious of you. That's a nice, all right. So let's talk about people being reluctant to use time out under these circumstances. And I think you, again, you've made a lot of, you guys have made a lot of great points already in this regard in terms of maybe aligning some concerns or fears people might have, but if there are other thoughts on this that you'd like to add before we move on to future directions in this line of research, I want to give you the opportunity. - Yeah, so I guess I first want to say, maybe I've spent too much time aligning concerns about the use of time out or other verses. I think people have good reason to be very concerned about adding a verse of contingencies to other people's lives. And I think that people should be concerned about time out use. I've seen time out misused more often in schools, as Gabby mentioned, we had, one of our participants was sent a time out for lying down in the grass and lost half his recess for it. Like that's ridiculous. We should be concerned about that. So I think being concerned is good. And I think I don't want people to not be concerned when we're considering using punishment procedures. I mean, I think we should also have some level of equal concern with any arrangement, right? So reinforcement is not without its side effects or problems too. So in any case, don't take away from this. Jeez, those use punishment all the time for everything. - Right. - I think be concerned, be thoughtful. But when time out is really a great intervention, tends to be times like recess or free play at home or times where time in is super enriched. It's, you've got access to a bunch of toys, you've got access to your friends, you know, no one is asking you to do anything. So this is a context where we need you to be learning the skills of playing appropriately with decent amount of freedom. And part of learning that is creating some boundaries around what is and is not appropriate free play. And I think time out creates that really nice kind of clear, very clear, it's so clear for the kids, you know, that if I do X, Y, or Z, then I sit out and I lose access to all this fun stuff. And so under those circumstances, then it works really well and we don't have to use it often. So again, when we have an effective punisher, then contact with the punisher should be very rare. And so if you find yourself, if that kid keeps going back to time out over and over, we're doing something wrong. You know, these kids spend a total of six minutes and a span of 332 minutes in time out. This is, you know, it doesn't take much when you've designed it in a context where it makes sense. And those are contexts in which again, I don't think we need to do a functional analysis. It's where our framework is a little different. We're thinking about, we have this really enriched play. We want kids to learn to play appropriately and part of that is creating boundaries around what isn't as not acceptable during play. - I would say, you know, it's not without analysis, even though you didn't do a functional analysis. You know, you know, the preference assessments are all sorts of things in there that are, you know, that seem individualized. And I guess I have kind of a follow-up question relative to that, you know, because of that, you know, I don't know if this is something that you've put thought to or not, but, you know, what I'm reading this paper, I'm taking in the procedures and things like that. And I'm trying to put my school consultant head on and think, okay, let's say tomorrow, I've asked to do a training on playground behavior with, you know, 30 paraprofessionals. And, you know, are there general practices that we can, you know, abstract from your work here, under circumstances where maybe the fidelity and some of those individual components that we just, you know, spend time doing and allying about, you know, are degraded, you know? So, I'm just curious how something like this might scale as it relates to doing staff training and creating kind of like a school-based playground procedure, you know, that may not allow for all the components of this intervention. - Yeah, in part, I love that question and in part, I hate it because-- - I know, I'm totally putting on the spot here. - I think you're not gonna love my answer. And so, and this, in part, my answer is based more on the Liz's dissertation data, which hopefully will submit to Jabba sometime this summer, but where we apply group contingencies at recess, because the reality is these kids were referred to us, but they're not alone in playing unsafely at recess. So, it seems like what teachers and paraprofessionals really need is something that is more scalable for everyone. And the reality is that we did Gabby implemented many of those sessions as did Liz, and it is stressful to manage the behavior of a ton of kids at recess, all at once. And so, I think I'm, in part, I can give suggestions for we saw great behavior change, but it was a ton of effort. I mean, even Liz and Gabby would go for these sessions, implement the sessions, and then report like, I'm so exhausted, but that's not teachers are in the classroom with those kids all day, managing behavior, then they get out to recess, it's chaos. If we are asking them to implement what are at recess, the interventions are more effortful. In the classroom, like you're already, the good behavior game is less effortful, I think, in the classroom, right? So we were embedding this, you're already designed a lesson, we can embed it into the lesson, you're already doing things as a teacher. But at recess, what we find is a lot of times, teachers are kind of taking a break. And I don't mean that, I want to be really careful about how I talk about teacher behavior, because teachers are under, I mean, a tremendous amount of stress, and the burdens we have placed on teachers, and the behavioral challenges that teachers are facing now seem to be greater than ever. And they still have inefficient or insufficient training and managing behavior, so this is not a blame the teacher for not overly supervising recess. But so I think when we're talking about rolling this out for teachers or paraprofessionals, we need to think about what are ways that we can incentivize implementing these kinds of interventions at recess, because just telling them, here's a thing that works, is I think not going to be good enough. Like we're going to need to make playground safety a priority. And by that, I mean, put some money or other perks behind doing a better job of managing recess. Otherwise, I think it's an unfair expectation to ask teachers to implement these effortful interventions at a really chaotic time in the middle of the day. We're in the South. It is hot. These sessions are sweaty. And so I don't have a great answer to how we improve this. I had a conversation with the principal we've been working with who basically said the same thing. We now have data for multiple years, showing that you can improve safety on the playground. I need my teachers doing this. And I'm like, you're going to need to pay them more. Or do you like to say, we can't just tell them do this. It's a lot of work. I see. That's fair, and I guess at the very least, this is something that might be helpful on an-- or it's clearly helpful on an individualized basis. So someone has a referral for a couple of students who are engaging in unsafe behavior. This represents something that an analyst can go to the literature and say, I can draw some inspiration from these procedures. But yeah, that's scaling questions. I mean, that could apply with so many different things that we do as behavior analysts for sure. Not an easy question, so I'll get it. I feel like I answered saying these have no actual applicability. And I don't think that's true, either. I think that-- and maybe this is a place where school psychologists or behavioral analysts working in schools could really be a support for this kind of thing. So it's in recognizing that this is high-effort at a time when teachers are not typically engaging in high-effort activity and in part because they're engaging in higher-effort activity on before and after and how much can we expect them to do. So bringing in other staff in the school who might be able to do these things, we know how to change recessed behavior. We know how to make both at the individual level if you have students who are engaging in particularly high rates and of concern. And we, like I said, forthcoming, we also have some great data on a couple of group contingencies that can really help at the whole grade level out at recess. But the practicality of teachers are going to need some help with this. So I think this is a great spot where a behavior analyst could do a lot of good and prevent injuries. I mean, I mentioned this conversation with the principal. And he's basically like, they have to do this because it's a liability. I'm filling out incident reports. In baseline, he's like, when are you starting intervention? Because I fear recess every day because how many incident reports are going to come across my desk. And so it is important. An administrator see it as important. But I think this is where behavior analysts in schools could really provide some help and support and make a big difference. And I think it's just really important to point out here that-- and I think the listeners will benefit from hearing this. You're attempting that scaling. So you are gathering those data to tell us where they are going to be challenges. But when we opened, we talked about 460,000 ER visits across a four-year period. So I think the matter of scale is one that really requires attention. And I'm so glad that you all are addressing that, pointing to what barriers and challenges exist to produce that upscale. I think is essential to reaching that terminal goal of having this applicable on mass. Yeah, that's a good point, John. So I was going to ask you what's next on the research agenda. And again, I think you already start to answer that question. Are there other things that you and your team are looking at that you haven't mentioned yet? Gabby's doing some really cool work on synchronous schedules of reinforcement. If you want to talk about that, Gabby. Yeah, I can talk about it a little bit. I mean, it wasn't in the playground. It was inside the classroom. And we were looking at also reducing rates of rural violations during independent work periods. So we use synchronous schedules of reinforcement to attempt to reduce those rates of rural violations. And increase in general the percentage of time in which students spent during that session following these rules that we collaborated with the teachers to come up with. And we got really great results. So I'm excited. All right. Well, when both of those studies get published, Gini knows how to get in touch with me. We'll have you back on the show to-- We just got the proofs on that one. That one will be in the next-- I think summer issue of Java, right, John? All right, very cool, very cool. Well, I could sit here and ask-- continue asking questions all morning long. Because there's so many-- again, I failed in my attempt not to fanboy this article utterly. But we need to draw this conversation to a close at some point or other. Are there any other aspects of this paper that you want to mention? Otherwise, we can conclude this installment of the Inside Java series. I guess the only other thing I want to highlight is come back to those choice data. That although there's pass literature indicating that sometimes individuals have a preference for a condition that included a punishment component, I was genuinely shocked that these preschool kids picked the timeout condition. So I think really-- it really challenges our assumptions about what people want to experience. And so I think that part-- kids' selection surprised me consistently. And maybe I need to be being so surprised. Although, I think that's part of the fun of getting these kinds of data. It's surprising, and that's interesting. But really considering removing my own thoughts about the acceptability of procedures and allowing the people who are going to experience them and the people who are going to implement them. But in this case, we were focusing on those who experience and really letting their choices drive our thinking on what's acceptable and what's socially valid to implement. I wonder if that's due to encountering a versus stimuli. It's just kind of an unavoidable part of life. In many respects, quite natural, not necessarily programmed in verses or anything like that. But yeah, maybe we shouldn't be surprised where folks outside of our own very specialized world view would find these things acceptable. Yeah, I think, too, there's the difference between consistency and contingencies and the inconsistencies of baseline. I think there was a kid I worked with in that 2013 timeout study that was working in his home. And so it was like this free play. We would be playing with blocks or trains. Or whatever you wanted to play with. And in baseline, he'd steal toys from me. He'd throw toys. And he just did not enjoy playing with me and would ask me to leave was ready for sessions to be over. Then when we move to the timeout condition, where the contingencies were super clear, if you do X-Wire, if you grab a toy from my hand or you throw a toy, you're going to go to timeout. We then had a ball. He loved playing. He wanted me to stay, would cry when sessions were over. And that was, again, kind of one of those early experiences that thought I didn't capture any of that in the data collection. But this felt like this kid who was largely-- and I spent time with him at school, too, not in a research context-- but I could tell. He was kind of an unhappy kid at school and an unhappy kid at home. And then you put these really clear boundaries in effect and implement them tremendously consistently. And he's so much happier. And it didn't seem to matter that the contingency was a punishment contingency. Again, he experienced it very infrequently because that's kind of the beauty of arranging an effective punishment procedure. So it wasn't like he's going to timeout all the time. He didn't love timeout. Nobody likes timeout. As Gabby said, kids cry in timeout. It is not that timeout is good or fun. It's that I think that consistency in boundaries is really safe and comforting for kids. And so I think sometimes what kids might be selecting when they're selecting a punishment-based procedure. In addition to it, forwarding more opportunities for reinforcers, I think might afford more opportunities for reinforcers that we're not even as aware of as-- not just the prize that we've added, but more natural reinforcers that come with the comfort of, I know exactly what's going to happen when I do X, Y, or Z, and what's going to happen when I do these other things. That is a really insightful observation. And man, we could do 10 podcasts on the importance of setting boundaries and how that can promote overall happiness and reduce anxiety and things like that. But again, I will try to rein in my instinct to continue the conversation along that line where we can have it some other time. I'm so grateful that you guys are doing this work. Supporting behavior analysts in schools, supporting-- and more importantly, the students and the staff who work there is huge, and it's something that I'm hopefully-- I'm hopeful to see more of in our field. So congrats on the paper. Again, folks go to behavioralobservation.com for the show notes. We'll have as many links as we can run down there. I'll have an image of at least figure one, maybe another one or two. And then of course, we'll have the link to the actual data that Genie has generously provided. So thanks so much for joining me, folks. I really appreciate it. Thanks for having us. This was awesome. Love chatting about her. So about our exciting work. Yeah. It's exciting, it is. Thank you for listening to The Behavioral Observations podcast with Matt Sikoria. You can find Matt's notes on this episode at www.behavioralobservations.com. We also invite you to stay connected with us on Facebook at facebook.com/behavioralobservations. And on Twitter @behaviorpodcast. [MUSIC PLAYING]