[music] Hey everybody! Welcome to ABA Inside Track, the podcast that's like reading in your car, but safer. I'm your host Robert Perry Cruze and with me is always are my fabulous co-hosts. Hey Rob, it's Diana and it's me Jackie. The songs are getting longer and longer and longer. The next one is going to be an hour and a half long, it's going to be like *sings* That was like your Delilah. I know. You become like a sh-- what I was going to say a shart cruise, but that's not what it's a shantuz! This is what I was trying to say. But that is what I will be. Okay. It's no longer going to be a behavioral Atlantic podcast. It's just going to be me singing my name. It's not going to be a podcast. So behavioral analysis and behavioral analytic research. It's going to be a podcast about Jackie singing her name for one hour. For a long time. That sounds like it's one of those terrible YouTube clips like a you-a-log kind of thing. Gosh, I'll be a crackling fireplace. One of those, I always want to watch the ones where it's like every time the B and B movie says B, the movie speeds up by 10%. But I want to do one of those for that requires too much knowledge of video editing I think. But as we said, this isn't a podcast at all about any of the things we've been discussing like video editing or singing. It's about behavior analysis. And this week we're going to do something a little bit different than our normal episodes where we pick a topic and then we have some research related to that topic. This one's more of a think piece if you will. I won't. I don't. I don't think. No. It's create. Oh, okay. Well, you will create a think piece. Through audio then. And this is actually going to be part one of two parts because when we started thinking about this think piece and started gathering some articles, it felt like there were two directions that we could take it in. We said, why not do both. So this week's episode is going to be titled from research to practice. And it is going to be all about how practitioners should use research in their clinical work. And it is a lot more than you should read research and then use it in your clinical work. Because that would be the end of this podcast. So it's going to be a little bit more about the pros of being a clinician, the cons of not being in research and sort of looking at how we can take these two delicious flavors and make them taste great together. We're choosing the path not travel. Oh, and that has made all the difference. So heavy. But as for that, the passing there had worn them really about the same. There's a poem that I always thought like I totally understood and then my teacher in high school English told me reminded me of that line of like, really, there's not a difference between the two. And I was thought of them as like, well, that one's, you know, the tough path. And at the end of it, won't you be a great person? And that was the easy path where you get what you want. But you want to learn as many life lessons and it was like, no, they're literally the same freaking road. Do you know how I learned that? And then we can go on with the episodes. I was watching the book club. Did it a movie? Yep, it's a movie from the 90s with older women. That's the joy look club. But I can't remember how it is. It's got Jane fond it in it. And I watched it on the airplane one time and the Jane fond is characters. She's like an older woman that has like multi-million dollars, but she passed out on love. She didn't like, like Mary loved her life said that. And then she's like, you know, they're the same path. And I was like, what? And I had to get it back and read it. So you could all have from watching TV. The Mind Lore. What was the other poem? The pants shared in this movie. No, there was books and wine. Okay. What's the other? The other rubber frost poem that I was thought was about Santa Claus. When I was told it was not, I was like, I don't want to read this poem. I don't know it. Stop it when I would. It's on a snowy evening. Oh, okay. My bad. I would see, we should have used this at the top of the show. But this is an apagaz from the poetry of Robert Frost. We really really blew it there, folks. Sorry, everyone. So part one anyway. I don't know the talk about bog crept in on little cat's feet, but I think that Carl Sandberg. Oh, I don't know. I only know Andy Sandberg. So I'm on a boat. Okay. Okay. I'm ready now. So this week, we'll be talking about from research to practice, which is going to be discussing a lot of think pieces about what is a practitioner is your responsibility to research. And then next week, we'll be talking about from practice to research, in which we'll be sort of doing a flip-flop, though it's not exactly kind of like a one to one point to point correspondence on the topic. So be a little bit different because the titles are the titles we just wanted to flip them. Yeah, nice boomerang if I really wanted everyone to be confused to which episode they were listening to. Yeah. And I also don't want people to turn this off right now to be like, I don't have a responsibility to research. I am too busy. I love research, right? So we're going to talk about those things too. We're going to talk about barriers to being a practitioner and conducting research. We know that it's not easy. This topic really came from a lot of thoughts I've been having as someone who's always been a practitioner. I've been interested in research very slightly, but it's not really been my jam and doing this podcast with two increasingly prolific researchers who've also done their fair share and can you do their fair share of practice? Maybe I felt a little, you know, I felt a little less and I thought, well, you know what? Maybe other people feel like me and they'll want to hear research about those thoughts and feel empowered. And maybe like Diana said, you turned off the episode in which case you won't get to hear any of the following thoughts, but it's your loss because you probably missed that whole Robert Frost poetry discussion. Here we go. Let's talk about the articles. Shall we? First, we'll be talking about evidence-based treatment and practice, new opportunities to bridge clinical research and practice, enhance the knowledge base and improve patient care by casinin. And that's from American psychologist 2008. We'll also be talking about overcoming barriers to applied research, a guide for practitioners by Valentino and Nuaniko from behavior analysis and practice 2020 2020. The evidence-based practice of applied behavior analysis by Slokum Dietrich, Willisinski, Spencer Lewis and Wolf from the behavior analyst 2014. Training practitioners to evaluate evidence about interventions by green from the European Journal of Behavior Analysis 2010. And then very briefly because it's a brief article in a response to a different article, icing on the cake, the role of research in practitioner training by Pitchard and Wine from behavior analysis and practice 2015. That sounds like a delicious appetizer. So great. Pitcher and Wine. No, I say, yeah, but Pitchard sounds like Pringles and Wine just sounds like delicious wine. I'm a pitcher of wine. Yeah. Oh yeah. Oh yeah. Pitcher of wine. With cake. With yummy cake and icing and wine. That's a little too much. Never. Never. So as I already said, I already admitted to everyone out there, I am not a researcher. I am a full practitioner. Though the idea of research has always seemed so tantalizing. I've always felt like I either don't have time or I don't have any good ideas or how would I do research in my setting. And I focused my career mostly on, well, I can use research, but how should I use research? Is there too much research in my practice? Do I go overboard in thinking about practice as a form of research and on and on and on? Lots of fun thoughts. But Jackie Diana, as researcher/practitioners, what are your thoughts on the matter? I think they go really nicely hand in hand, to be honest, being a research practitioner or a practitioner researcher doesn't matter. I like to call them science practitioners. I think that our field is specifically suited for this merge of the two. And I think your best practice will emerge if you think about your practice research wise, right, using evidence-based treatment and looking at the design of your research and align it with how you're going to summarize if your implementation of a treatment is effective or not. So I personally encourage everyone who is a clinician to also be thinking about the problems in your professional life as a researcher, because even if you have to cut some corners, cutting those corners is still going to be better, right? You're going to be a better clinician because you've thought about how you're going to implement your treatment more methodologically, right? And you've tried to figure out all the risks and benefits, the resources, right? You've outlined everything that you need. You've thought about, you know, how are you going to demonstrate that it's effective, right? Classically, right? In clinical work, we rely on that AB design, which doesn't tell us a whole lot, but how then can we jazz that up a tiny bit like an AB AB design, or can we just pop into multiple baseline design if other students are using this same treatment to demonstrate that behavior change is because of your treatment and not anything else. So I think it is a really weird and a really unique situation, I think, in regards to other people, other fields, I guess, because our science is based on scientific principles and it's based on critical thinking skills and skepticism, right? It all aligns to the task list. Spoiler alert. And I think, you know, just reconceptualizing how you think about your job in a way that is more research oriented, could actually improve your job and make you happy overall, right? Because you're not just be like, "Oh, I'm going to slap a DRO on it, bam." There you go. Back it out. Because you all know I hate DROs. I guess what I would say is if you are on the clinician side that there's no reason that you can't also be on the research side, right? And if you are doing research, then you could allow yourself and encourage yourself to be informed by. either your own or someone else you work with, clinical practice. Because if you are predominantly a researcher, that's a good thing. You know how to conduct research in such a way that you can definitively answer a question that's being asked, right? And there's probably talk about this next week, but there are many aspects of research that are kind of silly. Because the whole point is you have to isolate these variables in such a way that everyone can look at it and agree, like, oh yeah, exactly that one thing that you did is the thing that made the difference. But the way that you have to go about doing that is sometimes really slow and clunky and kind of redundant, right? Because you have to be able to prove this scientifically. So if you know how to go through all that process and all of that, you know, heartache of arranging this experiment, then you darn well better ask a good question in going through all of this work and you're going to best be able to ask the questions that actually matter by talking to someone who needs the questions answered. And that's going to be someone who's working clinically and needs to know what's the best way to do the thing that I need to do? It's not going to waste my client's time and it's going to teach them effectively or decrease this dangerous behavior effectively. And if you're not doing that, then you're probably just messing around with questions that don't really matter. Right. And then you can even take that one step further right and talk to the clinician and the client. Yeah, definitely. And answer the questions that the client is concerned about, right? In a research oriented way. Did that answer your question? It wasn't really a question so much as an initial kind of starter discussion case. Okay. Amanda was saying the same thing next week. So if that's the case, I apologize. I'm just hitting it home. It's bad. I mentioned it. We did, you know, it's summer vacation folks. We're just recording one episode and then we're going to play it backwards and that's the flip side. Let's start with the cast in 2008 article, which again was written in American psychologist, but really I think captures at least what I was thinking of when we sort of, you know, first started discussing this topic in terms of there being something of a divide between researchers and practitioners. But again, when I first was saying, hey, let's do this. It was kind of worried like, no, that's just my problem. That's just like my specific beef. I seem to be having in the moment, which sometimes is how these episodes get developed. It was nice to know that at least as of 2008, somebody else in the form of Alan Caston was sort of thinking about the same topic, at least in the idea of just, you know, psychology as a whole, which again, is a big field. So at least in psychology and I'm going to extend that and say, I think we probably have some similar thoughts in terms of the divide between practice and research in behavior analysis. Because again, a lot of folks think, all right, well, what's research? So research is, you know, having a question, answering up by controlling all these variables, and then bam, you come up with the one definitive answer and then practices, you take that definitive answer and now you have to go use that definitive answer and you have to use it as the researchers had it. And what Caston sort of puts out in his article is the issue with that as a thought process is, so you've got your researchers that you practitioners, like why aren't you taking advantage more of research? Because even looking at some of the fields like medicine, the numbers of individuals who will on surveys say like, I use research and making all my decisions. They're low, like 25% I think is, you know, one citation that in one of our articles from tonight. Unless you're my doctor, I specifically asked for research. You have 100%. Let me see that. I'm going to read this article before I tell you you can do this procedure. It's like, man, you're dying. Please, I have to. Where's the, where's the anti article of this article, doctor? I really want to, I want to counterize the extreme bleeding you're going through. I don't know if you're in the right. Then on the flip side, practitioners are saying, well, who cares about your researchers? You're asking dumb questions or you're controlling variables so heavily that there is nothing you're going to learn in your research or your randomized controlled trial that is going to be replicable to me because my patients and my clients are different than the ones you used in the study. Used a bunch of, you know, college students who needed a little extra credit and therefore, this research is irrelevant to me. So we increasingly could set up this divide where you've got two groups of individuals who really need each other to have meaningful practice. And at the end of the day, does what helps clients helps people solves problems socially significant? Socially significant. I was actually thinking about a few years ago when I was at our conference, our annual conference ABAI and how this is actually a discussion that they have that there's too many practitioners at ABAI and not enough research and that the majority of the people at this conference are practitioners of children and adults with autism, back in disorder, right? And how that has been like the dominant research that's been at the conference and they're like, shouldn't you just have your own conference? This has been a fight and that's why Squab was formed too. And now Squab is its own little side biz of ABAI. So because of a Squab, oh. But yeah, so, you know, as you bring it up, at first, I was like, we don't have this problem. But we do. We actually solely have this problem. Thank God I was really worried that I was like, you know what? Crying's my gears podcast on. It's listen to me scream for a while. So good. I'm glad that it's not just me. Nope, it is a problem. Good. It's a problem. And this is also a divide that we'll, you know, we'll talk about in this episode between the idea of evidence-based treatment versus evidence-based practice. Even though some people would use those terms synonymously, really the evidence-based treatment is speaking about like the interventions that have had thorough research behind them and have shown significance in RCTs or have shown significant levels of change in single subject design. Whereas evidence-based practice kind of sounds like the watered down version of, you know, the cool aid of like, yeah, but I want to add in my own opinion on the treatment. I want to add in how I feel that day. And again, I think researchers, you know, especially if they're like really snooty researchers, probably think of practitioners as, you know, throwing in way too much of their own opinions and feelings. Whereas practitioners might say to those snooty researchers, the context of my clients problems are what's really important and you didn't control for that or you controlled it out by having too many participants in your RCT. So again, we can have some really big divides. And if we don't have research and we don't have practitioners, we're really not going to get anywhere in terms of meaningful change. And you know, as behavior analysts, we should have less of that issue, right? Because we should be being objective in our treatment programs and our IEPs, but, you know, that's not always the case. No, because at the end of the day, we are human organisms who are trying to complete these tasks and sometimes issues related to personal vendettas and feelings and opinions, you know, readings of research that's published or that's out there or practices we've used in the past that we've met. Hey, those practices met with reinforcement. Why should I change what I'm doing after all this time, or even the short amount of time, can really run into some problems? So Casinley's out some of the major complaints in terms of, you know, that practitioners say, have to do with, well, this is why I might not look at research too much or why I think my opinion as a practitioner is more important than the research that's out there. And again, it has to do with a lot of the themes we've already talked about. The idea that, well, research uses specific analysis methods. And while that might be great, statistically to look at the efficacy of a treatment, you're looking at, you know, treatment versus no treatment. And you're looking for the statistical significance, which is not necessarily the same as I'm working with an individual who is not able to function. And just because you have a statistical difference in a couple measures that you're using, doesn't mean that it's going to lead to significant change in this real person's life that also, you know, speaks to the idea that the sample size variability within that sample size isn't going to necessarily capture everybody that a practitioner might need, you know, might be working with in that research article that you're reading. A lot of research might use, at least in psychology. This is more problem in psychology uses rating scales that don't have a one-to-one correspondence to actual life changes. So again, we dodged that bullet a little bit in the sense that we're often looking at single subject design. So we are looking at real life variables. So that does make it, make it a lot easier. However, the complaint still could be yes, but some of the measures you used wouldn't be ones that an individual would use with the timescale you're looking at is so much narrower than someone's entire life. You know, you're looking at a couple weeks or a couple months. Black of generalization. Like you said, Dan, they're not paired with all the context of the real world that one might consider important. But again, practitioners, I know, I know, you know, you might be listening to, yeah, those researchers think they're so great and they're every towers. Maybe you're not thinking that again, maybe this is just my beef right now. But let's be reflective. A lot of the issues that practitioners run into are also ones that I think researchers might point out, but that any good practitioner should really be thinking of specifically the idea that because you have a level of expertise and you have a specific judgment that you could make that somehow that is a best guide to what treatment will be best. And while I think we'd all agree that one's own expertise and opinion is important component of the final decision, it is not to be all end all. And that frankly, when you look at how, at least, you know, Kazan speaks to about, you know, psychology and psychology, and I'm going to assume it's pretty similar to behavior analysis. When you look at how individuals sort of make these decisions based on those gut reactions, there's very little reliability in any of their measures. People aren't writing down the systems they're using. They're making changes to what they're doing without actually, you know, documenting those changes. There's no systematic approach. A lot of times it's just why I feel like this way this time or this seems to me like a this kind of a problem. Therefore, I'll pull parts of these treatments out there. It's not a systematic, which means if it's not replicable, there's really a question about is it going to lead to meaningful change? And again, as human organisms, are we just going to kind of keep falling into the same patterns of treatment options? Because this feels like the last time I offered this is. the treatment option. You know, the DRO seems like it'll work because I've had such great response to it before without looking at say, you know, the function base of the treatment as much as I could be. There's also the issue of generalization in practice. Again, what judgment call are you making as a practitioner as to this treatment will work for client A as it would for client B because you feel like it. They look the same. The problem seems the same. It's exactly the same function. There are a lot of other variables and we're not always very systematic or even I think very transparent about what variables we're using in this decision making process. It just feels that way or it seems the same as before. You know, we're looking at pattern recognition, which might not necessarily result in best outcome treatments for clients. A lot of times, at least this is more a process, I think, in the psychological field than necessarily in behavior analysis, though it could also be a problem in behavior analysis, looking at a therapist opinion of how things are going rather than specific evaluation. I know for those of you who are BCBAs working in school systems, you've probably heard a lot of people saying it's working. It's working and you're asking them how you judging it. Well, it just is. There's a sense that everything's working until all of a sudden it's not and then the next day nothing's working even though as far as you can tell with your objective measure, maybe nothing has changed because there's not as much of the same, so I think, culture of database decision making, certainly in education, but I think this could be true for everyone. I don't want to throw education under the bus. I think everyone falls into that trap very frequently. Also, in some cases, again, this is more an issue for psychology, but as behavior analysts who are often working with psychologists, who are often working in a world where we might need to be pulling from psychological treatments for problems we've never, you know, client issues that we've never directly dealt with, there's so much out there. It's very hard to know what is actually meaningful research. What has a research base? I think back to our episode on bullying we did. Matt really came from the idea that, well, what is out there for bullying? And there were 30, 40, you know, all these treatments around the world that people were looking at for bullying. And if you were actually, you know, just a practitioner saying, I need to find a great bullying treatment. Man, good luck. You're going to have to buy every single one and somehow assess them all yourself because otherwise you're just looking at meta-analyses, which are a good start, but they're not quite the same as, ah-ha, this treatment will work for me or for my client, you know, in that case. And spoiler alert, green, who I'll talk about says that we should be teaching students and graduate programs, how to decipher what one of these programs are going to be good and like, what are some things like a checklist and that we should be looking for? Because we don't typically do that, right? We just say, like, oh, look, this one's got research. Yay. There we go. That's some research. Let's do it. And I think that brings us into kind of a good switching point switching into articles, you know, because again, he goes on and he gives some recommendations and we'll come back to some of those, I think, a little bit later. And really just the idea of it's not going to be as simple bridging this divide between improving practitioner practice, improving research practice, bridging that divide between research and practitioner is not going to just be a matter of like, sorry, practitioners, you have to read your research, sorry, researchers, you have to make better research for practitioners to read. It won't be that simple because we're talking about two groups that hopefully want the same thing, which is to come up with the best treatment options for the maximal number of individuals to make the world a better place, you know, our socially significant problem solving. So it's going to need to go a lot beyond that. And a lot of times I think it's going to come to that imperfect solution of, well, you might not always have all the research you need. Your practice might not be running exactly the same way as researchers is, but that doesn't mean we can't think of ways to improve the practice on both ends. And again, we'll be talking, you know, pretty much from the perspective of the practitioner this week. So there's a little preamble to get us started. We're going to take a little break. When we come back, let's talk about some of the other research articles and sort of how they dive a little deeper into some of the components of what is going to make a practitioner do their best work with the research at hand. We'll be right back. Do you want to be a BCBA? Sure. I want to. Now you can come to Regis College in Weston, Mass to get your graduate degree. Choose from any one of these courses, Masters of Science in Applied Behavior Analysis, Masters of Science in Special Education, Dual Degree in Special Ed and ABA, or be eligible for your post-master certificate. You can complete your degree and be ready to sit for the exam in two years, and our 2017 grads had 100% pass rate on the BACB exam. Come enjoy practical placement support, ethics, mini handbooks, PhD level professors, small class sizes, and a service trip to Iceland. If interested, don't delay! Supplies are limited. Learn more at regiscollege.edu. Again, that's www.regiscollege.edu. See you there! And we are back talking about using research practices to improve your quality as a practitioner or as we like to call it from research to practice. But before we get into some more discussion of articles, I wanted to remind you that ABA Inside Track is ace approved. By listening to this episode, you're able to earn one learning credit. Just finish listening to the episode, then go to our website, ABA Inside Track.com/getceuse. That's G-E-T-Hyphen-C-E-U-S. And enter in two secret code words that we've hidden in the episode. The first one is "Weat." W-H-E-A-T. "Weat." When you say that, it makes me think of "Weatin." "Weatin." "Weatin." I don't know who that is. Obviously. "Acto." "Festile from Star Trek." "You can stand by me." "In G." "Don't know." "No." "I watch some Star Trek when I was child." "Original or next generation." "Don't know." "There are two different shows." "But anyway, I'm actually going to interject myself and discuss." "What do you interject in yourself in Jackie?" "Because we hadn't introduced anyone who was going to talk." "Well, you were going to interject. You were going to have Diana talk about Pritchard and Wyin 2015. But I want to interject before she talks about that." "I won't allow it." "You will, because I'm going to do it." "Or in the court." "Order." "Order." "Order in this podcast, Proceedings." I want to make a point here that Pritchard and Wyin are directly addressing another article that was written by Dixon Reed, Smith, Belize and Jackson. And it specifically talked about how we should be ranking graduate programs and maybe using the BACB pass rate isn't enough. And so they suggested that they should look at the research productivity of graduate programs and use that as a ranking measure. And they were sorely disappointed when they found that only 50 percent of behavioral analytic programs have faculty that combined published ten or more articles in the field in the primary articles. So they were a little bit mad about that. And so their discussion was if we are a science-based field, you know, the faculty and behavior analytic department should be producing a lot of research. Right? So that is the crux of why Pritchard and Wyin made their response how they did. So I just wanted to kind of give a little overview of why they wrote this article. "I think that helps a lot." "Yeah, it does." "And understanding what this article is about and the perspective that they're trying to take." So thank you, Jackie. I have more to say on that topic. Sure. Maybe we can tie it into next week's. I think we will. Episode actually. Okay. So thank you for that. So Pritchard and Wyin present the question or try to answer the question, show to our graduate training programs, be preparing students for clinical work or research work and are kind of banding about this question of does faculty research really, is it really indicative of the quality of the practitioners that are going to be produced from that graduate program? And they're arguing that those are likely two separate skill sets. So you could be a really good graduate researcher, right? Or you could be a really good graduate professor teaching clinical skills, but those two are likely not really going to overlap. And even if you are a really good researcher, the chances of that sort of bleeding down into your student's clinical skills is not really that great. And it's not even necessarily known if being a really good researcher is going to bleed down into the research skills of your students either, right? Yeah. This is not a well traversed topic in any way. So everyone is really just conjecturing all over the place to be quite honest, right? They're just like, "But Pritchard Dixon was, Pritchard these guys are as well, right? It's definitely a think piece. It's written in like very loose, conversational kind of way." It was like a half hour. I'm not saying that it was like, but they were like, "I'm going to write something." I'm going to call. Yeah. And then they worked on it. And then, yeah. So, you know, they bring up the question of if we are producing all of these clinicians, but we're saying that our programs need to be based in research. Why are our clinicians not doing more research? And they present some reasons. Why? Which I'm not actually going to go into detail here because I'm going to go into more detail in the Valentino article here in a hot minute. So we'll just leave it as there are significant barriers. Why BCBAs may not be doing research?
even if they are well trained in it. But what they are more largely arguing is that we need to be focused on in our graduate programs ensuring sure that our students are getting experience with research, right? In whatever capacity that is, but that we're putting at the forefront the clinical experience that they're getting during their graduate training so that we're ensuring the quality of our graduate programs through the quality of our clinicians that we're producing. And they say there's a lot of different ways that we could be judging a graduate program on its merit, right? Including the pass rate data, the quality of instruction that's being provided, the selectivity of acceptance into the program, the types and quality of courses that are offered beyond the verified course sequence, as well as whatever thesis project requirements and the product of those projects. What does that look like? And then they, from there, go on to say, let's think about the model that we're presenting in our graduate programs and make a comparison between what we do in our field and preparing clinicians for being behavior analysts to a medical model. Did you love that in the article they were like the Boulder model and then the veil and I was like, why does everyone in Colorado get to name all these models? What about, I know. I was like, what are they talking about right now? The California model and then the, you know, the coniferous forest model. Right. Oh, I was thinking of like different. The Ketalpon model. But I thought that was really funny when I read that I was like, come on Colorado, why do you get to take all the models? I don't feel like that was fully explained. No, and this wouldn't have future. I'm going to look it up though. Okay. But what they talk about is that, you know, in medicine or in a psychologist, you're usually going to school for quite a long time. And you know, part of that time is focused on your course work, but there's also a significant amount of time that's focused on practice or what we would call practicum, right? So a rotating model of having you experience multiple types of practicum placements underneath multiple supervisors in order to get experience with multiple groups of people that you may find yourself working with one day. And that's not something that's very well overseen or regulated within our field. So what their recommendation is is to certainly keep practicum as part of the graduate programs course sequence. And to extend, I mean, of course, this is like maybe a little bit pie in the sky, right? But extend that graduate experience so that there is an inclusion period of maybe an additional third year or they say fourth year spent underneath the tutelage of your graduate structures in order to receive that type of varied experience with varied supervisors. And they say that we're really preparing people to be clinicians, that this is going to be much more valuable for them than getting extensive research training. I don't think they actually said this would be more what you would do in a doctoral program, but for me, that was the unwritten line there. Yeah, they didn't say that. I think they were just trying to make the statement that it doesn't have to just do with research productivity. But we need another way to evaluate the experience that we're giving students, if there are choices to be a clinician, how are we supporting them in providing them numerous experiences and supervision throughout their time? Well, they didn't say, which is what I'm going to say in the Valentino article, is a lot of practitioners indicate that they would really like to be involved in research, right? So if that's the case, then it is also important that we're preparing them in graduate programs so that they can become researchers if some of those barriers are removed. But they didn't say that. No, I really actually love, love, love the addition of this practicum component training where other fields, you know, do two years of coursework and then they have like two years of pre-coursework or post-coursework, right? Where they're doing a practicum at different places. I kind of love that idea. I would assume that that wasn't the development of behavior analysis just in terms of the practice we didn't have enough people. I think the economics started becoming pretty clear of like if we're going to grow this field, well, where is there a need? Well, it's all in treatment for autistic individuals. So all right, well, that's what you're going to learn how to do everybody, even though, you know, that's not what the task list is. Right. A lot of your individual practicums are going to be, you know, in schools where you're working with, you know, autistic individuals, adolescents, you know, preschoolers and everything. Right. I'd go one step farther than Purdue and why and I know that Diana would agree with me that we would have that practicum training in addition to a thesis project, right? So that you have both of those components, right? You can critically read research. You can conduct research even though it's not perfect right because it's never going to be perfect. It never is. Oh, for everyone that does it, you're like, it's okay. You're learning this whole idea. But everyone's like, yeah, I feel you, I feel you, Jack. Right. But the idea, if you're the practitioner, you don't need to write a research that is going to be published. But if you work in a model that mimics the best practices of research, you're probably doing better practice than if you were like, my house do whatever I want. And then I'll look at the data at some point and then, you know, if it looks better than great, whoops, I didn't have a baseline. But it looks low. And everyone seems happy. Social validity is good. But even whoops, I didn't have a baseline is better if you've noticed that you didn't have a baseline, right? That's a story of my life is like, oh, for the baseline, oh, forgot to stagger those baseline. Why did I tell everyone to start that treatment on Monday? I should have said Wednesday. Okay. So that's what they're reviewing there, right? And the threat of that really is, what is our goal in our field? Are we trying to develop clinicians or trying to develop researchers? Ideally, we could be doing some of both, right? But it's always going to be a balance. And I really do think that, you know, things shake themselves out. Some people prefer the clinician route. And so they're going to focus their energies there. Some people prefer the research route. They're going to focus their energies there. But recognizing that it's not that you can't switch, right? But there's not enough time in the day to do a lot of both. So you're going to end up specializing, at least during, you know, at the current time in one or the other and recognizing that you're overall going to benefit by partnering up with someone who's doing more of the other thing so that you can get the benefit of their expertise in enhancing what you're doing. That's what I got to say about that. Sure. I think that also is a good lead into, like you mentioned, you know, and WANACO article, really that discussion of, well, if we don't necessarily need to teach all of our future clinicians about how to conduct research, does that mean that they're not interested in research? And by looking at the survey that Valentino and WANACO did, that's not necessarily the case. Now, bull disclosure, for those of you who are in the New England area, Dr. Valentino did speak at a recent Babbit webinar on this topic. And this is the reason that we added this article to the discussion list for today. This is a super recent article. It's very recent article. And I did go to that webinar though we had this episode in the works. I hate when that happens, but it's like, we're going to hit this episode. We're going to do it next week. And then there's like a, you know, a new publication talking exactly what we're going to talk about. Or there's a webinar on that exact subject. And it's like, well, everyone's going to think we just ripped off the freaking webinar now. Or maybe they won't. And people will think that we just ripped off the webinar. So no, this one section of the discussion today, but I did think it was a relevant talk and a relevant article to the topic we already-- This was my favorite article of the ones we read. I know we're not supposed to pick favorites, but I just did. It wasn't my favorite. My record was my record was my favorite. Okay. That's why we gave you that one. Bam! I first hated it. I was like, oh, because it was all like, thing, thing. But then I like it. The problem with Think Beaces, you know what? You don't appreciate how much your research flies by when you got like seven graphs in your 20 page article, which is, it's just a breeze, but when it is non-stop blocks of text, whoa, wow, those 10 pages, you're really feeling them. Well, you have a little work sometimes. It's like a top 10 list, but even that. Maybe a table of the articles reviewed in this, yeah. Okay. Well, this one did fly by because it actually is a survey. So we got some participants, we got some methods, et cetera. And I'm basically just going to scatter shots, some stats at you guys, and then we'll respond to them. Okay. So Valentino and colleagues did a recent survey of 834 people who identify themselves as behavior analysts, either BCABAs, BCBAs or BCBATs, which is a wildly large number. I thought they got really good. I want to say I saw the survey. I'm like, something like this. Are you like the 1% of people that didn't respond to the survey? No, I always respond to surveys when I get to. I actually never responded to sure. I got to get better at like, out-and-so-so-participant in this study. I mean, I don't remember all can, but I'm like, I'm looking at the questions, like I feel like someone sent me an email that had some of these questions. I do try to respond when I get these types of surveys. They usually don't take them. I don't because I'm like, what if I answer the question, or I get like so nervous. I answer the questions to this opinion survey. I'm like, oh, well, maybe I answer them. I'm having a good confidence, Dave. I'm having a low confidence, Dave. I can't even answer anything. I can't answer anything. What is my name? Have you ever published research? Oh, I don't know. God, I've got horrible research in that journal as published, not sure. Not in sense, Jackie. Actually, don't think about any of my research that way. No. All right, so they got really good response rate for this study. They were all about asking people, do they want to do research, first of all, and then if they aren't able to do research, what's really preventing them?
from doing it, right? So the first question was, where you work, are you provided with research opportunities? And what kind of experiences have you had? So a large percentage of the respondents said that they had had the opportunity to present at conferences, like 79%, 71% said they got research mentoring and supervision, opportunities presented to them. 54% said that they were actively involved in research projects. 50% said they had the opportunity to write, and I find this interesting, only 42% said they had the opportunity to publish. What happened to the other 80%? They made me the work's just still going. They could write a novel about, you know, 18th century London, but not allowed to write relevant research. I just hope that no one is being asked to write something that they're not then included as an author. Oh, yeah. That makes me so mad. That would not be cool. No, they were writing a spec script on their own time for the Simpsons. You know, that's what they were doing. Maybe. And only 28% said they had research groups, which I feel like that's an area that could definitely change, which I'll talk about. Then they asked, is it okay? So you're doing some research. Then they asked, is your company providing you with research time in order to do this research? Only 13% said yes. 86% said no. So that means that a lot of people, if they are able to do these things, are doing them on their own time. Right? So we should already think of that as probably a barrier. And of those 13% that said yes, 37% said less than 10 hours a month are they being provided, which I mean 10 hours that's not too bad. And then a large percentage of them said other, it made these data very hard to interpret, basically. Next question. Do you have access to an IRB or an RRC, which luckily they defined as a research review committee, basically, like an IRB for your company? Right. So you set it up within your company. I don't know exactly how you get it approved. Sure. But someone's looking at the research. Anyway, only 28% of respondents said that even had access to either one of those options. Hot tip, everyone. If you have a university near you, just call them and be like, hey, can we collaborate on research and can we use your IRB? Most of the time, we're going to say yes. We define IRB for some full code. Institutional review board. Yeah. Thank you, Jack. I'm on ours at Regis College in Soiz Rock. I'm on one. And Rob is our community speaker. Are you allowed to say that? Yeah. I can. It's not published somewhere. It's published. Ooh, a published. I mean, it's like on a website, you know? On the web. I know. On the internet, I know. OK. Next question. Does having access to these research opportunities impact your long-term commitment to your job? Now, this is where it gets interesting. 62% said yes, it does. Meaning they want to have access to these research opportunities in order to stay where they are. 15% said no, it doesn't. 22% did not care about this question. I like stop asking me questions. But 60% said, yeah, if I don't get these opportunities, I might look elsewhere. That makes sense, though, right? Because when you're a new BCBA or starting out, you just want to do your job great. And then when you're doing your job great, you're like, I need more, right? And that research may be the key to your more. Well, you're in the field long enough. You start reading articles and you're like, what the-- I had that idea 10 years ago. And I did this five times already based on these principles. I could have published this so long. And you start getting a little sad. You're like, well, why aren't I a big researcher? I have good ideas. My work, good, my clients are happy. I have good data. And you just really all-- you didn't take that extra step. Because you didn't know how or you didn't have time, where you weren't allowed to. Yeah, all these barriers. I think that's a good thing, right? So it says that a large percentage of our colleagues are interested in doing research. And so much so that they want to seek out a place, ideally, that's going to allow them the chance to do this. So I think that that speaks well to the future of research in our field. Kind of like the flip side question, did having access to research opportunities impact choosing your job? In this case, 65% of people said no. 34% said yes. But likely this is related to the fact that almost no places present these types of opportunities to a large degree. They might, if you were a really experienced clinician, they might mention, oh, this is like an extra. But if it's your first job, they're probably not. And we're really looking for people who want to find research opportunities. Most places are just like, we need staff who are good at the basics right now. And then once you get in there, that's when you can create your own research opportunity. Right. So then the article breaks down what the main barriers are to clinicians conducting research, which I feel like this is why I liked it. Right. So it was just like so clear. And they really, I think, hit on some important points. So the first one, which is, I think, a huge barrier is lack of an IRB or an RRC. This is a huge barrier, right? If you want to have published research, it needs to have gone through an IRB. You shouldn't be conducting research at all without even if you're not planning to publish it, right? Without someone looking at over and making sure that your participants are protected. That's really key, right? It's right there in the Belmont report. And a lot of places don't have this. So that, like right there is like complete full stop going to prevent you from doing any type of publishable research. I will attest to this as I had at least two slam dunk research ideas maybe about three or so years ago. And I said them today in it. And I thought she'd be like, that's a great idea. You got to go on that. And she said, well, you're going to need an IRB. And I said, I don't know what that is. And I don't want to know. And that was the end of those research ideas. I think they're on a post it somewhere. They never left the post it stage. That's what disheartened I was by that barrier. Now that I've been a part of the IRB process is not so scary. So, but at the time, it was like, I don't know what the hell that is. Like, who am I going to call? Like you said, Jackie, it's so easy. Doesn't sound easy. When someone mentions that, it's like, whoa, what's that? I don't know what the, where would I find that? There's no way, they won't talk to me. I'm a nobody. I just had an idea. I just had an dream. Oh, well. So there's a couple of ways around that so that you don't end up with crushed dreams, just like Rob. I'm stuck here doing this. Yeah, right. One thing that could occur is like, Jackie, you already stated, you could find a place that does have an IRB and potentially partner up with them. Lots of places may, if you have a good idea, right? And the whole idea of research, at least applied research, is that you're doing something that's going to benefit the clients and then you're going to disseminate that to other people so that they can also now do the thing that is better than the thing that they were doing before. And every study is just one tiny little piece, right? It's just a little piece, a little piece, a little piece, and you're adding and adding and adding. And then someone writes a review. Yeah, exactly. And then you add more. Yeah, but-- The Oroboros of research. So you could find a place to partner up with, right? Or you could create your research review committee, like LeBlanc, Nozick, Peter's daughter, talked about in BAP in 2018, so that everyone is looking over and there's some level of protection in place for any potential participants. Next up, it barrier is lack of time. This is a huge one. For everyone. Right, absolutely. So like I noted up there, chances are, if folks are doing research and their clinician, then that research is probably not written into their job description is probably extra stuff that they're doing on the side. And let's face it, clinicians are very, very, very busy. People doing a lot of work already. So trying to add on research onto that plate, I know that sounds wild, right? But there are a couple things you could do here. One is, if you work at an organization, try to ask them if they could get access for you to journals. That's step number one, right, is being able to access those journals so that you then can read them and think about research. I also like to say if you're a BCBA, you can get the Journal of the Private Behavior Analysis Behavior interventions and another journal. - The Eric Database. - Yeah, the Eric Database, if you're not part of a university, and the Journal of the Experimental Analysis Behavior. For free, under Resources tab in your gateway account. - That's right, great. And then the authors here, they just say, you just gotta make time, right? I know everyone is busy. Everyone says they don't have time. - Everyone's favorite piece of advice. - I know. - Just make some time. - I know. (laughs) - I don't know. This one to me reads like, listen, if you want it, then you better find the time. And if you're not finding the time, then I guess you don't want it bad enough, which is one of those statements that it's like, "Tiger state, this is a hundred percent true. "It is also just, I don't like that as advised. "There's gotta be a better way to phrase that." Well, Diana, you can always wake up before in the morning. - That's right. That is the recommendation in the book and the article I mean, wake up before I am. - I would love to see Diana for a way. - Diana would have murdered everyone, you know. - There's, that we're part of her writing plan. - Diana's not a morning person for everyone involved. Like I say, I'm a fairly good morning person. I wake up fairly early, you know, do a lot of things before Diana wakes up. But she also stays up later than I do. - Yeah. - And make it a point, I always want to call her at like 6am because I've done a ton of things and I want to like talk to her about them. And I always have to be like, look at the clock. We'll Diana kill you. Or will she just not answer the phone because she's such a heavy saber? - Yeah. - I definitely would not answer the phone. - You know, I would be like, stop calling. - I'm already up. I might have left the house already. - Sometimes our group. - You will be able to chat. - A group friend text. - Oh yeah. - Starts going off at 6am like, what do you people do? - Not me. - I am asleep. - I'm so calm. I never do it. Thank you. - Why are we going early in the morning? That's my time. Like I want everyone to leave me alone. - Hashtag my time. - That's right.
- You know, do you want my coffee? - You know, and if you want to be a prolific published author, maybe you do need to wake up before I am, but I can't do that, so no, you don't. - I mean, it really just comes down to the idea, like if this is what you want to do, you're just gonna have to make time. And it's just whether it's just finding time when you're doing notes, like at tack on an extra, you know, 30 minutes there, or just the first thing you do when you get home is like, write down all your data and like start writing it or write up your research notes or whatever it is. You're just gonna have to start doing it at a time and just make it a part of your routine. - Right, you know, the larger advice for writing is build in a block of time where you treat it as a meeting, you're not doing anything else during that time and you dedicate your full attention to writing. The other piece is write something every day, even if it's a little bit, and you'll slowly chip away the thing that you need to work on. Right, so keeping whatever you are trying to work on going as a, you know, living document and just going at it day by day, you will eventually get there, whether you do that at 4am or 11pm, depending, right? So that's the advice there is, you know, try to find the time, but the beautiful thing about being a clinician researcher is, the questions that you wanna ask should be built in to your day to day, 'cause you're trying to ask really applied, meaningful questions, and you know, they talk about this too in the article and saying that the opportunities for research are there, you just have to focus, make sure that you're asking questions in the way, that's gonna allow them to be answered sufficiently through experimental design, right? So there may be some upfront planning that needs to occur, but the questions are at the ready because are the questions that you're asking day to day as you're working with clients, as you're trying to do the best thing across your clinic. So that's all there, it's just, you know, organizing yourself in a way that allows you to ask those questions and there may be times when, you know, a participant doesn't work out, right? You have to drop out a particular data set, but it's okay because you're just eventually going to amass the data that you need in order to publish, it may take a long time. Bury number three was lack of research, mentorship, and opportunity. This, you can reach out to people to ask them to be your mentors, bury your number four, lack of a research community. This may be something that if you're dedicated to it and your organization, you may be the one that starts it, right? Maybe you want to set up a research group. Maybe you want to, again, try to find access to those databases within your community, thinking about how you can do that. And then the other thing I added here was the pandemic and Zoom have shown us how we can really create communities across locations, right? So it doesn't have to be the people that you work with directly that you create a research community with. And maybe the people you went to grad school with and you want to get them back together as a group and find ways to continue doing, encourage one another to promote research. - You can't find a group that wants to sit down and talk about research with you on the regular. You know, start a podcast about it. - That's right. - And there you go. - And you might create yourself a big ol' group. Who knows? All right, so those are their basic takeaways, right? There are barriers, but there are also ways to overcome those barriers. It is a matter of thinking beyond, you know, sometimes you just get so caught in the day to day, right? Like I just have to solve this problem right now. But as you're solving that problem right now, you could also be creating something larger that would allow you to then share how you solve that problem with someone else. So it does require forethought and planning, but it can be done. And that's what they had to say. - Yeah, I do love the point, and I think you made it very nice, Adina, of you do have to find the time to do the writing, but you don't have to find the time to do the quote unquote research because research is just answering questions. And you're always answering questions in your practice. Sometimes the questions are ones that there's already a pretty established answer to, but a lot of times there is not, or there's a partial answer, or there's a question of an extension. You know, will this treatment, this treatment in the research works, but will it work with this group that I'm supporting? I'd like to think it will, so I'm going to give it a try. And if it does, great, that's research that people should know because that speaks to the generalization of the treatment. If it doesn't work, it's probably not gonna get published or anything, but it's still good to have, you know, a research base or a research, you know, component in your practice so that you're able to say, it was not effective. Here's why I don't think it was. And then that becomes research. Maybe someone else has to pick up that question because, you know, as a practitioner, you're not gonna be able to answer the questions about why don't things work because you're getting paid to make things work. But still, it's useful to have that information and to be able to share it with colleagues at the very least. Yeah. This actually ties really well into the article that I'm gonna discuss. And it shifts a little bit by talking about how we need to train practitioners to evaluate evidence about specific interventions, right? So taking that research and pulling it back to how we need to be training students and behavior analysts on how to identify if something is going to be effective or not based on the actual data, right? So this author, Gene Green, suggests that graduate programs are teaching what's on the task list, but they're not necessarily teaching students how to critically think and use the basic scientific principles in their coursework, as well as the attitudes of science. I wrote out my two. I was like, oh, right? 'Cause as a full disclosure, right? We run graduate programs. And I was like, I think I'm doing that, right? But I also think that this was written in 2010 under a different task list, right? And it's also written under a different ethics code. So in this article, the author, Gina, who I love, by the way, hey, Gina, I feel listening. Ties, why we should be teaching practitioners to look at evidence about interventions and ties it to the ethical codes. So what I've done is gone one step further and tied it to our 2022 code. And then she proposes some new content area for a task list. And what I did is look to see whether the fit addition task like which we're all currently in, if you're listening in the future. (laughs) - Might be in the sixth edition task list by the time you listen. - On whether the task list significantly addresses what she says, and if not, how we can address it in graduate programs. And so this is a really nice article if you're looking for resources on skepticism and critical thinking skills. She gives you some good articles. And I have added a few from 2010 on, and I will let you know what those are. But I think, you know, one thing that I think we do really well now is that and the introduction of the ethics code 2022, if you've not read it, introduces the four core principles of behavior analysis, right? And they are benefiting others, treating others with compassion, dignity, respect, behaving with integrity and ensuring competence. I like how it says, ensure their competence, but it's like ensure the BCBA's competence. And this ties in very well with what green proposes we should be doing when we're thinking about interventions, right? Because we should be looking at making sure that the interventions that our clients are using are in the best interest of them, like they have data to support them. We're looking at the risks and the benefits. We're looking at all of the background information about that intervention. And so I think this new code, even though the core principles were already there, but now it's highlighted in that introduction, really solidifies genes suggestion that we should be teaching practitioners how to evaluate evidence about interventions or they think is really neat. And so it's really important that our practitioners can differentiate between good and bad interventions based on data interpretation, right? Because we know there is a lot of stuff out there, right? Parents are gonna come in and be like, I'm doing this, right? A school person might come in and be like, you should do this. I found it on the internet, right? Like, wow, they have flashy pictures. You can change behavior, right? I see tons of those. Like the bad kid program. I don't know if that's a program, but-- Gosh, I hope not. I hope not, but I've seen something like that, right? And so-- - So I have my students learn about the Bologna sandwich. - Right, I couldn't remember what that was called. - Yeah. - Yeah. - It's all the things you need to look out for because if they're in there, it's probably Bologna. (mimics Bologna) So yeah. But then when I started to look, I wanted to see if any of the other ethical principles would relate to how we as practitioners need to be evaluating our interventions. And I just pulled a few, you know, 2.1 is providing effective treatment. We need to prioritize client rights. And we do that by evaluating the data that we've taken on a specific intervention to see if behavior change is effective or not. Right, holistically, not just is this changing in one's situation because that's not helpful unless it's only happening in that situation. So I think that's really helpful. 2.13, selecting, designing and implementing assessments. Here I love this because they added in that assessments must be conceptually consistent with behavioral principles that they're based on scientific evidence and meet the diverse needs, context and resources of the client and stakeholders. So this I think has got a major upgrade and addresses this need to look at the individual as a whole person, right, in their specific context and the data that will be supporting that intervention is conceptually systematic. So I love that one. Same thing with 2.14, they just added in that based in its conceptually systematic. It's looking at the scientific framework. Same thing with 2.15. So all three of those kind of hit, they've upgraded.
by saying they need to be concept police systematic, you need to be looking at the whole individual. And then this one, I love that they had to put it in 2.18, Continual Evvaluation of Intervention. Right. So this one hits so hard to this evaluation of interventions because if we are not doing that, we're not continually evaluating intervention, we're not being a good behavior analyst. And that ties really strongly into responsibility to our clients that we are looking and making sure that we are upholding our clients rights and their dignities and using effective treatments that work. So the authors make a point that it's really important for those that are working with autistic clients to review the research as there's many treatments that have said to be effective with no underlying evidence. We all know what they are, right? Facilitate communication, wrapped in prompting method. I'm going to add social stories in there, right? There are a lot of things that tons of people use that don't really have supporting evidence. So it's really important that we're teaching our graduate students to critically examine the designs that are used and not necessarily just focus on single subject designs because that's what we're going to come in contact all the time. So the point, right? Yeah. I love that, right? So make sure they understand single subject design, because that's on the task list. Right. And what single subject designs are, but also understanding the limits in confounds or group designs, how do you read a group design research? How do you read a qualitative research, right? People would be like qualitative. Oh, but there is some merit there, right? Depending on the question that you're asked. So I think these ideas are not on either the fourth edition task, or the fifth edition task. So this is something that, you know, when I was reading this, got me thinking that I need to do a better job here as looking at group design, because I never want to look at group designs to be fair. I don't want to look at group designs. But I think we need to teach our students how to critically evaluate research outside of our field in order to more accurately assess whether the intervention that someone else is providing for them is effective. Yeah. And it's not just single subject good group design bad. There's times and places for each and teaching them the distinction between the two as well as how to assess quality is really important. Right. So then what I did is green recommended two suggestions to add to a future task list. And they were basic scientific concepts and reasoning, as well as evaluating claims about interventions. So I think under the basic concepts and reasoning, they did include that under A1 through 3, D1, 2, D2, and D4 on the fifth edition task list. But they did not include common myths or how to distinguish between pseudoscience in the task list specifically. So I found three things that if you're looking to read about this, you could read a book by Freeman. It's called The Complete Guide to Autism Treatments, Appearance Handbook. Make sure your child get what works. So that was written in 2007. Autism False Profits was written in 2010 by Paul Offett, he's a doctor. It's a good book. Yep. In Norman 2008 wrote an article in Behavioral Analysis and Practice called Science Skepticism and Applied Behavior Analysis. So you could add those in to supplement your fifth edition task list knowledge to make sure that you are thinking critically and skeptically about the things that you're reading. When we talked about content area, when they she wanted to add evaluating claims about intervention, she said we need to talk about controlled experiments, measurement, evidence of effects, evidence that it's credible, protocols, and sources of reliable evidence. So here, for the controlled experience, we have kind of-- but not really. We have in D3 and D4, they're talking about single subject design and what it is, as well as advantages of single subject design versus group designs. But I don't think it hits totally because they don't talk about what are the advantages of group design. It's not always an or. It could be an an there. So we could do a little bit better. Measurement, I think, we've got covered. Now, it's like all the measurements do this, do this, do this. Describe the advantage between evidence of effects, I think, all of sex and H would be in there, right? Section H is selecting and implementing interventions, specifically recommending intervention goals and strategies based on client preferences that's new, plan for unwanted effects when using reinforcement, extinction and punishment, and monitor client progress. So those are things that I think hit that one. Protocols for designing evidence-based intervention was not there. It's still needed. I found an article by Miller in 2017 that was entitled the use of evaluation and treatment programs for children with autism looking at a global treatment of an agency. And so that might be interesting to look at. And there's nothing in the task list about where to look for reliable evidence, although I'm hoping that in a graduate program, they're telling you not to look on the internet, right? But I thought they are the card 2010 article strategies for making regular contact with the scholarly literature and behavior analysis and practice might be a nice addition if you're not already using that in your coursework. So I think we're doing an OK job based on the recommendations in 2010 to teach practitioners how to evaluate specific interventions, but I do think we might need to do a little better. But again, our task list isn't about just autism, right? So making it expand broader to looking at interventions across all areas. Thank you, Jackie. Thank you. We're getting close to wrap up time. I did want to touch base on Slokeman All's 2014 article. Now, if you have the article in hand, or you're one of those folks who's like, I'm going to read the articles before I listen to the discussion, we're not really going to go in in depth, both because A, this podcast is starting to run a little bit long. And B, overall, the article is really response to Smith at All article from 2013, sort of discussing the definition of empirically supported treatments versus evidence-based practice and sort of trying to tighten up the definition. And there were a lot of agreements between the two articles, some disagreements, but the components of the Slokeman All article that I thought were relevant to this discussion was the more global defining of what should be part of an evidence-based practice. Because as clinicians and practitioners, a lot of what Slokeman colleagues are talking about goes into what are the decisions that a clinician must make in regards to contacting the literature that goes beyond what we all would like to be, which is, man, my client is having problem X. Let me check Java. Aha, 10 treatments covering the whole gamut of issues I might run into that show the super effectiveness of treatment A across a wide range of participants from different backgrounds. And here are some modifications to speed up the process and get it done in a week. Hooray, I'll implement this treatment right away. We'd all love to say that's probably what our evidence-based practice. We would love it to look like that. But for many of us, it doesn't look like that. Usually what it looks like is I have to solve a problem or client. My client X is coming to me with a problem, problem A. And I need to look in the research. Uh-oh, the only articles that have any treatment, even close to related to this problem are not even in a behavior analytic journal. Or there's a brief mention of it in a webinar I went to years ago and I took some notes somewhere. And that's my evidence-based. So I guess I have to call my client and say, there is no empirically supported treatment. Sorry, nothing I can do for you, but bye. And while there certainly might be times that you would say, I'm going to recommend you go to a different practitioner, because it's a problem outside my scope of practice, per se. We're not talking about change in scope of practice. We're talking about solutions and treatments that could be a part of behavior analysis and just haven't been touched for whatever reason. Or they haven't been touched by researchers in our field. Or there are only RCTs describing some treatment options, which may or may not be helpful. I know my big problem, jaguar group designs, when I try to read the research, they spend so much time on the statistics. I'm like, but what did you actually do? Please tell me, because I'd like to try it myself. I can't replicate your statistics. This little dig, this little dig on RCTs right there. So what does it mean to talk about evidence-based practice? And I think Slokum and colleagues really want to think of evidence-based practice as a framework. There's different than evidence-based treatments, which are going to be talking about treatments that have robust studies behind their efficacy. And evidence-based practice is really a framework that supports the work that we need to do that includes research evidence as well as clinical expertise and very important client values and context. I know we talked to the beginning about Kazzin and All, talking about how many practitioners in psychology want to use their opinion. He certainly didn't not talk about client values and context, but Slokum and All come back to the point of really paying attention to the values and the context in which our clients are living as a component of evidence-based practice in a way that I think we always mention and we talk about and a lot of the research talks about, but I don't always think of it as part of my practice is what my client values are, which we really should. He and his colleagues really get into the idea of sometimes we're going to be looking at the best available evidence. It's not going to be as easy as I'm just going to look at Java, I'm just going to look at behavioral interventions and BAP, and then I should have all the answers I need. is that we really. really are going to need to start asking ourselves what is going to be best available evidence. Sometimes best available evidence will be multiple published studies in multiple design formats published across multiple journals with multiple participants. And in those cases, we really should be doing pretty thorough analyses of the research that is available to make sure that we can tailor what's in there to replicate or extend to our specific client and their specific issues. We're really looking at a very narrow evidence base. And in these cases, we're really going to have to use a lot more of our own clinical judgment. And this is where the idea of thinking about yourself as a researcher solving problems can come into play because you don't have as much research to base your claim on. So you're going to have to look at what research is available. You're going to have to look outside your comfort zone of research as company, you know, used to reading. So again, maybe you're not going to be looking at single subject designs. And then think about what are the contexts that you would use some of these components. And then again, how are you going to set up your treatment so that you can make sure that the variables that are important to the client or the variables you are studying, the variables that you are able to make judgments as to whether the treatment that there's some research on it has any efficacy for your client. I think of some, you know, issues I've run into. And you know, in the question comes up a lot about generalization and in the school system, things got to have an assessment. And usually it's an assessment where you sit down with a student and you say, here, here are a bunch of questions, answer these questions for me. And then I will tell you something about yourself as a learner or your ability to write or your ability to have cognitive thoughts or your executive functioning skills. And when people ask me as a behavioral analyst, okay, let's do do that functional behavior assessment. I think they assume I will ask a number of questions and then say, aha, it's a scape maintained behavior. Here's the solution. And when you try to explain the process, you get a lot of glazed looks like why are you trying to make this harder than needs to be just give them the questionnaire? Isn't that what testing should be, right? It's like, it's a piece of paper. And I'll get a lot of questions about generalization. You need to do a test of generalization. Like that's a thing I'm just going to go, I can go botch, need like 50 bucks to go by the testing booklet. And it was a problem where I had many families asking for this. And instead of saying, that's not a thing. You usually test it as part of, you know, a treatment evaluation. I said, you know what? Fine. I'm going to look at what research is out there. And I'm going to put together a protocol. Is it the best protocol that's ever been written? No, because there hasn't been a protocol that's been completely written as to like what that assessment would look like. But I took behavioral principles. I took the research that existed. I took my own understanding of how generalization is typically tested for in research. And I put something together that I knew could be controlled. I looked at Barry, you know, light variables because I knew I wasn't basing this on elaborate research on like an assessment tool. But at the end of the day, I was able to use what I knew of research and what I knew of research design to put together something that was able to teleclient. Curious some information that you will want to look at and tell an educational team, here are some areas that you might want to put new educational programming in. And it was one of those, one of those areas that I think if I've been newer in my career or if I had been stuck on the idea of everything being evidence-based, practice being just based on empirically-sported treatments, I would have just had to shrug and say, "There's nothing I can do. I'm sorry. I can't tell you anything. I don't know what to do." Or I would have just written a fluff piece report, which I hate doing because I think it is a waste of everybody's time to be like, "Look at me. I can write a report because nobody wants that. They want help. They want you to help them solve a problem." So again, that's a lot of what evidence-based practice is going to be. And so, it goes into some areas and colleagues go into some areas, you know, talking about practice guides and how you can get research in ways that are not just a research article with a single subject design, how when we look at analyzing effectiveness, sometimes it goes beyond just looking at your ABAB graph, your multiple baseline graphs, kind of, you know, the unit of measurement being change in DV that's relevant to your client also goes into, you know, more interview with the client working with your clients. And really, how can you put all of these pieces together? So it's a lot of understanding what's out there in the research. That's definitely going to be a component of it. But it's also understanding what's not out in the research. What you yourself can look for and what have you seen in practice, in talks, in practice guidelines that you might have read in the past, which again, probably are based on research and expertise, but aren't necessarily going to cite all of their research or be transparent about what research was used and sort of, here is the great idea that this smart group of people had for you. And that just, you know, thinking about treatment as well, I'll just find the thing that has the most research and is written in a guideline format for me that I can easily follow is probably doing yourself as a practitioner and definitely doing your client as an individual to service because you can't just manualize every single treatment. You can manualize components of treatments or you can manualize specific treatments for specific issues for very specific individuals. But then you're talking about really narrowing down what you're going to be able to use that manual for so that it almost becomes ineffective in terms of I can use this in my practice. Unless you're like, I only treat people who have this one problem and they have to be exactly like the people in this manual, in which case I'm guessing you're not going to stand business very long. Maybe, I don't know. Maybe it's a good manual, but I highly doubt it. The best manual. The best manual. So thinking about what's best available evidence and best available evidence, not always being a huge quantity of evidence, thinking about your values of your client, the context in which their problems are occurring that will relate to what's the generality of the research that's out there. Even if it's great research, it may not be the research that you need. And then also being able to speak to your own expertise. I know about this problem because I've read about it. I've been to trainings about this. I worked on this with my supervisor. I reached out and got a mentor in this area and really putting it all together, which again, you read an article like this and you think about your practice and you're probably saying, oh, I don't do all this all the time. But again, you probably do a lot of it. It's just always asking yourself those questions. You know, am I using all of these components of being a practitioner? I'm not just counting on the research. I'm not just counting on my own opinion and my expertise. I'm not just listening to my client because again, that is 100% important that we care about what our clients are trying to solve and they care, they like or don't like our treatments. But again, that can't be the whole picture that we have to put it all together as as practitioner. And then again, what do we not know that we need to learn in the future? So again, I think a nice article, again, if you're reading it, you might be able to skim some of it because some of it's referring specifically to a totally different article. But those are the pieces I thought were most relevant to our discussion or tonight. And I guess with that out of the way, we should move into dissemination station, wrap up. Okay. So when it comes to the kind of the thesis of this whole episode, I think we've talked about some other lists in terms of as a practitioner, how are we using research to improve our practice? And you know, I was really hoping that the answer was going to be just read more research. And it was a more interesting episode. I think the fact that it wasn't just like you just need to contact the literature more and then everything will be fine. Right. Because it's not that simple. But I mean, that also could be a dissemination statement point, valid point if you are not contacting lists. Yes. Right. So you should be encouraging your coworkers to contact the literature and providing a space for them during their job day, right? Not like nighttime. Right. But during their job day to have time and space to critically think about their programs and to have that time to look at the research. I've had people say, I've had practitioners in, you know, or colleagues I've known or people I've known that I've talked about like, oh, I booked out some time because I will book out time. Like I'll call it office time. Like I have to read research so I can develop a treatment because I don't know enough about this topic. And I've had people say, oh, man, I've been so much trouble. If someone saw me just sitting there and reading to which I say, what do you, like I really hope you've only ever had to solve one problem in this job because you read about it in grad school and you've just been able to do the same thing over and over because if you're not able to even not contacting literature and, you know, maybe you're waking up at 4 a.m. and that's your journal reading time. But I don't know about you, but about six o'clock I can't do too much productive work. I know some people are flipping, you know, flip it around. You know, I like to get up early and go, but you're not going to be able to contact the literature if you're not going to do it at least in some capacity during your actual job out. Right. And your job hours may be different, right? But your job is just your job and your life should just be your life. Yep. So yes. So that is a good point. That is a number one. You do have to contact research. So again, putting in many of you were probably saying, but I am contact research or or oh, I listen to your podcast every week. Oh, thank you so much. So I am at least contacting a couple articles on a pretty regular basis. And that's great. We appreciate that. And we'll be contacting other literature because we can only do so many episodes. There's only so many podcasts out there to listen to some other pieces I think that are important or thinking about what is your evidence based practice? Are you using systematic measures to evaluate the progress of your clients and not just in the sense of I have one DV and I'm looking at that one variable and that's the thing I'm measuring. But how are you measuring it? Are you measuring it in ways that might replicate reversal design or withdrawal design or multiple baseline design? You know, are you thinking about some of your work with clients in ways that that might be a great addition? Either because it's relevant to a question that the client has or stakeholder has regarding your treatment or because you're working on a treatment
that there's just not a lot of literature out there and maybe you are, you are asking, could this be a relevant question that is going to be bigger than just my one client in this one situation that I might want to have so that other people could replicate this and get more information that I could share with other researchers or other practitioners because other people might have the same problem someday too and if all I ever did was look at social validity measure for a poorly controlled treatment that I kind of cobbled together from what research I could find that's better than nothing but that's also not taking advantage of the technology that we have and the knowledge and the skills that we all have even if we're not in a research lab every day. Yeah, I think that's our take home is find ways to do it be creative maybe just switch your mindset and think about your regular job as informing your research and forming your practice. Stay tuned for part two. Alright, well that brings us to the end of the episode. We hope you enjoyed it. This one was a little bit different than some of our regular episodes. We just sort of hit up a topic, hit up some research. I think this one required a little bit more soul searching. So we hope that you found it interesting. Hey, whether you did or you didn't, we would love to hear from you and there are a lot of ways that you can share your thoughts. ABAinsidetrack.com or again, you can leave comments. You can find other episodes that we've done so you have a comparison and also where you can purchase CEs for episodes listened to. You can find us on social media as ABA Insider track, Ron Facebook, Twitter, Pinterest and Instagram or you can leave us any sorts of thoughts on any of the episodes that we do. If you're saying you know what I did love this content and I love some more, you could also check us out on patreon patreon.com/ABAinsidetrack where for five dollars a month you can get episodes a week ahead of time as well as access to some other bonuses like five monthly social meetup and well as discount to the CEs store and where if you're interested in even more content, maybe the book club variety, you can subscribe to the $10 level to get access to our quarterly book club podcast. We actually should have our newest one out. If not right when you're getting this, maybe in a few days should actually be coming out. There'll be a preview on the main feed as well. That's patreon.com/ABAinsidetrack or you know what? If you just want to shoot us a line about, hey, this episode was fill in the blank, ABAinside
[email protected] will always work for us. And finally, I want to make sure that if you're interested in getting CEs for this episode, you get the second secret code word, which is fog F.O.G. The fog is getting thicker and Leon is getting larger. Remember that? I do that one is. No. I don't either. I don't worry. No. All right. Write in at
[email protected]. Should we give it free C.E. if someone actually remembers what that quote is from? No. Everyone's checking their head. No. You can't get it free C. I will be impressed. If you get that quote, it's pretty obscure. Fog. Thanks, Jackie and Diana for joining me on this jazz odyssey of research discussion. You're welcome. Thanks also to Dr. Jim Carr for recording our intro and outro music, Kyle Sturry for Interstitial Music, the entabit of the podcast doctors for his editing and also everyone from the Sikomar Workshop for his visual design. We'll be back next week with another full-length episode of Flippin' the Script on From a Practice to Research. But until then, keep responding. Bye. Bye. Bye. Bye.
PodcastTranscript AI