Regression to the Mean Girls | Mixed Effects, Mixed Signals: Love Island Kissing Scores with Hannah Barton

This is Regression to the Mean Girls, where we chat with brilliant data people about their silliest sidequest. We are mean as an average, but actually very nice. Come sit with us. I'm Lucy Dagostino McGowan, and I'm Sarah Lotz Beach. And today we're really excited to be chatting with Hannah Barton, who is a PhD student at the University of Washington. And I believe, so I saw that your background is in speech language pathology, right? Mm-hmm. Yeah. So what is your PhD in? The same or something related? So the program is speech and hearing sciences. I mostly work on language development and neurodevelopmental disabilities. So specifically right now I'm very much in the autism world, but I've also worked on like fragile X syndrome, fetal alcohol spectrum, things like that. That's incredible. Thank you so much for being here today with us, Hannah. Yeah. Thanks for having me. So to kick off, we usually break the ice with what we call a media minute, but tends to skew more towards a media many minutes if we really get going. So we prompt this as, what kind of media are you enjoying right now? Books, TV, movies. Hi, since this is our first recording of the fall, I'm going to talk about, there's a new season of Ted Lasso. I don't know if anybody watches Ted Lasso, but I really thought we'd close the door on that a couple of years ago and now we're getting a new season, now it's a women's team. And I am enjoying the new, the new Ted Lasso. It's one of the only things I watch probably that is actually current and comes out weekly and I have to have some sort of control. Yeah. We are also watching Ted Lasso in my household and I also thought it was going to be done years ago, but we're hanging on. So we'll see how it carries on. If you're a fan, I recommend if you're ever in London, there's a Ted Lasso tour of Richmond. There's like one lady because, yeah, apparently for whatever reason, Ted Lasso never really caught on in the UK like it did in the US. And so there's this one lady. She is the official Ted Lasso tour of Richmond and she takes you to all the filming locations and also Richmond's just like very, like very ritzy, very nice. We toured the, or we had a drink at the pub that floods if you've ever seen that like on social media, there's like a pub right on the river towns that when the tides come up, it floods. So I highly endorse the official Ted Lasso tour of Richmond. I can go next and keep it on the Women's Sports theme. I've been watching the WNB in playoffs, which are good fun and as a Seattle storm fan, I am not invested in it at all now because we are very good this year. So I'm just having a fun time watching everyone and hoping everyone has fun. That's awesome. I'm the first person to use sports as a media minute, which is a surprising. There we go. Maybe it was our timing before because we recorded over the summer. Maybe there were fewer sports opportunities in the summer. But that's awesome. But you did like March, which had like March Madness, like some people, even I know about that. So I think it's just our, I think it's selection bias but good to hand it in here. So we have sports media. I am going to, so over the summer, my brother and I read Children of Time together, which is a sci-fi book, which sort of got me back on I historically have done a good amount of sci-fi reading and a lot of like, I really, I went through a phase like two years ago where I read all the Isaac Asimov short stories. And anyway, so this got me like on recently, my, my brother also suggested that I read some Philip K. Dick, which I had not actually ever read. So I read Do Android Stream of Electric Sheep, which is like the book that Blade Runner is based on, although I will say I loved the book. I had not seen the movie, which is surprising, because it feels like it's one that I would have seen and I watched it after and I like loved the, the look of the movie. Like I liked, how I like that vibe, there's like steam punky, I don't know, I liked that. But I liked the books, like the movie really changes like what the meaning is, which I was interesting. And then I read the minority report, which I had already seen that movie, but anyways, and then, and then I started a scanner darkly, which was too dark for me. So I had to stop. So I'm now looking for new sci-fi adjacent short stories or novels if anyone has recommendations in our listenership. I mean, not sci-fi related, but I did realize, chatting with Hannah before you joined, that one of the only books that I read that you might have had even the slightest bit of interest in is like kind of related to today's topic, but it's like a, I think, is it technically satire? Have you read it, Hannah? The compound? Yeah, yeah. So I think it's satire that blends like animal farm and present day. So it's certainly not sci-fi because unfortunately these are real people and I suppose in a way these things could happen. But as I said it, I was like, that's one of the few books that I've read that Lucy might like. So it's an energy to sci-fi, yeah. Yeah. Mm-hmm. I can see why you would think I would like this. What is the way of saying that you would not like it? It's a way of saying that you would not like it. Yeah. It does actually say it won good read, choice in science fiction. So I think someone would consider it science fiction, but I don't know, and I'm just looking on. Not me. But cool. Yeah. Nice. Great. Well, this was a successful medium in it. Last, so our last several episodes which we kind of recorded in succession a semester ago, I was really stuck in a loop of only consuming the same media. So good thing that we can now do a more diverse, well, thank you here again for joining us. Do you want to give us some context for your deep dive? Yeah. Absolutely. So I'll give my personal context and then I'll say what love island is and all of that. So this past spring, I was finishing up my dissertation and for that, I'm using this model called the social relations model that we're going to talk a whole bunch about because it's what I used for this. And then I was doing that and I thought, wow, I don't want to think about any real data or any language samples or what could say for like a little bit. And we were coming up on summer and I thought, oh my gosh, love islands on. What can I do with love island and I thought back to this fun challenge that we're going to talk about today where people kiss each other and it just occurred that this is like the perfect data set to do this kind of analysis on. So this whole thing was really done on just feeling like, I should probably start figuring out how I'm actually going to use this model and I don't want to do it in a fun way that is not actually thinking about real data or all of the complications that come with dealing with real people and all of the things that they say. So that's kind of the impetus for what I started doing. And basically, this whole thing centers around the show love island, which I think, I don't know, you guys know a little bit about it, but I can explain kind of the whole thing. So it's essentially this reality show. There's versions originally in the UK, there's one in the US doing this. I learned there's one in Malta, but I've never seen it. So that exists too, which is great. But all of these shows basically they have a fancy villa and they dump like 10 to 12 really hot single people. And then they say, you guys couple up, it's like five men and five women and they couple up. And then about every week or so, they say, now you re couple like find a new person, pick the same person, maybe the women pick, maybe the men pick, maybe the public votes on who's couple a and usually as part of that, people get voted out of the villa or they get dumped from the villa or various ways. And I think the key thing about this show is it airs six days a week. So there's so much content and you really are just like you're in it with them and also no one's watching it that carefully because it's at least six hours of content a week. It's too much content. So you're just getting so much in throughout each week before the eliminations or anything like that. They're doing various challenges. So like a lot of times they have ones where they all like crawl through slime and then sometimes they like baby bird feed each other like catch up, which is just absurd to say. I'm sure there's other ones, but those are the ones that really stand out over the course of a season. And then they do all kinds of like diabolical things where like there's a dumping and they just make the most popular three couples choose a couple to dump. And they're just like send one of your friends away. So there's all kinds of wacky interpersonal dynamics going on. And the last element is that throughout the show, which goes on like all summer. So this is like six episodes a week every week all summer. They're sending in more people. So they're dumping people from the villa, but then there's new people coming in who are called bombshells and those are like additional hot people that they can then couple up with or flirt with or whatever. So that's kind of the premise of love Island on the whole, but I specifically looked at as one challenge that's called the kissing challenge or think sometimes it's called kissing booths. I don't know. And they do this on a lot of the seasons. There's many, many, many iterations of this challenge. And it's essentially like either all the men or all the women become like the kissies or the people who are going to be kissed and all of those people put on blindfolds and headphones. So they can't see anything and they line up. And then the other gender goes one at a time and they just kiss everyone. And then the ones in blindfolds give a score and they note it down. Oh, this is amazing. And it's a perfect data set. If you're having trouble visualizing this, I do have some pictures of it happening. Please. context. Yes. Let me share these and And we have like a little blog that you'll be able to populate. And so we'll get these pictures so that the audience can enjoy. >> No, it's going to make it not a thing, and I unfortunately don't have a double thing. So we're just going to look at it in good old drive. So this is like all the men lined up with their blindfolds and their headphones on. >> Okay. >> And then this is what it looks like when they're actually kissing, which it's weird to watch. As an audience member, you're kind of like, this is not something I should be watching. But you watch it anyway, or you do your chores while you're doing it. And then they end up with this lovely table that they fill out in the moment of each score that each person has given each other. >> Do they score, do they do it like right after they say their score, or do they have to like remember 5, 4, 3, 2, 1 or whatever? >> I was re-watching this just to make sure that I knew all of the weird things that happened. And I think the way it works is like one man goes down the whole line of women. And then all the women take their headphones off and give their scores for that man. Which does sometimes, like there's a few instances where the early on women say something in giving their score. That's the way to influence. >> Yeah. >> Yeah, so that does happen. >> Love. >> And it's super mixed in terms of what happened. So like there was one where someone said something and then someone down the line was like, wait, that's not who I thought it was. Like, and then it changed everything about how they were scoring and what they were doing. So it's good and silly. >> A messy data. >> A messy data. >> A messy data. >> Yes. >> Yeah. And the other messy thing that they do is like sometimes you'll have people who say like, I'm having an interpersonal conflict with this person. I'm not going to kiss them or I'm going to kiss them really poorly or I'm going to kiss this person really nicely because I'm interested in them. And then sometimes the show will throw an extra element in. So one of the seasons we're going to talk about today instead of just having all the men who were already there kissing all the women. They actually took away the last man's turn and they brought in a new bombshell man and had him kiss all the women. >> Like a brando. >> Yeah. So there's all kinds of like fun wacky elements to this kind of data set that make it just like perfect for this kind of analysis. And I think you could, I mean, you could do so many other kinds of analysis on this. But for me personally, I was looking at diatic interactions. And this is like the king of all diatic interactions. >> Yes. Can you, okay, can you explain to us what diatic interactions are in just? >> Absolutely. Yeah. So basically it's just anything where two people are talking, interacting, even like just looking at pictures of each other and judging them. It's basically just the idea that like when two people or a person and some stuff are interacting with each other. There are multiple things at play. It's not just like me talking into a void. You guys being there also impacts how I talk. And then what you say influences my speech and on and on and on. >> Nice. Okay. That makes sense. I can get back. So these are, this is like some kind of pairing due to differences in both maybe the estimates themselves and how correlated they would be expected to be based on the fact that some of them. Like if you have multiple estimates across the same people that are, yeah, I think I understand. >> Yeah, yeah. >> So can I ask, so I definitely can see like, is it diatic? Is that how he said it? >> Yes, yeah. >> So diatic interactions, is that kind of like the down the line where like the first person's response might inform the grade that like a later, no, it's the kissing? >> Yeah, so it's specifically the two people kissing are having a diatic interaction. >> So is it because the other is one sided that it doesn't count? Because it feels like if like I inform this answer and Lucy informs my answer, then it would also be a diatic interaction or no? >> I think you could frame it that way. I think the problem with that is then like I would frame that more as an order effect. Then a specific like diatic interaction because you just end up with like exponential diatic interactions if you consider that like the first person is then impacting six more people down the line. Which impacts the second person and then that impacts four more people that are like whatever you want to call it. So yeah, that part, I don't have modeled at all and I have no idea how to go about modeling it because it's just there's so many different things happening. But there's there is one aspect of the social relations model that captures some of that messiness, but it's not, it's not great. >> Cool, thank you. >> Yeah. All right. Shall we jump into some analysis? >> Yes, please. >> A rabbit hole. >> Great. >> I should have also said that the scores are out of 10, so that's kind of the basis of how they're being scored. Okay, so the social relations model is both like a theoretical or like a way of approaching these kinds of diatic interactions or just like social perception in general. So as far as I can tell it's big in the like intergroup psychology and like perception of others world. And it's basically the idea that when you have these kinds of diatic interactions where anyone is talking to someone else giving each other a score, those kinds of things, you can separate that score and the variance in that score into the actor component, which is like the person giving the score. The partner component, which is the person being given a score about or the person being interacted with, and then a relationship component, which is like the specific special spark that two people might have and then you have the error component. So it's basically this idea that you can divide up the variance that way and there's a bunch of different ways that people go about getting those estimates depending on what the design is and all kinds of things. Most people, I think are using essentially multi level modeling and using both like the actor and partner as a random effects and then adding and other predictors as fixed effects, but it seems pretty variable and then there's one really good simple package on our called triple R that runs. I think it's like the method of moments, estimation, yeah, and so they they use that, which is really nice and easy, but it's very specifically for only perfectly round robin data, which is where everyone's interacting with everybody, and there's not really any missing data and it's all wonderful and good, which is not a reality in most of most of the time. So that's kind of a broad overview of the social relations model, can I ask the question that's so it makes so much sense to person I've heard of these models I'm so excited we learn so many different ways that we can use like the same statistical tools for different things. So in the most common case that you said was spending like a multi level model, so we're going to have like a random effect for the person the kissy and a random effect for the kisser. The part that I'm trying to disentangle my mind right now is you said that we can separate out the kissy effect the kisser effect that makes sense to me because we have repeated measures for each kisser and then we've got an air term also makes sense to me. The parts that I'm struggling with is how do you how would you quantify the kissy kisser interaction when like because you only would have one measure per right or am I right. The answer is you can't in this kind of case you can so the theoretical construct is that that exists separately. In this case where there's only one score. You can't do it so you just get this big giant chunk of relationship and error so if you look. Yeah, this is actually just a breakdown so this is treating it just as a round robin so this is the first thing I did is just like let's pretend it's perfect wonderful data. Let's just look at the components and that relationship variance is huge it's like 70% of the variance, but that includes error variance and there's just no way to separate that as far as I know. Which was a big I looked at this and said man it's a great thing I'm planning to do multiple measures in my dissertation so I can. We can do this in tangle yeah this is so cool though and like I do think also you you couldn't disentangle it but you could think through magnitude and like this to me and I could be thinking about this very naively but I'm looking at so we'll put these again in the show notes for our listeners but we've got a kind of small sliver of actor variance of maybe like three or four times that sliver of partner variance and then I would say like 75% of the space is taken up by this relationship variance and if I'm just like intuitively looking at that. I would I would not expect the air to be like I would expect a good chunk of that to actually be relationship variance due to the magnitude of the actor and partner variance because once we've like accounted for your individual differences and your partners individual differences like what would would there be what would like conceptually what would this other air be if not the relationship you know what I mean like and so I do think we can make some kind of like at least magnitude type of discussions as a maybe. not absolute discussions about the relationship variance size, but we could sort of say that it's probably big. But would that be where the ordering currently falls? Like if we think there's that ordering effect of like your scoring being informed by people who answered before you? Because I was trying to think of to your point, like, what would cause me to erroneously report a score if not, yeah. Yeah. So that's where it gets funky because that relationship variance is just like special sparks with those people. And that special spark could be, wow, you kissed really, really well together, and that was great. Or it could be, man, the person before me said that you did something that you didn't do to me. And so now I know that you didn't prioritize me in the kissing challenge. And so I'm going to give you a lower score. And I think both of those would fall into relationship variance. And both of those things happen. So it's kind of a mixed bad. I guess to like Sarah, you're saying that I mean, I would think because we only have one everyone does everyone stay in the same order for the yes, for the okay. So I would think like it the ordering effect in terms of just like you were third, you were fourth, you were fifth, that should be in the actor variance because it's constant. Like the, I think, right? It's constant for you. So it's like, but the, but the specific, yes. So the order of like you were third, you were fourth, you're just that piece, I think would be in the actor variance. But the like someone said that they were nicer to me. And then like that influencing, that makes sense that that would be in this relationship on because that would be different for eat. Like that didn't happen consistently every time. But if there was like, if there was just a constant ordering effect that like being third makes you on average score lower or whatever, I would think that would sort of get subsumed into the actor variance because that actor was always third. And so they like. What if it went both ways? It's what if it's, yeah, or what if it's like a, almost like a, like a regression to the mean, like with self-report, or if I ask you to like dietary intake, if you're worried yours is too high, you'll go to the middle. If you're worried yours is too low, you'll go to the middle. Like what if you had a higher score for that person and then when they give a low score, you're like, Oh, maybe I'm like, maybe my standards are wrong. And like you lower, you know what I mean? Like if it's not a consistent shift in the same direction, I wonder. Does it cancel out? Sorry. No, and I would say though, there's a bunch of times where people are giving widely different scores. Like they're pretty comfortable with like one woman saying like, that was terrible. That was a two. And it's pretty clear that it's because she knows who it is and doesn't like that guy or wants to send a message. And then like two women down says like, that was a 10. That was incredible. And that is like a known version of conflicts, I would say. Yeah. This is so fun. Yeah. So then, so I did this. And I would say this is really treating the data as way simpler than it is. So this is acting as though it's like a perfect red rob and everyone's interacting with each other, which they're not because the men aren't kissing the men and the women aren't kissing the women. And this is already getting rid of the many, many seasons where only the men or only the women do the kissing. So that's a bummer in terms of how much data we actually have available. And I think there's probably a way to estimate those kinds of seasons as well. But I have not figured out what it is yet because technically they're not very reciprocal. But for these seasons where the men did the kissing and then the women also did the kissing. So we have scores on both sides. What it actually is design lines is an asymmetric walk design, which is basically just where people are interacting with each other. But they're there within their own in-group and they're only interacting with people of the out-group. And so that you can estimate, I think, more appropriately with like a mixed-effect model where you also then incorporate group. So season in this case. And if you're doing a mixed-effect model, what's really fun is then you can start incorporating gender to see if like are the men behaving differently than the women. Because I would say like behaviorally the men are behaving very differently than the women. They're like often crawling on top of each other and like high five A and having a really good time about how their buddies are really kissing these girls that they all like. The women are usually just kind of upset. So it's an interesting dynamic. But then when we look at the variance components, there's not really much difference at all. The actor variance looks slightly different but it's so minimal regardless that kind of across the men's scores, the women's scores, and overall scores. That partner variance. So the person doing the kissing always accounts for somewhere between 20 and 30% of the variance. And then the relationship variance always accounts for somewhere between like 70, 75% of the variance. And actor makes up the rest and it's pretty tiny. Yeah. That's so interesting. But that sort of so the partner is just so to me that suggests that people are somewhat behaving consistently for the same partners, right? Like they are if there's very little actor variance, that would mean that like the not necessarily the scores themselves, but the variation in the scores is not differing that much between actors. So people might all have their own scale. Like I might like only range between a two and a three. And as we know about Sarah, she's always going to be in the eight, nine or 10, the way she rates books. So you say, I don't know how I would rate a line of men, but it is true. But at least on like good fair. How are you rate? Good read. Oh, anyway, so no, that you think so. Yeah, tell me. But their variance, right? Am I understanding? Am I interpreting this correctly that their variance seems to be consistent? There's not really actor variance effects that we're witnessing here. Yeah. That's my interpretation as well. It's essentially like there aren't not only is there not a lot of variance in the actor, but like none of the actors are none of the people judging kisses are wildly different from each other as well. Yeah. Because we're not seeing that as a like a significant effect. So everyone is for the most part scoring within that same kind of band, which is usually between like seven and nine. They're they're given each other pretty high scores. And then there's a few of those kind of outliers where you can see, oh, there's a very clear social dynamic happening here, which is why she's giving him a two or things like that. Yeah. Yeah. Okay. Fascinating. Okay. And that would show up, right? That would show up in the relationship. If it was like yes, yes, that would show up in there. Yeah. If all is going as planned. Yes. Yeah. It's a big if, but it could it could happen. So because the relationship and error variance is so big, I was struck with like, uh, wouldn't it be so cool if we could see how these things are related? And because we have these two seasons where everyone kissed each other, all the men kissed all the women and all the women kissed all the men, what I was able to do is actually make a bigger merge data set where each row, instead of being one kiss, we're going to talk about emoji. I as well love this emoji plot. I went through a real emoji plot era in my grad school days. I want to know all the details. This is emo gg. Emo gg. Yeah. Emo gg. This is incredible. Yeah. So I interrupted. No, you're so fine. So what I did is I made each row into like a couple. So rather than just being one kiss, each row is a couple. And then I have the man's score for when the woman was kissing him and the woman's score when the man was kissing him. So the scores don't represent the same kiss. I wish they did, but they do represent each person being the actor and being the partner. So this is a plot of how those scores relate to each other. As a way of starting to think about that relationship effect, it's not a perfect estimation of it. But they are correlated. And you can see like, yeah, when when one and scores men high or scores a man higher, that man also tends to score her higher. So there is something going on where people are generally pretty reciprocal with each other, which for SRM terms would be called like, dyadic reciprocity. Yeah. Like that even this poor little pairing in the bottom left. Yeah. It seems to have also been reciprocal, but it's funny that like as mean as the women get, like the low score, there's these scores are about a five. The men are like, no, that's a two in his hero. Like the men are so much meaner. Yeah. On this particular example. Yeah. But it does stand out so much versus like basically beyond that, the men are like, it feels like they're all, I guess it's foreigned up, but it does seem like the majority are like six and higher like you're saying versus the women have like this wider range. Yeah. And I think a lot of that is also somewhat the dynamics on the show, like, I don't know, I'm biased off the most recent season of Love Island where the women like really banded together against men being annoying. And And in that, it was very much like, how dare the men ever do this kind of mean thing to us? But like as women, it's totally fine if we're kind of meeting to them, right? So that dynamic kind of process here, where you see there's a bunch of women who are like, I feel totally comfortable giving a six 'cause that was kind of a mediocre kiss. But then the men are still giving them like, an eight, even up to like, I think there's one that gives a 10, so like, who knows? Yeah. Is there a time difference? So like, I know you said, it's not the same kiss, but is it the same episode? Mm-hmm. Okay, so there's always one. Could there be like a falling out? If you do like a kiss on episode one and a kiss again on episode three, and it, okay. I was like, that feels like that's how we end up with these like really aggressive kind of opposites of like a 10 and a zero or a zero and a 10. Yeah, no, this one, it's all in one block, which this is the problem is what I want to say to love Island producers is like, please keep doing these where the men and the women are both the kissers and also maybe do multiple kisses and let them rank them all. And then I went back and I watched just two seasons worth of kissing challenge scenes. And I was like, this is a ridiculous amount of kisses. I don't want to be watching this for this long. I get's bad television. So I can't rightly ask for that, but that is what I want data less. How did you, did you manually, you like, took, you watched it and just recorded? No, thankfully I was going to do that. And then I said, let me just check and see if this fan wiki has it. And thank God for people on fan wikis. Because it's everywhere just. I think this is the third or fourth episode with a fan wiki shout out. Yeah. Because I think our law and order data came from there. My severance transcripts came from a fan wiki, although I also had some manual. Yeah, my scandal survival, which we haven't talked about on here, but was like how long until Shonda Rhymes kills characters off in scandal, also from the wiki. Amazing. Yeah. OK, I love this. So yes, does look like positive correlation and also like hilarious mismatches occasionally where people can I ask as a presumptive, we do a little review or two bit later. But in the rules of the game, does one couple win? One woman and one man wins. But not necessarily having been coupled. No. OK, because I was like, I'd love to see like highlighted in like a different like which one of these won. But that's so interesting. So you win for like your independent merit. You win for your high score. And all you win is like bragging rights. So yeah, yeah. And I would say also like there are ways that people can socially win this game. Even if they don't win like the score. Like you can, for instance, just give a really good kiss to your couple partner and then give like really mediocre kisses to everyone else. And I think more people in the game would be like, yeah, that's so responsible of you. Great job. You're such a like nice partner. And that is I think a greater win in the context of the game. Like no one ever goes back and says like, I can't believe she got 40 points in the kissing challenge. Incredible. Not super much of thing. Yeah. What I do have in terms of manual variables and highlighting specific things is I went through because it was only two seasons. I did this manually because I didn't feel like figuring it out. And I marked everyone who was ever in a couple throughout the whole season. So good. And look at that zero. Yes, this is what I wanted to know. Yeah. And still she's like, he's a five. Yeah, it's fine. Yeah. But yeah, you can see like the people who were in a couple either at this point or end up in a couple at some point. Almost all of the score way higher for each other. And then you have this one that's having a fight and are upset with each other. And so they're giving each other the lowest possible scores they can imagine, which for a man is a zero. And for a man is a five. Well, there's all kinds of things you could ask about that. Yeah. What was I going to say? And for this, I was-- I originally did if they were in a couple like at the time that they kissed. And that was like marginally significant as a fixed effect and didn't look merely as convincing on the graph. And then I went back and I was actually like remembering some of the plot points. And there's plenty of people where they're not coupled up right then because say this bombshell has just entered the villa. So they haven't had a chance to couple up with their person yet, but they couple up the very next day. Or this couple's in the process of breaking up, like this one that we see that's scored really low, is in the process of breaking up. But then she's going to get with someone else and she's going to score him really high. So I went through and changed it from being a couple at that time to being a couple all over the whole season, which was less-- there was less flopping around than I remember to anecdotally from watching the show. I was like, wow, these people stay in their couples. Kind of consistently, which is a whole nother fun analysis that someone could definitely do. Yeah. And then I also did a fixed effect. This is my last plot. I said I included a fixed effect of that couple status. And it's significant. There was no significant interaction between men and women and their couple status. But if you look at this figure, there's a bigger gap for women between the scores for men they are never in a couple with and men they are in a couple with, whereas men are approaching more similarity between women they're not in a couple with and women they might ever be in a couple with. Yeah. It looks like, right. So like women, it's like a seven and a half on average for non-couples and an eight and a half for couples. Whereas men, it's like, what is that? Like 7.75 versus 8. Yeah. And maybe, yeah, like really, yeah, those are, I do. I think I, yeah. I can see, I can see what you're saying. Yeah. This is great. So, and like practically, so these are just fit. Again, you use like, there's a random effect for a person for kiss or a random effect for kissy. And then you just put in a fixed effect for gender, couple gender, couple status and interaction between the two. Or sort of. So this one, with the asymmetric block design, I ran using some code. There's a paper that came out last year, specifically I'd using asymmetric block design. OK. And doing that analysis in R. So it's a bit more complicated than that where they have some, there's, there's an additional contrast code for if it's a man kissing a woman or a woman kissing a man as a fixed effect. So we can see those general differences. And then they add in basically like each, instead of just having an actor effect and a partner effect, you have those four women kissing men and men kissing women who basically separate out those actors. So there's like three different random effects. Three, four, or maybe four, I can't remember. Their code is very clearly laid out and very easy to write. No, three maybe makes sense because one would, yeah. Three, I could see three because one would be the, like if it's an indicator, like the one would maybe be completely contained in the other three, I think I could see that being three. That makes sense, that OK, I love it. So that was fun. It's great. Yeah. And I love, it's so interesting. Sarah, I feel like are we, this is Sarah and I both are biostatisticians by training. And like a lot of these, so are not most recent, but second to most, two episodes ago, we had Matana on talking about this marshmallow test and he was talking about like he used a triangle test. And then he also, which I, it was basically like a way to do, I guess it's used a lot in food science or someone commented on a blue sky to say they also use a similar kind of testing in like audio, like to, for audio detection. But it's like, you have, there's two things you're trying to see if people can distinguish between. So you give them three, where two are the same and one is different. And you just ask them to identify which one is different. Yep. So they stay like, you can, they can remain blinded to the end, you also can like make sure. I guess it sort of was like a self control a little bit for like, you know, you're not having, and you're not having to identify which is which. So it's not like, they don't have to, in that construct, they don't say which is which, they can just say which, they can just see if they can detect the difference. So I had never heard of this and I feel like it's, and then he was telling us about these like signal to noise models that like it was just like a GLM with a pro bit link, which again is like something that we, we talk about in biostat. And I guess like afterwards, I was talking to one of our colleagues, Dacey, who does a lot of Asian inference and she was telling you, they always use pro bit links and most of what she's doing just like for convenience. But I like had never heard of these applications and it was the same for, I mean, we used a lot, mixed effects models are like, I feel like the bread and butter most of what we do. And biostat because we always have repeated measures, but I've never heard of this type of, you know, I guess because our data doesn't often look like this, [BLANK_AUDIO] is it's so cool to hear about these like other ways of using the same statistical techniques to answer like different questions. And even just like the interpretation of what the different coefficients mean under whatever paradigm. It's so cool. I was just going to say in like the part of part of what's so helpful about this paper from last year about running this kind of code is less so the actual model itself that like it helps you build it and that's great. But they also have all the supplementary material of various tabulation functions because so much of it is you can get the standard output from the mix effects model. But what you actually want is the variance estimates and then you want them standardized within the amount of variance. It's in the whole thing and they have written all of that. So you can just run the one line and you get your beautiful little table. But it has all the things. And it's similarly I had never worked with like transforming any of those outputs into anything like this. And I said, oh my gosh, this is so much easier. I don't have to do any of the crazy math. I don't have to figure out like what any of it means. They can actually just transfer it which is really just a lovely lovely component. Yes, I think that the distinction Lucy to is like we both do a lot with like missing data and measurement error. But we don't at least I don't often live in a space where things are truly latent. Like the idea of like that relationship spark, it is not directly measurable. It's a latent characteristic or like an abstract construct. And I feel like that's something we we usually have it for some people. We don't have it for everybody, but we have it for some. And when you get into this space where it's like we acknowledge that it exists, but we have it for no one. That is also a lot of the time like models that were used to but used in a different way. So I am learning like like factor analysis for a paper on like alcohol and substance use where alcohol use is a latent construct that you're trying to capture from like this audit. It's like a 10-question inventory of different behaviors. And I was reading papers in different journals and at the end of the day, I was like, oh, this is just a linear regression with a latent covariate. But it is funny to come across where it's like mathematically we could do that, but I hadn't encountered a reason to do it before. But I do think there is, I don't know, I don't know Hannah, have you ever done a factor analysis? Because that was the other thing I wondered. I was like, do you have these scores? And could you treat them as like being the end like being the result of that relationship spark and use it as a different way of modeling that piece? So I think you could. I've done I would dabble in factor analysis in a course that I took a few years ago. So not confident in it at all. Me either. I think the issue that you run into is there's so much of this like specific like there is some level of partner specific behavior. Like you do have people who are just better at kissing and worse at kissing. And so you would also need to capture that as well. Because otherwise it would kind of start to cancel out some of that relationship effect and you'd say like, oh, this person is just consistently bad. Yeah. I also just think this, I think with this amount of data, it would be nothing because there's like no data at all. Which is part of the, like I'm I'm a lot less familiar with some of these things because I work with child language data where like you're very lucky if you have 40 kids. That'd be incredible. And you're so lucky if 30 of them talk. And so you just don't have the number of data points that you need to do those kinds of analyses, which is always kind of a bummer. It's the reality though. I get it. Yeah. It's also so refreshing to hear about like small and like real small and scenarios. Because I feel like stats like we have so many beautiful methods for small and scenarios that require like thinking. It's like it requires our expertise in a way that I feel like the large and scenario. I mean, it's not to say you can't have bias in these large and scenarios. But I think the large engineers offer more opportunities for kind of throwing an algorithm and letting or throwing a suite of algorithms and letting them just sort of like sort out what they can make from it. And you like can't do that. If you have 30 observations, you can't do that because you won't get anything meaningful. You'll mostly just be fitting to noise like it. And so I think I love hearing about examples where N is small. So we still need we're still needed. Our discipline is still important. Yeah. And I should add with SRM, you can actually do you can have the outcome kind of like go to a latent construct. You can put multiple outcome scores into that like what you're regressing on. Which is kind of nice if you have multiple scores. So there's a lot of really fun papers out there that do like this example paper was combining like people's attractiveness ratings and people's sexiness ratings and then saying if that makes them go on another date. And so you can do things like that where you're kind of combining those two scores into sort of a latent construct. Yeah. But again, you really have to think about like okay, which thing is leading to what and what am I actually asking? And it gets really messy with like who's the actor and who's the partner depending on what you're doing. Like even with this stuff, I feel like I have to keep being like okay, the kissy is the one doing this and the kisser is the one doing this. And it just gets mucky. Yeah. This is so fun. Well, this has been excellent. I we so usually I well, I think we've already kind of like got to some of the review or two Sarah had some some points there. And the last thing we usually do is this fetch rating. We try to have you rate your own analysis on a scale. Which I was actually going back and listening and realized some of the more recent episodes we've claimed that the scale is one to five fetches. But I actually think with someone like I think maybe with Emily, we didn't give her a max. Like we just were like yes, like we were like it's just someone described themselves as well. So Maton described himself as fetch maxing, which I loved. Yes, yes, he took the the hat the top. So anyways, yeah, you could pick it up about, I mean, it might be festive to give a max of 10 fetches. That's what I was going to say. That's true. To agree with the given. Yeah. Yeah. Okay. Perfect. Like 7.5 to 8. And I think a lot of that is that I wish there was more data and more people were doing more full round robin kissing on me. Yes. That's my that's my big zone box. If I have to stand on one from this, you want to you would like for it to exist for the data. You don't want to watch it, but yeah, but if the fans could put it on the fan wiki, that would be yeah. If they want to go to a full round robin where we do the whole interpersonal issues of like all the men also have to kiss all the men and all the women also have to kiss all the women, I would absolutely watch a full episode of that. I think that would be fascinating. But if it's just going to be men and women just kissing multiple times, I think that could be a behind the scenes, but then give us the data. Yes. Just give us the data. Yeah. Just just show us that little that little that little whatever the key by the table. That's all I need. Yeah. Oh, that's so good. Okay. 7.5 fetches. I think that's the most fetches we've had. So this is excellent. Although spirit. I mean, I don't know. We haven't always. Yeah. I love it. This is that we created no real good metric on this. And so everything is good. That's the beauty. You know, it's the beauty. Yes. Yes. Well, this was so awesome. I think it's also a good opportunity to remind people that we do have a Gmail account. If you like Hannah, have a really cool data side quest that you'd like to share, you can email us at regression to the [email protected]. Thank you so much for joining us. You go, go, go. We are regression to the meangirls who are mean as an average, but actually very nice to do it next time.