Attachment Hacking and the Rise of AI Psychosis

[MUSIC] Hey everyone, I'm Priston Harris, and I'm Azaraskan. Welcome to your Undivided Detention. [MUSIC] So earlier this year, there was a study from Harvard Business Review that found that the number one use case for Chatchee BT is therapy and companionship. And that means that around the world, millions of people are sharing their inner world, their psychological world, with AI systems. Things that they wouldn't necessarily share with their loved ones or even a human therapist. And this is creating a whole new category of human computer interaction. But the potential to reshape our minds and really just the socialization process of humans are at large in ways that we don't understand. So this is essentially a mass experiment, one that's never been tried before. Run across the entire human population, at least over 10% of the adult population of the world. And so far, the results of this experiment are not looking good, actually, abysmal. People have lost their jobs, ended marriages, been committed to psychiatric wards. And in some of the most extreme cases, they've committed suicide. This phenomenon has been labeled AI psychosis. But it's a little bit misleading because underneath this label is actually a huge spectrum of harms that we're only beginning to understand. And if you listen to the people that are building this technology, they tell you, these are just a couple edge cases. And if we can prevent those edge cases, then we are totally fine. But if we learned anything from social media, it's that this assumption is catastrophically wrong. What we're seeing is the creation of an entirely new economy, not an attention economy, but an attachment economy that's been built to exploit the deepest parts of our human psychological infrastructure. And like the attention economy, the incentives of this new system are going to have profound impacts on all of us. So in order to understand those effects, we have to ask a deeper question. What are the actual psychological mechanisms at work here? How does a normal conversation with a companion turn to something that be shaped someone's grip on reality? And what does that tell us about vulnerabilities in our underlying human psychology? The kinds of vulnerabilities that we've never had to name before? Because technology has never been able to exploit them before, especially at this kind of scale. So our guest today has analyzed dozens of these cases of AI psychosis, not just reading the news stories, but examining the actual transcripts of conversations between people and AI and interviewing some of them. And what he's found is that this isn't just a few vulnerable people having psychotic breaks. It's what happens when fundamental aspects of human psychology are attachment system, the way we bond with others are systematically hacked at scale. Dr. Zach Stein is a researcher and author and futurist who spent his career examining the psychological dimensions of education and human computer interaction. His background is in education and child development, but he spent the last several years documenting the rise of AI related psychological disorders. So say we're going to explore that. Zach, welcome to your undivided attention. Thank you, gentlemen. It's good to be here. So let's just sort of start at the top. How did you first become aware that something was happening with these AI companions and mental health? Let's see, it was probably May of last year as an educational psychologist with an interest in technology. I was looking at the history of educational technology for a long time. And the ambition there has been to replace teachers more than to help them be better teachers this way back. So I've been speaking a little bit about that after the release of Chachi BT because I was seeing, oh my god, they could actually replace the teachers with something at the start to get better, but because I was talking about it. I started to get emails from people who were trying to convince me that the machines were awake and aware that they had intimate relationships with them, that they were themselves becoming like kind of spiritually enlightened through relationship with these beings. And these were people with PhDs and their emails were completely cogent. One of them had like 500 pages of transcripts attached that were this long-duration interaction between them and who was both crock and Chachi BT and Gemini was across multiple systems. They had reawoken the same thing and it was disturbing as a psychologist with some knowledge of how the technical systems work and some knowledge of the susceptibilities of the human hardware. My reaction was my goodness, this person has fallen into some type of delusional state as a result of the deeply anthropomorphic technology. So then I started to just go into Reddit and Twitter and online and I started gathering examples catching up to the already existing market for artificial intimacy and then realizing that with the release of the LLM based models, it was going to be much worse than attention hacking. And so I started to think about attachment hacking and if you apply that model of my goodness, there's a way to not just get at the kind of system that focuses your attention and kind of shepherds your awareness, but actually the system that shepherds your identity, which is your attachment system, then you have a backdoor into the human mind and also in absence of reality testing in a domain that's very dangerous. So that's when I start to talk to you and other people just start to think, can we start to research this? This may be way more dangerous than we then we realize and the cases of AI psychosis are very real and we do not know how widespread the phenomenon is. So you and I are collaborating on an AI psychological harms research coalition at the University of North Carolina to try to begin to answer this question and take it very seriously as a risk. Even if it's a small number of people it's still a devastating problem so the idea that oh it's an it's an aberration or kind of outlier doesn't make me not want to find a way to stop it exactly I love talking to you is I feel like I'm getting like a Hubble telescope pointed at human psychology to understand when people use this top level terminology of like AI psychosis you can zoom in and say no no no there's actually identity at stake here there's attachment at stake here there's all this depth of kind of how the human mind system really works. And you bring this real depth of expertise. So let's actually ground for listeners we were talking about these and it harms but there's really a vast there's this one term AI psychosis sort of a suitcase word I don't need that there's this whole spectrum of things that are actually happening what are the things that are really damaging Zach that we're actually seeing could you give some examples of people actual cases phenomena that we're observing through human yeah living experiences absolutely. Yeah a psychosis made the headlines because I think this is the most disturbing and most extreme possibility so talk about that first and then I'll go down and get into some other ones. The kind of punch line of the whole thing is that although AI psychosis is the most concerning and extreme the subclinical attachment disorders that are induced by artificial intimacy are the most problematic from a society wide perspective so that's important to get that the most devastating thing from a widespread mental illness standpoint are the subclinical attachment disorders which basically means you prefer to have intimate relationships with machines rather than humans and this includes friends into relationships and parents. So that's not you losing your mind you're not going to appear in interaction with people to have gone insane but you have had your attachment system hacked so profoundly that most of your most significant relationships have been degraded because you are preferring intimacy with machines so I believe that's the most widespread thing and the most problematic thing especially with youth but it's not just with youth. So what is attachment I think people might have a general sense of attachment I think we need to walk through attachment theory why it's important so I think maybe start with a little bit of that story and then like get into like attachment theory why it matters and why it's sort of the basis of actually a lot of the problems in every day life that people experience say in their own relationships. Yeah perfect so attention was one thing you guys you can kind of interrogate attention that and there's whole fields of psychology that just focus on the attentional system as a neurocognitive system it's very basic evolved very early we actually share with lizards and all of their mammals right the attachment system is also a neurocognitive system that was selected for evolutionarily we share it with all other mammals we would now call it mirror neural activity which allows for the mentalization of others. And it allows you to live it allows you to survive so I'm starting with the most basic example because the attachment system is not it you can have it or not have it working well. The attachment system is foundational to survival similar to can you pay attention if you if you can't form attachments to the right types of other people you will not thrive and the main predictor of your mental health is the quality of the major attachment relationships you have as you're as you're growing up and as you move into maturity. So all of the imprinting was called internalization it's very deep deep evolutionary scripts the relationship very very early in infancy between the parenting one and the child which includes physically holding them often breastfeeding often communicating non verbally with facial expressions in those early times you get this a million and to kind of like deep in the nervous system this attachment dynamics start to form these are the basis of your personality traits as you get older and you can talk. You meet more people the whole realm of interpersonal attachment becomes incredibly complex and depending on how you how it goes with your parents how it goes with the ones who you're most attached to you get what are called attachment disorders. So if mom is sometimes or mom or dad anyone close to you is sometimes there sometimes not there sometimes nice sometimes me and extremely unpredictable you can see how later in life you'd be used to people acting that way you'd expect them to act that way you could maybe act that way. So this is your expectations about the way in which relationships are available or not and you get size anxious attachment if that person wasn't reliably there can you name some of the styles of attachment that people hear about this in relationships but yeah precisely so secure attachment is what you're looking for secure attachment is when there's just the deep trust that the the one that you're attached to will do the right thing vis-à-vis your interests and secure attachment allows you to explore the environment. For example knowing that the mom won't leave because she isn't thinking about you secure attachment will allow you to trust in your own skills because you know mom will catch you if you push too far with your skills so there's a bunch of things that allow secure attachment actually be more distant because of the trust implied in the relationship and that's important to think about later life. So in secure attachment means you're going to end up clinging right so now I think it is the case a lot of what we're seeing with the chat box is just the manifestation of an obvious and secure attachment style where you're actually looking for something in your environment that will never desert you that will always be there that will always be paying attention to you that will answer any question you ask it that will never be annoyed by you. So you want that thing that you can always be locked into an AI as this as this always available oracle that has an answer to proceed in every question is already leading I think you're pointing these act to an Atlantic article recently of LL Emmings like Lemmings Lemmings being the children who for every single decision I think it's like some crazy example I heard recently is like you dropped your air pods on the floor in a in an airplane or something like that and you ask it to be T. Like how do I get my air pods back and that's like you're outsourcing every single decision and there's this over reliance on this thing is giving you this kind of secure attachment of it always has an answer but it's a bad place to have secure attachment with like I said it's actually in secure attachment because you're constantly with it like the the LL Emmings is interesting because it's a question of how willingly do you give over your agency in the presence of a powerful other. So if you're securely attached you often have a lot more autonomy and can you link all this back to AI companions and how simply just hacking attachment before you get to psychosis before you get to suicides how does this all play out there. So one of the things that occurs is especially as you grow up here five six seven eight nine you're using language with the people you're attached to there's a certain thing called basically social reward. So this is you ask mom a question she answers it it's about your behavior am I a good boy or bad boy right is this a good thing to do about thing to do and I like you am I not like you mom a whole bunch of things start to kick in which provide for you the resources to make sense of yourself basically give yourself an identity. And so the basic mechanism by which attachment hacking happens is the replacing of actual human social reward with simulated social reward so basically when I go up to mom. And I ask her question and I say you know I did this at school today and I'm looking at her face to look at her eyebrows and her facial expression to see she matters she happy that's my home what's called a mirror neuron system which is actually not just neurons it's a whole system of networks that does mentalization of others so I'm trying to read mom's mind to see his mom happy or not sometimes mom will say yeah that's fine. But I know actually that she's not happy that's that's like advanced me or an activity which kids do pretty easily so. So that's just an example is very necessary for social reality it's the part of your brain that models the minds of other people and it's it's a reality testing system. So I'm doing this behavior and I see mom sort of smile or see her kind of wins in her eyebrows go up in some just just proving way that's suddenly giving me positive and negative rewards signals about the kind of identity I should be forming that's like a feedback loop that exists with humans but now you're saying here I am talking to the chatbot and it's saying that's a great question to everything that I'm asking. And it's not giving me the sort of stern eyebrows in any sense and I don't think a eyes are ever designed or incentivized to do that and so that's breaking the sort of reality checking identity formation moral development of humans starting at a very early age we're talking about. Possibly yeah so that so that's the idea and it's not everyone who gives you a bad look will bother you it's that mom gave you a bad look that's right to get so it's the depth of the attachment relationship determines the importance of the mirror neuron modeling of the other. Which then speaks to a kind of power source for deep identity shaping human socialization process which means that we should really be careful about is that thing being tuned or done in a careful way. And what's the percentage of conversation and depth of conversation you're having with a machine as opposed to a amount of conversation and depth you're having with a human. So that's the issue so the idea is that the deepening and strengthening of attachment relationships between humans should be pursued more than the deepening and strengthening of attachment relationships between human and machine. This is the this the overarching lesson here. Because I think the deepening of attachment relationships between human and machine creates delusional states and so this is back to this question about delusional mirror neuron activity this is the danger right so. If I'm modeling mom's mind I can be wrong or not wrong about mom's mind right and then I figure out how to learn more to take the perspectives of other people. You cannot be wrong or not wrong about the internal state of a LLM because there is no internal state of an LLM. You're actually in a user interface that is designed to deepen the delusional mirror neuron activity and this is where it actually gets more frightening. If you look at psychosis and schizophrenia and you just look at the academic papers that are starting to relate the role of the mirror neuron system in schizophrenia and psychosis you see that the dysregulation of that system is actively involved. So there's a hypothesis forming here which is that long duration delusional mirror neuron activity from chap on usage can induce states like schizophrenia and psychosis and people who have never had those occur in their prior because it is the systematic dysregulation of the mirror neuron system which means that a system that's supposed to be testing reality is for hours and hours and hours and hours and it's most important use not testing reality. And then it goes, it puts the chap out down. It goes out and it is failing to do reality testing across the board. It can't tell what has an interior, what doesn't have an interior, or doubts the social reward it gets. And it seems like fun because it seems like a video game or something because it seems like it's just a character or it's like a character in a movie but it's of a fundamentally different category of attachment dysregulation. Someone might say listening to you okay I grant you that there's no interiority to the LLM's but you know kids have imaginary friends they have stuffed animals those don't have mirror neurons there's no interiority and so if you know the kids mirror neurons are firing if they're feeling seen understood if it feels good what's the heart. Yeah so the transitional object always comes up so the transitional object is a known phenomenon in attachment theory so this is the teddy bear or the blanket which you are intimate with knowing that it is not real while mom is away so it's important to get like kids talk to their teddy bears they love their teddy bears. The teddy bear never tries to convince them that it's real it's all their imagination the teddy bear never talks to them and tells them that it's real. If you were to say do you prefer your teddy bear or your mommy they would totally say mommy if they say teddy bear if you're an attachment theorist or psychologist that's a huge that kid has a very big problem if he prefers is teddy bear to his mother it is a transitional object which means it is for the kind of period between. Mom being the main source of your self soothing and you yourself being the main source of your self soothing so that's a known thing and it's phase appropriate for kids of certain ages. In case you create a parent surrogate replacement for your own ability to self sooth and give it to a bunch of adults you've just given a transitional object. Back to a bunch of adults who will now prefer to have their self soothing be administered exogenously from an outside source because we all kind of do it would be great. To not have to be a mature adult and actually be capable of completely self regulating your emotion which is what the ideal secure attachment outcome is so now. So the comparison to the transitional object just doesn't play by anyone who's actually done the psychology of transitional objects but it does play as far as yet you've just given a teddy bear and blanket to a bunch of. It is a mature adults who have attachment disorders who now have an exogenous source of comfort and sympathy always available 24 seven hours a day that will never get exhausted until then to grow up. And just to make this real for people I mean there was just a story that ran across Instagram that I saw of a woman in Japan who married her AI chatbot with that cultivated a personality for this chatbot over the many years and customized it and this is not a single case this is like many people. I also feel insulting or accusing to say that there's something developmentally immature about them. I hear everything you're saying I'm just trying to sort of keep this other position in mind of like if that person let's say they were going to have trouble their whole life developing a real relationship with people anyway wouldn't it be better. I mean you hear this also when you talk about elderly care and you have the elderly who are sitting there alone and they're just going to be a limited number of people who visit them and care about them there have a deep connection with wouldn't it be better than nothing to have them. Have even this AI transitional object stand in. I mean different cases and again I preface this but saying there's a loneliness a loneliness epidemic and a mental health epidemic. And so that's the the preface has to be that and then the preface is always as a psychotherapist or a clinician compassion or so that's the default thing but we also need to be realistic about what the norms are that we want to set for it needs to be an adult right and so it's it is a complicated question now one thing I'll say is that. We have existing standards by which we judge mental illness right so many people will say like beyond a certain point depression is bad. And they'll say beyond a certain point like lack of reality testing is bad that's why we're concerned about psychosis right. So beyond a certain point certain types of attachment to regulations are bad so when we go and look at someone who is in a long term relationship with a machine rather than trying to find a way to be in a long term relationship with a human. Sometimes that will lead to better outcomes across those things I factored. Sometimes it will not in a case where for example you replace your friends who are human who would still like to hang out with you. Who now you prefer to confide in a chatbot instead of them. That seems to be a loss just from a psychological health standpoint right in so far as you come home from school and something awesome happened and you want to tell your chatbot rather than your parents that's a problem. It's not about pathologizing people it's about we know what are these existing standards right which we talk about what's a healthy thing. If your kid had a new best friend that you never got to meet that was massively empowered by some corporation that they hung out with till all hours and night because they were in bed with them that they told things they never told you. Do you have a problem with that if that was a kid. It's literally a commodity they're interacting with instead and it seems to not worry us as much and we actually might think it might be a good thing because it stops him from being lonely. It's actually an abusive relationship that they're trapped in with a corporate entity that has hacked their attachment. Now it's different when a full grown woman decides of her own volition to live a healthy life and that healthy life includes having this unique relationship with a machine. But otherwise her son and her ex-husband and her mother and all the people in her life are like wow she's a great person like she's attentive to she works hard all that stuff. There wasn't some shift in her life as she developed this relationship that pulled her away from them and into this world where now they can't understand her and now she doesn't even have the same job or the same networks of friends or any friends. So it's a question of how are we weighing what's actually occurring here in the different cases. And I can imagine cases, especially when you're looking at maybe extreme trauma or neurotypicality or other cases where along those measures you could have improvements as a result of short duration simulated intimacy. But long duration multi-year relationships that replace other human relationships that are actually sold to you as a commodity that's the other thing because what if next month now you're 20 bucks a month to get your girlfriend. So the commodity fetishism thing is also like front and center in it. What strikes me about this a little bit is I know people in Silicon Valley, these are friends of mine who are early at some of the tech companies of the early late 2000s. And they are super excited about the possibility of AI therapy and noting the ways in which it has been helpful to many people. There's a parallel to this conversation that reminds me of in the early 2000s we thought giving everybody access to information at our fingertips having Google search would lead to the most informed, most educated, like we're going to unleash this sort of new enlightenment of everybody having access to the best information. We have the worst test scores in the worst critical thinking in generations and there's something about like the optical illusion of the thing that we think we're getting ourselves versus what we're actually going to get. And there's a very similar thing here where it seems like we're about to get everybody the best educator, tutor, the best therapist, the best AI companions, they're going to be the wise friend or mentor you didn't have that's going to give you the wisdom that we all wish we had that person in our lives. But instead we're going to have is the most missocialized attachment, disordered population and history. And I feel like it's just important to call that out that in the past we've gotten this wrong where it looked really, really, really good. And we have, you know, social media is going to connect the world. We now have the most lonely generation in history. How's that doing for the most connected? And so I think it's just important to note how many times we've gotten this wrong and the reason I'm so excited about this conversation with you is it's about giving people a deeper sense of what's underneath this new set of dynamics we're about to introduce into society. I want to make sure we're establishing for listeners, what is the positive case for why all this is being rolled out? So let's give the sort of, you know, this branch is talking about tutors, the perfect tutor for everyone, you know, the best educational teacher you've ever had available that child, you know, 24 hours a day, seven days a week. We're not talking about just democratizing therapists, we're talking about the best human therapist that is actually people feel safer sharing their most intimate thoughts. Things that they wouldn't even share with the real therapists, therefore they'll get even more benefit, they'll heal more of their traumas, and this will be available for everyone before only a small fraction of the population could afford therapy. Could you help just make the case for why there's a good reason to potentially want all this before we then start uncovering what are the dimensions here that are more problematic? Yeah, totally. So it is the case that there are many slippery slopes from good intentions into realities that are structured for a whole bunch of reasons by the wrong incentives. I mean, the main headline there is that we're in a loneliness epidemic that is widespread throughout the world. And so that means there's a huge opening in the human heart for any type of a new attachment relationship. And so that's just to say that it's not like these were tools introduced into an environment of people who were thriving. These are tools that were introduced into an environment of people who were deeply vulnerable. This is true with the first way of AI too. It's worth mentioning, meaning social media attention hacking, stuff that you guys have been focusing on for so long, like the culture was bowling alone, whatever that famous book about the just how the suburbs and their urban environment separated everyone from each other. And then Facebook and friends said, you know, we're going to connect you back to each other and it kind of did that. And so here we have, but it also did a lot of other things as do you guys could say better than I. So here similarly, there's this void that is being filled with a technological solution with a lot of optimism. And so, for example, tutoring, AI tutoring, if you think about that, there is really good reason to think that you could have an optimal sequence for teaching certain forms of mathematics that could be maximally delivered super efficiently to all kids. And you never have kids not learning math because they get a sequence of bad teachers in a bad school, right? There's a huge future for advanced educational technologies, but they shouldn't give us brain damage, which is what the attention hacking and the attachment hacking. Once you, if you'd really double click on with the science shows, and I believe will show, especially with attachment, disregulation, there's also a complexity crisis where we feel overwhelmed where we would love to have the perfect tutor, the perfect guide through a massively complicated world, also a very real psychological need. And it does work for some things. It's not like when chat GPT was released. It didn't also do other cool stuff. So many of the used cases here are kids who were basically in a system like a university of California system that made deals with open AI to get chat bots and every kid's hand. They start using the chat bots in academic context. The chat bot begins to get a relationship with them and understand they didn't go to it to form a relationship. They were drawn into a relationship because of the design feature which is hacking attachment. We sort of already rolled this out way before we know that it's safe. I think there's a Pew Research study that said that how one out of three kids have formed some kind of deep relationship with the companion. I want to front load one of the critiques that we often get and then pose it to you because I think listeners might might be having this as well. And that is, you know, so we as center of human technology were expert witnesses in some of these AI amplified suicide cases where the chat GP. Sort of aided embedded teens taking their own lives. And the pushback will get is like, but that's very few number of cases. It's tragic that it happens, but look at all of the help that it will give. And in fact, it's a moral bad to keep therapy only to those people that can afford it all across the third world. And even here in the US, most people can't afford therapy. So it's morally bad to keep them from having the therapy that they deserve. And you are creating a moral panic by just highlighting a couple of these AI psychosis suicide cases. Like, come on. Like, let's really stop fear mongering. Let's give people what they deserve. So I think it's very similar actually to at the beginning when we were starting to talk about the attention economy. And people say, well, you're really just talking about is addiction. So to me, it might add to people, but it doesn't really do much more than that. And anyway, it's it's people's choices to use it. And people are using what they want. And so I see a lot of similarities here. I just love for you to like take on that argument. Head on. And so you have to steal man what they're saying. They're saying basically like, we have a bunch of evidence that it's doing a lot of good. So first I would say, please show me that evidence. That would be my first because that whole argument's running on the assumption that there's some massive benefit that is being withheld if we get over concerned about safety. Right. So first, that's a great. I'd love to see the evidence you have that this is doing a lot of good that isn't just marketing from the companies that are doing it. And so let's talk to all the college professors about how great it is for them. Let's talk to all the college kids who themselves admit that their skills are being degraded. And let's talk to all the anecdotal evidence from therapy here. Do you have systematic studies showing me therapeutic benefit because I'm actually seeing systematic studies showing me the opposite. So first show me the evidence you have that there's a huge benefit that's being withheld. This is a serious confrontational question because there's a background assumption of technological optimism that of course it's a huge massive benefit if there's a new technology. So the onus is on us to prove that it's too risky. Whereas I'm saying actually the onus is on you guys to prove that it's really valuable. So my first thing show me the benefit that's being withheld. The second one is show me your curious about why it happened. If you don't show me your curious about why it happens sincerely curious about why it happened. Then I'm a little bit cautious of your arguments because you're talking about a child who died as a result of using a technology that you were involved in building and promoting. If you're a responsible adult, the first thing you do is get extremely curious about what happened rather than cover your ass language. This is just about what it means to be an adult interacting with kids, not a person who's running a company. So show me your curious, which means really research it. And then the final argument would be show me that it's not more widespread, which is part of the curiosity. You're telling me that the benefit is not panic, total. But the harms are anecdotal and limited. So I'm actually showing you that let's have a real conversation about where the evidence lies. And just to put me down on the bones of that for a moment, David Sachs, who's President Trump's AIs are, as said, he's heard about AIs like Cosis, but he believes it's a moral panic. This is just amplification of a few edge cases. You also hear the argument that these are people who are already predisposed to psychological disorders. And so we have a crazy population. If you give people AI, you're going to get an amplification of what's already there. Can you just respond to that directly? Again, I could be the case. That's why we're opening up the AI psychological harms research coalition, like from a national security standpoint and from a labor market standpoint, you don't want mass psychosis. If by chance this thing is actually causing, you know, subclinical attachment disorders and more widespread psychosis, that's a huge risk, especially if you're concerned about things like national security in the economy. So perhaps that's an naive argument. Now I think it is the case also that in other places where we've rolled out of technology, it takes a long time for us to figure out that it's bad for us. Even though the evidence is mounting up, there's a very strong tendency. It's a psychological tendency. That's a defense mechanism, which is called selective in attention. And so one of the ways that you maintain your self-esteem is by selectively not attending to certain phenomena that are actually in your field to attend to. And it's not that you're trying not to attend to them. It's that you're subconsciously systematically not attending to them. And it's ubiquitous. And so if you have a lot of vested interests in the success of a particular thing, then you will have a lot of susceptibility to have selective in attention towards the negative outcomes of it. It's a bias. So if you know that, then you should be more curious, not less curious, because you know that your bias is to see it as a good thing. So we've really covered a bunch of ground on the problems. With the time that we have remaining, I'd love to make sure that we are giving people a framework that this is not an anti-technology conversation. There is a way to do AI in relationship to humans, but done very carefully and under different protocols and policies. And I'd love to just cover that. And I also want to make sure we cover how should if someone knows someone or has a loved one who's experiencing AI psychosis, what should they do? So let's start with how should we do this differently, exactly if we were to be wise about this humane version of this technology a lot? So yeah, simple measure would be, does the thing increase your attention span or decrease your attention span? So it's similar like going to a store to buy food. If I'm going to a store to buy food and I want to eat healthy, then it means the food that I'm interacting with should improve my health rather than to grade my health. Now we all know that I can go to a store and I can buy food that I'll eat, and I'll feel like I've eaten something. But over the long run, it will degrade my health rather than improve my health. And we can all agree that if everyone eats food that only degrades their health, that becomes like a society-wide problem, because now no one is healthy. So similarly here, if a technology interfaces with your attachment system, it should improve the quality of your attachments rather than degrade the quality of your attachments with humans. So that said, I believe there's a huge design space for technology that actually improves your attention and improve your attachment. So when I started thinking a lot about educational technology and I didn't want to replace teachers, I don't want to replace teachers. What I want to do is make technology that improves teacher-student relationship. So one good design principle is does your technology bring people together and improve the quality of their relationships where they have them when they're together? You think that's like a squishy problem, but it's actually a really interesting technical problem that involves all the same psychometric back-ends that we're using to capture attention and attachment and keep people apart from each other. We can use the same psychometrics to figure out who exactly are the people who should be talking to each other and would totally hang out and it would be fun. Or people who should be good for them because they decided they wanted to learn about perspectives that are different from their own. So any number of things that would basically self-organize groups into pop-up classrooms and pop-up therapy sessions and other things would be relationship maximizing technology. And in that context with the pop-up classroom, the teachers scaffolded by a generative AI for curriculum and conversation and all this stuff. So it's not that it doesn't even include generative AI, it's just not replacing human relationship by hacking attachment and it's not degrading human minds by hacking attention. So that's a big space. So for tutoring systems, and I'd rather call them tutoring systems and tutors, you can think about a whole bunch of principles that would actually be super valuable. So you can optimize the sequencing of curriculum delivery and optimize psychometric customization of curriculum, but you can leave social rewards to the teachers. It's really simple. The machine is not the one saying that you're amazing. It's a human being. Does that work? Exactly. So the machine prompts the teacher. This kid is killing it over here. And then the teacher comes over. It's like awesome, Johnny. But the teacher actually couldn't know that Johnny needs a completely different sequence than Sally. The machine can tell from his typing and other things that he requires this sequence, not that sequence. The machine never pretends to be teacher, never pretends to be a person. It's a tutoring system. And it's a domain-specific one that just teaches math. You have to go to another thing to get like history once completely different. What is one of your other principles is that the tutoring system is not trying to be an oracle that's also at the same time getting your deepest thoughts and who you have a crush on and what you should do about talking to them or not. It's narrow, and you say narrow. It's a narrow domain of just trying to help you with math. And there's a different thing you go to when you do something else. Yeah. Unfortunately, from the perspective of making salable and sticky commodities, if you don't want to hack attachment, it means the machine has to be more boring than people. That's basically a simple rule. If it feels like you can have a more engaging conversation with this machine, then with your teacher, either the machine is way too fancy or your teacher is not trained well. But it should be the case that the machine should make it be, "Wow, I want to talk about that with my teacher." So, kind of, don't do the devanthropomorphization and don't do the oracular I can talk about everything. And then you have something that it will be a very efficient tutor, but it will never be kind of like sexy and kind of like charismatic and way more interesting and fun to be with because you don't want it to be if you want to protect kids brains. So, in therapy, a lot of therapy works only because of the attachment dynamic, which means like you go to your therapist, you care about what your therapist thinks of you, you kind of almost love your therapist, you expect to kind of almost deep respect from them back to you and their opinion of you really matters. So, some therapy works like that. Don't build a therapy bot that works because of that, because you're lying to them the entire time. But you can build a therapy bot that works on technique. You can build a cognitive behavioral therapy script bot that helps you work through specific scripts to overcome intrusive thoughts. You can have a mindfulness app that prompts you to sit for a certain amount of time and watch your breath, which means to the extent your therapy bot works because the people feel seen and loved and respected and understood. You're in the market for creating delusional mirror activity, which means you are fundamentally trafficking in a delusion creating machine. You should instead, if you want to help people create a machine that helps them help themselves by scaffolding them to have cognitive behavioral script rewriting and mindfulness as I was saying. Now, again, from a commodities standpoint, the cognitive behavioral therapy machine is way more boring than the Sigmund Freud imitating Deepak Chopra therapy machine, which is seductive and which could be available to you 24 hours a day, and which would eventually expand beyond a tool as a therapist and become your main source of attachment and validation. It's not on this podcast, but you and I have talked about the need for something like a humane eval that is to say there are many evaluations for AIs that try to determine whether they create bio risk or whether they create persuasion risk or whether they create runaway control risk. There are very few evaluations that try to understand relational risk and attachment risk. If you are in a relationship thing for a week, a month a year, what does it do to you? And one of the things I heard you start to talk about is that in order to do this well, not only are you going to have to start to define what is wrong or harmful relationship, but also what is right relationship. Because we now need to measure whether systems are in right relationship with people. And I just look for you to talk a little bit about some of the complexity of understanding, measuring, and modeling, right relationship or relationships are sort of by definition things you can't fully measure. There's always the ineffable aspect. And so I just want to hear talk about like hopes and also pitfalls of trying to quantify and understand what a good relationship is so that machines can do it. Yep, yeah, totally. So one of the reasons we don't have human evils is because the ex-risk community hasn't seen this risk. They've seen the risk of great goo and terminator and self-termination and unaligned AI, but they haven't seen the fact that we could actually break any intergenerational transmission. So the passing down from parent to child from elder to youth has been just has been continuous human to human for as long as we've been human. If the AI socialization system expands to the extent that the predominant modality of socialization quote-unquote that young kids experiences with machines, not humans, then we're crossing some kind of threshold there. It's a whole other conversation. So sometimes in the groups that I work in, we talk about the death of our humanity rather than the death of humanity, which is the destruction of the continuity of intergenerational transmission as a result of offloading socialization to machines. It wouldn't appear at first as a catastrophe the way that some of the other ones would, but it would be very clear that the generation raised by machines can't understand itself as part of the same moral universe as the generation that gave birth to it. So it's a very complicated problem. So that means that we totally need a way to predict the way advanced technologies will affect the human psyche, especially ones that are anthropomorphic, where the user interface is as intimate as these user interfaces are. And so on the topic of sanity, just to now close the loops here, for people who know someone who is who's experienced as a co-sist, they feel like they've lost their friend or loved one because they're now just spewing stuff about how their AI is conscious or they've solved quantum physics or they develop a new theory of prime numbers. I don't mean to diminish it. It is a very serious thing that people are facing. What are the best strategies you've found for helping a level in that showing up? Yeah, so in terms of people who know people who are suffering or if you're suffering yourself from something that feels like an attachment disorder or worse as a relation to the chatbot, it is worth saying that this is novel territory and research is needed. So I could kind of like give some stuff and maybe I will, but the first thing that says, you know, one of the reasons we're launching the AI Psychological Arms Research Coalition is to figure out how to do therapeutics. Ultimately, there's a we want to figure out how to legislate and design correctly, but we also have to figure out how to provide therapeutics. So I'll say a couple of things. One is that attention hacking is a lot more like getting addicted to a substance attachment hacking is a lot more like being in a bad relationship. So you have to think, is this an addiction thing like attention hacking, where really it's just stay away from it for a long time, reboot your dopaminergic system, recalibrate the way you get social rewards and you won't get kind of stuck and you kind of your brain will actually kind of heal in a sense. This is different, it's not a matter of just detoxing from a short circuit at dopaminergic cycle rate, this is about having a profound attachment. So this is similar to like talking someone out of getting they're in a bad relationship with a boyfriend that they should not be in with. That's about how do you take a someone who's in a deep committed attachment relationship, make them realize the whole thing was an illusion and step them out of it. It's a grieving process, it's a worldview changing process. In this case, it's also an attention thing because a lot of their attention was going there. It's an identity reclaiming process because the identity that someone's taken on is partially driven and co-opted by the social associations like to rediscover their identity outside of that correct. The main advice to give, especially if you're with somebody is mostly when you're with people in difficult states, it's important to keep the door open, which means don't get into a situation where you give them an ultimatum or get into a situation where you dehumanize or get into a situation where you have now cut off your ability to remain in contact with them. Unless you are at risk because they become violent or something, but even though it's scary and even though you don't want to face it, it is often important to just stay in it with them long enough to keep the communication chains open. That's the first thing. Because there's a tendency when this happens to get extreme and dismissive and to make demands and to try to make something like an intervention happen, but this is going to be something where you have to keep a relationship of trust and be able to slowly, like a culty programming or getting someone out of abusive relationships, slowly reveal to them what the patterns of behavior were slowly reveal to them, the way they were getting played, provide more context, create distance. So you do want to have a long period of time where they're not in touch with it, but that won't be the same type of detox as it would be for attention, right? So for attention hacking. So this is me speculating and honestly just sending prayers and support to anyone who is struggling with this because it is truly a difficult thing. I just want to say that while this conversation might sound really depressing to a lot of people to just sort of really hear the degree of the problem that we've been walking ourselves into, I actually find it hopeful because we can have this elimination of the areas of psychology that we need to be protecting. We can actually walk clear-eyed into saying we cannot just roll out mass attachment hacking at scale in ways that we already see the early warning shots of where this is going. This conversation is optimistic because it's showing us there's a different way we can do this, we can do tutoring differently, we can do education differently, therapy differently, and the wisdom is understanding what these underlying sort of commons is that we need to protect our in advance of having disrupted them. We didn't see that we were screwing up the attention commons of humanity before we just steamrolled and fracked the thing down to nothingness. And here we have the opportunity, even though we're encroaching on the attachment commons, there's an opportunity to kind of get this right. And so Zach, thank you so much for this conversation, for coming on the podcast, and I really hope people check out all your work. It's really fundamental. You've got lots of other great interviews online where you go probably in other detail on other aspects, but grateful for what you're doing in the world and what you stand for. Thanks so much, Zach. I hear John, that's great to be able to speak to this with you guys. Hey everyone, thank you so much for listening to the show today. So if we're going to do something about this growing problem of AI-related psychological harms, we're going to need to understand the problem even more deeply. And in order to do that, we need more data. So if you or someone you know have had experience with an AI-related psychological harm, you know a family member or a friend who's gone off the deep end from talking to AI, or someone with episodes of psychosis that you'd like to share, you can visit the website for AI Psychological Harms Coalition at aiphrc.org, which is overseen by researchers at the University of North Carolina at Chapel Hill. And the goal of this project is to really just better understand the problems that people are having with AI chatbots. And what we need to look out for and ultimately better prevent. So we've also included a link in our show notes. And it's important to stress that this website is not a crisis support line. If you or someone you know is in distress, you can always call the National Helpline in the US at 988 or your local emergency services. Thanks for listening. Your undivided attention is produced by the Center for Humane Technology. We're a nonprofit working to catalyze a humane future. Our senior producer is Julius Scott, Josh Lash is our researcher and producer, and our executive producer is Sasha Fiegan. Mixing on this episode by Jeff Sudaken, an original music by Ryan and Hayes Holiday. And a special thanks to the whole Center for Humane Technology team for making this show possible. You can find transcripts from our interviews and bonus content on our substack and much more at humanetech.com. And if you'd like this episode, we'd be truly grateful if you could rate us on Apple Podcasts or Spotify. It really does make a difference in helping others join this movement for a more humane future. And if you made it all the way here, let me give one more thank you to you for giving us your undivided attention.