The Wild West of AI Therapy Laws

I think I was in Chicago and I had a like a three hour layover and so I decided, hey, this is a good time. That's Tyler Lord. He's 36 and he's been dealing with depression and he says what it was a good time for was using an AI mental health chatbot called Ash. He figured he could get a quick session in on his layover. I probably couldn't talk in the airport, but I could type out whatever I needed. But it turns out he couldn't. As soon as I opened it, get a pop-up saying, hey, the legislators in this state have banned AI therapy, so unfortunately, you cannot use it here. I was like, oh, man, that sucks. He did a quick Google. Yup, AI tools for therapy were restricted in Illinois. And for Lord who'd been considering moving to Chicago, not having Ash was a deal breaker. The fact that I wouldn't be able to use it if I was in a state like Illinois that could have profound consequences and maybe my dream job is in Chicago and I don't know it. And I'll never know what, because I'll never move there. Because I like having the safety net that Ash provides, even if it's not a requirement, it just makes dealing with those tough moments so much easier. And so it's like, why am I going to make life harder for me by moving to a place where I don't have it anymore? Actually, the amount of time that Ash wasn't available in Illinois was pretty brief. We'll get into why it went away and why it came back a little later. But Illinois isn't the only state that's setting limits on AI therapy. Lawmakers are worried about these chatbots, so what do these laws do? And how are they changing the industry? I'm Alex Osola for The Wall Street Journal. This is What's New Sunday, in the third and final episode of The AI Therapist. So far, we've looked at why AI tools for mental health are catching on, the risks that may come with using them, and why the companies behind the chatbots say they're safe. You should go back and listen to those two episodes if you haven't yet. Today, we'll be talking about what the law says about how people should be using these tools, where some say it falls short and the future of AI tools for mental health. Let's travel back in time for a second to July 2025. Okay, that was not a very long journey. It was only about a year ago. A startup called Slingshot AI was launching its AI chatbot Ash. And Slingshot CEO Daniel Khan was busy promoting it with a video. I'm happy to share that today, we're announcing the launch of Ash, the first AI designed for therapy. We built a world class AI designed for therapy. That turned out to be a significant word choice. If you go to Ash's website today, the tagline says AI designed for mental health. It doesn't say therapy. I asked Slingshot co-founder Neil Perique about it. He said the company changed the wording on its website last fall. And they did it because of state laws that say something can't be called therapy unless a licensed professional is doing it. But he says that doesn't change Ash's mission. You don't call it AI therapy, right, but when we think about the value proposition to people is similar, which is like you have a problem, you can show up and Ash is there to help you think through that. So Ash might be therapy in everything but name, but some companies embrace the label. On its website, one startup, Harmony calls itself the first emotionally intelligent AI therapist. And still others say that actually they're not therapy at all. For example, Talkspace, it's a publicly traded company that connects people to mental health providers online. Earlier this summer, it launched an AI tool called T and John Cohen, the CEO of Talkspace, made it very clear to me. T is not therapy. So it's a different population of people seeking a different type of conversation than a person who's actually seeking Talkspace for therapy. It's a different leap. I say, I've recognized I have a problem, whatever it is, I really want professional help. These are people who haven't said they want professional help. They just want to talk to somebody. It's different. So why are some of these companies touchy about the word therapy? One big reason as Lingshot told us is that several U.S. states are passing laws to restrict the use of AI therapy. That's what Tyler Lord ran into in Illinois. Last year, the state's governor, JB Pritzker, signed a law prohibiting AI therapy. There wasn't much disagreement about this among legislators. The state house and senate both passed the bill unanimously. Simply put, the law requires that mental health services be provided by humans. The man in charge of enforcing this law is Mario Treto, Jr. He's the secretary of the Illinois Department of Financial and Professional Regulation. The purpose of it was to ensure that we're protecting consumers and setting the appropriate guardrails and rapidly evolving industry. There's a reason why we have licensed therapists, licensed clinical social workers that provide this critical care. Really what we're wanting to make sure is that we have a licensed individual providing the mental health services. We want to make sure that there's a human in the loop. Other states have recently passed laws limiting AI for therapy, such as Nevada, Tennessee, Maine, Colorado, Vermont, and Rhode Island, those last three just this summer. And lawmakers in other states, including New York and California, are considering similar legislation. Michelle Mello is a professor of health policy at Stanford's Medical School. She's also a law professor, and she's been following the rollout of this legislation. Broadly speaking, she says it's early dates for laws like these. It looks like a lot of early experimentation among people who are trying hard to be helpful, but still figuring their way. To her, these laws fall into a few different categories, starting with the kind of AI they address. And there I would distinguish between the three types, the therapy bots that are built for the purposes of providing mental health care, general stuff like chat GPT, and then the companion bots that are built to provide social and emotional support for mental health care, most of the laws target the first thing that therapy bots. And then the second thing is what is it doing, and for whom. And there we see a lot of states and the Congress focusing on protection of children and teenagers. And then there's the question of enforcement. The way it works in several of these states is that someone has to complain that a company is violating the law before regulators get involved. Mello says that's pretty much the only way for governments to know what's going on. Mello says it has received consumer complaints related to the AI therapy law, but it hasn't issued any fines, at least not yet. We asked other states with similar laws about this too, and none of them from what we found have find any companies. Still, chatbot makers are changing their approach. We'll get into how, after the break. So, a growing number of states are implementing laws to limit how AI can be used for mental health treatment, not everyone's a fan, particularly some of the scientists and companies behind those AI mental health tools. Take Nick Jacobson. He's an associate professor of biomedical data science, psychiatry, and computer science at Dartmouth. We started working on trying to use generative AI to provide evidence-based mental health care starting in 2019. He's been developing his own AI therapy tool called TheraBot, but it's still being tested and is not available to download for just the average person. Jacobson says that it's the general use AI chatbots like ChatGPT that are linked to psychosis. You heard about that on our earlier episodes, but he thinks that the state laws target specialized mental health apps more, which means he says that these state bans don't actually do what they set out to. They don't often have exemptions for these clinical products that are effective and safe, so that. Illinois's regulator, however, did tell us that the law applies to any company that provides therapy. Michelle Mello, the Stanford law professor, says chatbots designed for therapy do tend to be the one's most subject to state regulation. Some, not all, but some of the therapy bots have worked really hard to share out evidence through rigorous study that they produce better patient outcomes or comparable patient outcomes to human care. They're the ones that are easy to regulate because they're tied into structures that we already have for regulation. So we told you about Tyler Lord at the top of the show. He's the one who couldn't use ash during the layover in Chicago last year, but earlier this month, we asked one of our Wall Street Journal colleagues in Illinois to see if he could use ash, and it turns out he could. I asked Neil Perique, the co-founder of Slingshot, why ash is now available in Illinois. He said that ash went dark for a few months in Illinois and Nevada, while Slingshot figured out how these new laws applied. But remember what I mentioned earlier about how Slingshot did away with the word therapy on its website? After making that change, the company restored access to its services. Some markets have proven trickier to navigate. In January, Slingshot pulled ash from the United Kingdom, setting concerns about how regulation works there. The country doesn't have a ban on AI therapy tools, but it does have rules around how AI can be used for medical purposes. Other companies say they'd like to see more federal regulation here in the US. John Cohen, the CEO of Talkspace, says it's tool T. complies with the laws in all 50 states. In fact, he says federal regulation would actually help T. - I would love to see a federal solution so that we don't have to deal with 50 different laws and states and regulatory oversight that's different from state to state. I'm not worried about what we built. I think other people worried about what they built because they don't have the safeguards that we put in place. What honestly? Coming up, if you can find in an AI therapist, are your chats private? That's after the break. (upbeat music) So, AI therapy restrictions are becoming more common, but that still leaves a lot of gray areas. Like, what about apps that say they're wellness tools or coaching tools? Through apps like these, which is most of them, there are very few rules in place. They don't have to work with clinicians to come up with guardrails, and they don't have to have a human in the loop for potentially risky conversations. In our reporting about the regulatory landscape, there was one phrase that kept coming up. - On the regulation front is quite the Wild West. - There's no real regulation in this space because it's kind of the Wild West. - It gets over said a little bit of the Wild West. - One area where experts said more regulation is needed is privacy. They'll write senior director of healthcare innovation at the American Psychological Association, told me it's definitely something users should be thinking about. - It's important to remember that these chats are not private, and data breaches are 100% not preventable. So, if you don't want somebody to know what you're chatting, if it's something that you feel too sensitive for, say, your family or your boss or somebody to know about, you may want to think twice about putting it into these chatbots. John Cohen, the CEO of Talkspace, talked a lot in our interview about how T protects user privacy. But on one key topic, he kept correcting himself. - We built T with HIPAA, HIPAA-type protections on it. - HIPAA is the longstanding federal law that safeguards patient medical data. I asked him about the distinction he was making between HIPAA and HIPAA-like. - The law doesn't provide us to say that this is a HIPAA-protected delivery because it's not a medical device or a medical person. So, what we did is we used all of the issues around how you make something HIPAA-protected and apply it to that. But I can't say it's HIPAA-protected because HIPAA only applies to live human interaction. - So, T promises basically the same kind of privacy protections you would get if you went to your doctor. Among other things, not selling your data to another company without your permission. Many of these tools say user data is protected via encryption, but they also have optional settings where they save user conversations to train their models. John Torres, a psychiatrist at Boston's Beth Israel Deaconess Medical Center, says this presents a risk that users might not fully realize. - You're probably not being given clear, direct informed consent, where you are explained to risks and benefits and what happens to your data. And that is not gonna be a bright spot just for the field of how these companies built these models. But for many of us, privacy is just one of those worries that tends to fall by the wayside. You could say it's a casualty of life spent interacting with the internet. That's how Tyler Lord thinks about it. He's the man from Kentucky that you heard at the beginning of the episode. He says what he gets out of talking to Ash is worth the risk of a potential leak of his data. - Every, I don't know a couple of minds who you hear about some big company that had some data breach and a ton of stuff is leaked. - It's always gonna be a risk, but to me it's worth the risk because the benefits would outweigh the possibility of that risk 'cause I know it's not a guarantee thing but I'm kinda guaranteed at least from my experience to get help from the app. - I asked another Ash user, Jennifer Millier, a 45 year old attorney living in Los Angeles, whether privacy was something she worried about when using the app. - As a lawyer, I should say yes, as a person with an iPhone and a TV and a laptop and Alexa and all of the things, I figure all of my personal information is already out there so I'm pretty open about it. - So for now, regulation may be a wild west but the federal government has started making some early moves. - Good morning. I would like to call this meeting of the FDA's Digital Health Advisory Committee to order on November 6th, 2025. - In November, the FDA's Advisory Committee for Digital Health held a meeting about generative AI-enabled tools used for mental health. The FDA makes a distinction between general use AI tools like ChatGPT or Clawed and those that use AI to do things like diagnosis and treat mental health conditions. Here's how Michelle Tarver, the director of the FDA's Center for Devices and Radiological Health, described the goal of the meeting in her opening remarks. And yes, for these purposes, the FDA considers chatbots medical devices. The FDA has authorized over 1200 AI-enabled medical devices. These innovations help strengthen healthcare and provide diagnosis, cures and treatment to patients for a range of diseases and conditions. However, to date, there are no generative AI-enabled medical devices that have been authorized for use in mental health conditions. - Over the course of the eight-hour meeting, participants hashed out many of the issues we've talked about in this series, like AI sycophancy, bots failing to escalate serious issues and data privacy. Still, though, the FDA is allowing some experimentation in small ways. Last month, it announced the participants in a new pilot program for medical devices. One of them is called Unpacked from a company called Limbic. It's an AI tool for people with depression or anxiety. Patients get calls from an AI voice agent and it's overseen by clinicians. If the trial goes well, Limbic says it would be allowed to make its product widely available on the market. So the jury's still out on federal rules for these AI mental health chatbots, but the companies that make them aren't waiting around. They want their apps to be woven deeper into their user's lives. Talkspace and Slingshot are both planning to find ways for users to send their sessions from the AI chatbot to their human therapist, so the counselors can see what their patients were talking about between sessions. Because Talkspace is John Cohen says, patients are doing this anyway. We hear it from psychiatrists all the time, they say, I see a patient on Tuesday, and they come into the office on Friday and say, oh, by the way, I had an issue on Thursday and I discussed it with X. Meanwhile, Slingshot wants Ash to feel even more real to its users, like by giving Ash an avatar. You can imagine a world in which you're going to FaceTime with Ash, and then you have interesting questions of, is it a human? Is it a character? Is it more like an animated character? Parik says that could add a layer of communication that users currently aren't getting with voice and text. Body language could be an important part of helping a person see like, okay, what is Ash thinking? Like, is Ash pausing or ruminating or whatever? And while Talkspace might have built a conversational AI chatbot, CEO John Cohen still says AI isn't therapy, and he's not sure AI tools will ever take its place. Therapy is not just a Q&A. If you're wanting to ask Google or ChatGPT or whatever you're going to ask them a question, and you get an answer, you know, how far is the Sun? How fast does this speed of light, right? Question, answer. Therapy is not that way. There's a significant nuance to therapy that is not just a Q&A. That's why it's so complicated. And that's what's new Sunday for August 23rd. Today's show was produced and mixed by Pierre Bienemet with supervising producer Tally Arbel, additional editorial support from Chris Zinsley. I'm Alex O'Soleff for The Wall Street Journal. We'll be back with our regular show on Monday morning. Thanks for listening.