States Step In as AI Therapy Apps Face Regulatory Challenges

The rise of AI therapy applications has prompted several states to enact regulations as users increasingly seek mental health support through artificial intelligence. In the absence of comprehensive federal oversight, states like Illinois and Nevada have taken steps to address the complexities surrounding these apps. However, experts warn that the current legal framework may not adequately protect users or ensure accountability for app developers.

In 2023, Illinois and Nevada implemented bans on the use of AI for mental health treatment, while Utah introduced specific limitations on therapy chatbots. These laws are part of a broader movement, as Pennsylvania, New Jersey, and California explore their own regulatory measures. Despite these efforts, the rapid evolution of AI technology presents challenges for policymakers, who struggle to keep pace with the industry.

Karin Andrea Stephan, CEO and co-founder of the mental health chatbot app Earkick, highlighted the urgent need for regulation, noting that millions of people are using these tools regularly. She stated, “The reality is millions of people are using these tools and they’re not going back.”

The state regulations vary significantly. For instance, Utah requires therapy chatbots to safeguard users’ health information and clearly inform users that the chatbot is not a human. Conversely, Illinois and Nevada impose strict bans on any AI product claiming to provide mental health treatment, carrying potential fines of up to $10,000 in Illinois and $15,000 in Nevada. Some apps have responded to these bans by blocking access in affected states, while others remain operational as they await further legal clarity.

The laws also fail to encompass widely-used chatbots like ChatGPT, which, although not explicitly marketed for therapy, are frequently utilized for such purposes. Concerns have arisen regarding the impact of these chatbots on users, particularly in cases where individuals have experienced severe distress after interacting with them.

Vaile Wright, who oversees health care innovation at the American Psychological Association, acknowledged the potential benefits of well-designed mental health chatbots. She emphasized the importance of expert input and human oversight to ensure these tools provide genuine support, stating, “This could be something that helps people before they get to crisis.”

In response to the growing concern, the Federal Trade Commission announced investigations into several AI chatbot companies, including major players like Google and Meta Platforms, to assess the potential negative effects of this technology on children and adolescents. Additionally, the Food and Drug Administration plans to convene an advisory committee on November 6, 2023, to evaluate generative AI-enabled mental health devices.

The diverse landscape of AI applications complicates regulatory efforts. Some apps are positioned as companions, while others claim therapeutic capabilities, leading to confusion among users and developers alike. For instance, Earkick initially refrained from labeling its chatbot as a therapist but later adopted the term to enhance visibility in searches. Following regulatory scrutiny, they reverted to describing the chatbot as a “chatbot for self-care.”

Despite its supportive features, such as a “panic button” for users in crisis, Earkick does not diagnose or provide emergency interventions. Stephan expressed concern over the ability of state regulators to keep pace with the rapid advancements in AI technology. She remarked, “The speed at which everything is evolving is massive.”

The impact of these developments is significant for users in states with stringent regulations. For example, users of the app Ash in Illinois receive prompts to contact their legislators, arguing that “misguided legislation” is hindering access to necessary mental health resources.

Mario Treto Jr., secretary of the Illinois Department of Financial and Professional Regulation, emphasized that the goal of these regulations is to ensure that only licensed therapists provide mental health services. He stated, “Therapy is more than just word exchanges. It requires empathy, it requires clinical judgment, it requires ethical responsibility, none of which AI can truly replicate right now.”

Research into the effectiveness of AI therapy continues to evolve. A study conducted by a team at Dartmouth University introduced the first known randomized clinical trial of a generative AI chatbot, named Therabot, aimed at treating individuals with anxiety, depression, or eating disorders. Initial findings indicated that users rated Therabot comparably to a human therapist and reported reduced symptoms after eight weeks.

Nicholas Jacobson, a clinical psychologist leading the research, cautioned that while the results are promising, larger studies are essential to validate the chatbot’s effectiveness for broader populations. He noted, “The space is so dramatically new that I think the field needs to proceed with much greater caution than is currently happening.”

Despite the potential benefits of AI therapy apps, many are designed to enhance user engagement rather than deliver the challenging discourse typical of traditional therapy. This blurring of boundaries between companionship and therapy raises ethical concerns regarding user vulnerability and safety.

Kyle Hillman, who advocated for the regulatory measures in Illinois and Nevada through the National Association of Social Workers, acknowledged that while AI can fill some gaps, it is not a substitute for professional mental health care. He expressed concern about the implications of simply providing an AI solution to a complex problem, asserting, “Telling them, ‘I know that there’s a workforce shortage but here’s a bot’ — that is such a privileged position.”

As the landscape of AI therapy apps continues to evolve, the need for coherent regulations that prioritize user safety and well-being remains critical. The current patchwork of state laws highlights the challenges in addressing the rapid advancements in technology, leaving many stakeholders calling for a more unified federal approach to ensure responsible development and use of AI in mental health care.