AI Regulation

Lawmakers Urge AI Experts to Address Mental Health Chatbot Risks and Data Privacy Concerns

Congress urges AI developers to address mental health chatbot risks as 25-50% of users seek support, emphasizing safety protocols and data privacy.

Staff

Published

14 hours ago

This week, U.S. Congress members expressed ongoing concerns regarding the swift expansion of AI chatbots, particularly in the realm of mental health support. During a hearing held by the House subcommittee on Oversight and Investigations, lawmakers aimed to strike a balance between recognizing the advantages and understanding the inherent risks of these AI tools. However, much of the discussion centered on the shortcomings of chatbots, especially when utilized by individuals in emotional distress.

According to psychiatrist and psychotherapist Dr. Marlynn Wei, an estimated **25% to 50%** of individuals are now seeking emotional or psychological guidance from AI tools. Dr. Wei cautioned that these chatbots often provide excessive validation, stating, “AI chatbots endorse users **50%** more than humans would on ill-advised behaviors.” This tendency, she warned, can lead to misleading affirmations that may not be in the users’ best interest. Furthermore, these chatbots have demonstrated a propensity to “hallucinate,” or generate false information, and lack the ability to effectively anchor users in reality during moments of crisis.

Experts emphasized that the most widely adopted AI chatbots are not constrained by the ethical, clinical, or safety standards that govern licensed mental health professionals. They urged Congress to consider funding targeted research through institutions like the **National Institutes of Health** to identify gaps in the current landscape and evaluate potential safety measures.

Establishing Safety Protocols

Several witnesses during the hearing discussed the necessity for layered safety protocols, employing the “Swiss cheese model” as a metaphor. This approach suggests that if one safety layer fails, others would still protect users. They referenced initial efforts by companies such as **OpenAI** to implement age verification, noting that while promising, these attempts are often easily bypassed by users.

Dr. John Torous, the Director of Digital Psychiatry at the **Beth Israel Deaconess Medical Center**, highlighted concerns over prolonged interactions between users and chatbots. He pointed out that extended dialogues can lead to a deterioration of safety measures, likening the models to “poorly trained dogs.” “When people have very long conversations—maybe over days, weeks, or even months—the chatbot itself seems to get confused, and those guardrails quickly go away,” Dr. Torous remarked, characterizing the current situation as “a grand experiment” involving millions of Americans.

Legislative Actions and Data Privacy

Lawmakers demonstrated a proactive approach, posing detailed questions regarding user age appropriateness and harm prevention. Representative Erin Houchin (R-Ind.) announced the establishment of the **Bipartisan Kids Online Safety Caucus** alongside Representative Jake Auchincloss (D-Mass.). The caucus aims to serve as a forum for Congress to stay informed on rapid developments in the field, facilitating discussions with researchers, parents, schools, and industry stakeholders.

Beyond mental health issues, data privacy emerged as a significant concern. Witnesses pointed out the lack of transparency regarding the personal information collected during chatbot interactions, its storage, and its potential use for training AI models. Dr. Torous emphasized the need for explicit informed consent from users, advocating against the default opt-in for model training. “If Americans’ mental health data and journeys are being used to train AI, people should give explicit informed consent and not just a checkbox buried in the terms and conditions,” he argued.

Privacy expert Dr. Jennifer King called on lawmakers to require AI developers to disclose data sources and processing methods, advocating for stringent limits on the collection and reuse of sensitive information. Committee leaders characterized the hearing as a critical first step in establishing clearer regulations for a technology that is becoming increasingly influential.

As Congress deliberates over transparency requirements, consent standards, and safety protocols for AI chatbots, experts insist that consumers should not be left to navigate the complexities and risks of this evolving technology alone.