By Ian Karbal | Pennsylvania Capital-Star
Amid growing concerns over unregulated artificial intelligence chatbots, Pennsylvania state senators have nearly unanimously passed a bill aimed at introducing regulations for AI companion services. The legislation, sponsored by Sen. Tracy Pennycuick (R-Montgomery), was approved on Tuesday and seeks to implement limited safeguards for operators of chatbot services designed to simulate human relationships.
The proposal establishes specific requirements for chatbot operators, particularly when there is reason to believe that a user is a minor. However, the bill notably does not mandate explicit age verification measures, a point that has raised questions among critics. A bipartisan group of lawmakers co-sponsored the bill, with only Sen. Doug Mastriano (R-Franklin) opposing it.
“These systems are designed to simulate human interaction, and for vulnerable users, they can feel very real,” Pennycuick remarked. “When a young person begins to rely on a machine for emotional support without proper safeguards in place, the consequences can be devastating.”
Pennycuick highlighted ongoing lawsuits against AI companies, which have faced scrutiny for allegedly contributing to incidents of suicide and self-harm among users. Under the new measure, AI companion operators will be required to take steps to prevent their services from promoting self-harm, suicide, or violence. They will also be obliged to provide users with resources such as crisis hotline numbers if users discuss thoughts of self-harm.
Furthermore, the bill mandates that operators publish protocols related to these safeguards on their publicly accessible websites. If an operator suspects that a user is a minor, they must notify the user that the system is not human, do so every three hours, and advise them to take breaks from the service.
In cases where operators know a user is a minor, they will be prohibited from generating sexually explicit content or encouraging sexually explicit actions. Chatbots aimed at minors must also inform users that the service may not be suitable for individuals under 18.
The enforcement of the new law would fall under the jurisdiction of the state Attorney General’s office, with civil penalties of up to $10,000 imposed on operators who fail to comply.
After the bill’s passage, Sen. Pennycuick expressed gratitude to her colleagues for their support, emphasizing the real dangers associated with overreliance on artificial intelligence. “The potential dangers tied to the overreliance on artificial intelligence are very real,” she said. “We must do everything we can to keep children and vulnerable residents away from these toxic online interactions.”
The proposed legislation still requires approval from the Democrat-controlled House and must be signed by Gov. Josh Shapiro to take effect. Shapiro has previously indicated his intent to explore ways to regulate AI chatbot services, aligning with the growing recognition of the need for oversight in this rapidly evolving sector.
See also
OpenAI’s Rogue AI Safeguards: Decoding the 2025 Safety Revolution
US AI Developments in 2025 Set Stage for 2026 Compliance Challenges and Strategies
Trump Drafts Executive Order to Block State AI Regulations, Centralizing Authority Under Federal Control
California Court Rules AI Misuse Heightens Lawyer’s Responsibilities in Noland Case
Policymakers Urged to Establish Comprehensive Regulations for AI in Mental Health
