Artificial intelligence (AI) is reshaping the health care landscape, presenting both opportunities for innovation and complexities in regulatory oversight. In the absence of a comprehensive federal framework, state governments have increasingly taken the initiative to regulate AI applications within health care. This shift comes as organizations must adapt to a rapidly evolving legal environment, particularly as regulatory scrutiny intensifies. As the landscape continues to unfold, health care providers and companies are urged to prepare for anticipated changes by 2026 and beyond.
In 2025, many states enacted legislation aimed at governing AI’s use in health care, particularly concerning health insurance claims adjudication, clinical decision-making, and the operation of AI-powered chatbots. Texas, for instance, passed legislation prohibiting utilization review agents from employing AI to make adverse determinations, while Maryland enacted similar restrictions in its utilization review processes. These measures aim to ensure that human oversight remains integral in decision-making involving patient care.
Legislation also focused on AI’s role in clinical decision-making. In Illinois, lawmakers prohibited the use of AI in developing mental health treatment plans or direct patient interactions, allowing it instead for administrative support tasks. This approach highlights a growing consensus among states to confine AI to auxiliary roles, ensuring that licensed health care professionals continue to play a vital role in patient care.
The use of AI-powered chatbots has also drawn attention from state legislators. Several states have introduced laws mandating transparency regarding the functionality of these technologies, as concerns grow over their potential to misrepresent human clinicians. In California, legislation prohibits the use of terms that imply an AI system operates as a licensed health care provider, empowering state authorities to act against violations. This includes a broader review of existing state AI laws in light of President Trump’s Executive Order from December 2025, which advocates for a more laissez-faire approach to AI regulation.
The lack of a cohesive federal strategy has led to a patchwork of regulations. This regulatory ambiguity raises concerns, particularly as the Department of Justice (DOJ) and other federal entities ramp up their scrutiny of AI in health care. Enforcement risks have emerged, particularly related to the False Claims Act (FCA). Recent prosecutions have highlighted the potential for AI-generated fraudulent activity, underscoring the need for health care organizations to adopt robust compliance measures.
Federal agencies are not only regulating AI but also using it as a tool for enforcement. The Centers for Medicare & Medicaid Services (CMS) has developed the WISeR Model, which will incorporate AI and machine learning to improve oversight of certain outpatient services. This initiative reflects a broader trend toward utilizing technology to enhance fraud detection and streamline enforcement processes. The DOJ has also initiated a Health Care Fraud Data Fusion Center aimed at leveraging AI for identifying emerging fraud schemes, indicating a significant shift in how health care fraud is investigated.
As medical associations and accrediting bodies issue guidance on the responsible use of AI, health care organizations must navigate the dual challenge of promoting innovation while ensuring compliance. The Joint Commission and the Coalition for Health AI recently provided frameworks for ethical AI deployment, emphasizing governance and oversight. Similarly, the American Medical Association has highlighted the importance of a human-centered approach to mitigate bias and secure patient data.
Looking ahead, health care organizations are encouraged to lead with human oversight by establishing dedicated AI governance committees and implementing ongoing monitoring processes. Transparency will be critical as organizations communicate the limitations and risks associated with AI technologies. Staying abreast of federal and state regulations will be essential, especially as the DOJ’s AI Use Case Inventory provides insights into analytical tools that can be employed for internal audits.
As AI continues to transform health care, the balance between innovation and regulatory compliance will be increasingly challenged. The evolving legal landscape calls for proactive measures to ensure that patient safety and data protection remain paramount as these technologies advance. For stakeholders in the health care sector, adapting to these changes is not just a matter of compliance, but a necessary step toward securing a sustainable future in a rapidly changing environment.
See also
90% of Compliance Leaders Adopt AI Tools, Yet Governance Remains a Challenge
South Korea Enacts AI Safety Laws, Poised to Surpass EU Amid Regulatory Shifts
Nasscom Reports Responsible AI Progress Amid Ongoing Data and Regulatory Challenges
Oregon Lawmaker Proposes AI Companion Regulations to Safeguard Youth Mental Health
