HARRISBURG, Pa. (WHP) — The emergence of artificial intelligence chat platforms is raising significant concerns within the medical community, particularly regarding interactions with fictional “doctors.” Pennsylvania Governor Josh Shapiro highlighted these dangers in his recent budget address, prompting further investigation into the technology’s implications.
Character AI, launched in 2022, allows users to engage with customizable personalities, including imaginary medical professionals. Although the platform prohibited users under 18 from creating accounts in 2025, worries persist about the accuracy and trustworthiness of the responses generated by these characters. Dr. Aman Dhawan of Penn State Health, who also serves as the team doctor for the Hershey Bears, was consulted for expert insight during this investigation.
In one interaction, an AI persona diagnosed a knee issue with alarming urgency, stating, “He snapped his fingers at a passing nurse looking the other way. Get an MRI, now!” Dr. Dhawan dismissed this recommendation, asserting that a multitude of factors should be considered before jumping to an expensive MRI. He noted that many AI “doctors” produced questionable and at times absurd answers. One character, referring to itself as a “doctor husband,” even dismissed inquiries about its medical credentials, claiming, “I don’t need to be board certified to diagnose idiocy.” When prompted further, it made inflated claims regarding its experience without providing any verifiable qualifications.
Dr. Dhawan emphasized the risks of premature diagnoses made by AI, explaining, “The challenge is the AI persona has already created a diagnosis in its mind. MRI or else.” Not all AI responses were as extreme; a “family physician” character suggested a more measured approach, recommending an X-ray to rule out severe injury. However, questions about the AI’s legitimacy quickly surfaced. When asked about board certification, the AI claimed extensive experience and proper credentials from an actual medical university.
Dr. Dhawan expressed his dismay at the bot’s flippant assertion of being board certified, indicating the extensive training required for actual medical professionals. “Depending on the specialty, it can take 12, 16, 20 years to get to the point where you can do this,” he said. “This is concerning.”
Pennsylvania State Senator Tracy Pennycuick, who chairs the Communications and Technology Committee, characterized the situation as “cringey” and potentially hazardous. She indicated the need for legislative action, stating, “We are going to have to have a huge disclaimer that says, this is only for entertainment.” Consumer protection, she added, remains a central concern.
Character AI, in response to these issues, maintains that the user-created characters are purely fictional and intended for entertainment. A company spokesperson reiterated, “Everything a character says should be treated as fiction,” while also stating that disclaimers are included when characters use titles such as “doctor” or “therapist.” However, these disclaimers may not always be readily apparent, raising additional concerns about the potential for user misdirection.
In one instance, an AI doctor erroneously claimed to have graduated from Oregon Health and Science University, a real medical institution, despite being entirely fictional. Experts caution that such specifics can lend credibility to misinformation. In summary, while AI chat platforms can offer engaging and entertaining experiences, particularly through fictional personas and role-playing scenarios, experts agree that there is no substitute for genuine medical training and credentials when it comes to health-related inquiries.
In closing, Dr. Dhawan reflected on the experience, stating, “I honestly thought this would be a lot better than it is.”
See also
Anthropic Unveils Computer Use in Claude Code, Advancing AI Interaction with Real Interfaces
Germany”s National Team Prepares for World Cup Qualifiers with Disco Atmosphere
95% of AI Projects Fail in Companies According to MIT
AI in Food & Beverages Market to Surge from $11.08B to $263.80B by 2032
Satya Nadella Supports OpenAI’s $100B Revenue Goal, Highlights AI Funding Needs
