In a significant development for medical education, researchers from Juntendo University in Japan have demonstrated that generative artificial intelligence (AI) can effectively evaluate clinical interviewing skills, traditionally assessed by experienced clinicians. Published on February 17, 2026, in the journal JMIR Medical Education, the study led by Dr. Hiromizu Takahashi and Professor Toshio Naito examined the efficiency of AI-based assessment (ABA) compared to human-based assessment (HBA), addressing a critical challenge in medical training.
Clinical interviewing is foundational for accurate diagnosis and effective patient care. However, the evaluation process is often labor-intensive, requiring repeated observations and detailed feedback. As medical education expands, the burden of assessment has become increasingly challenging. “Our central message is that AI may help make medical training fairer, faster, and more scalable,” Professor Naito stated.
The research team designed a cross-sectional validation study involving seven participants—medical students, resident physicians, and attending physicians—who conducted clinical interviews with an AI-simulated patient presenting with bilateral leg weakness. These interactions were recorded and converted into transcripts, which were evaluated using the Master Interview Rating Scale, a standardized tool assessing various communication aspects, including information gathering and empathy. The transcripts were analyzed through AI models, specifically GPT-o1 Pro and GPT-5 Pro, and concurrently reviewed by five experienced clinical instructors.
Results indicated a strong agreement between the AI evaluations and those of the clinicians, with only minimal score discrepancies. Notably, the AI’s assessments were more consistent across repeated evaluations, and the time spent on evaluating each transcript was reduced by over half. “Rather than replacing teachers, this research suggests a practical ‘AI-first, faculty-verified’ model in which AI handles the first pass and educators focus their time on coaching, judgment, and high-stakes decisions,” Dr. Takahashi explained.
This advancement has significant implications for medical education. In many training programs, delays in feedback limit opportunities for students to refine their communication skills. The ability for students to receive rapid, consistent evaluations could make repeated practice more accessible, especially in environments with constrained faculty resources. “Students could interview an AI-simulated patient and receive feedback almost immediately instead of waiting days or weeks,” Professor Naito added, emphasizing the potential for enhancing learning experiences.
However, the researchers caution that AI must be employed judiciously. While the technology performed well in their study, it was based on a limited participant pool and a singular clinical scenario. Furthermore, transcript-based evaluations cannot capture critical nonverbal cues, tone, or cultural nuances that play a vital role in real-world patient interactions. “AI should be used with human oversight, because text-only scoring can miss nuances such as tone, nonverbal communication, and cultural context,” both researchers noted.
This study highlights the increasing role of AI in medical education, suggesting that integrating the speed and consistency of AI with the expertise of clinicians could create more efficient and scalable training systems. As demand for high-quality medical education rises, such approaches may be essential in ensuring that future medical professionals receive optimal training while alleviating the workload for educators.
Source: Takahashi, H., et al. (2025). AI- vs Human-Based Assessment of Medical Interview Transcripts in a Generative AI–Simulated Patient System: Cross-Sectional Validation Study. JMIR Medical Education. DOI: 10.2196/81673. https://mededu.jmir.org/2026/1/e81673
See also
Mark Zuckerberg Joins Meta’s AI Lab, Actively Coding with Engineers to Boost Innovation
Sygaldry Technologies Secures $139M to Build Quantum AI Servers for Efficient Data Centers
Harbin Institute and Peking University Launch Adaptive-SN2N Framework for Noise Reduction in Microscopy
Google’s AMIE AI Achieves Doctor-Level Diagnostic Insights in Urgent Care Study
Norm Ai Launches Legal AGI Lab to Shape AI Governance in High-Stakes Industries
