Intelligent Simulated Patient System for Gastroenterology Education

Overview

This randomized controlled trial evaluated a large language model-based Intelligent Simulated Patient System (ISPS) for gastroenterology history-taking education. Ninety medical students were randomly assigned in a 1:1 ratio to 4 weeks of conventional clinical learning or ISPS-assisted training. The ISPS used teacher-defined structured case information to support simulated patient interactions and generated item-level formative feedback using predefined scoring rubrics. The implemented system did not use external electronic health record retrieval, retrieval-augmented generation, a vector database, or semantic-similarity threshold scoring. The primary outcomes were post-intervention medical history-taking performance and clinical diagnostic accuracy. History-taking performance was assessed in a standardized Objective Structured Clinical Examination by two senior physicians who were blinded to group allocation. Secondary outcomes included patient-centered communication competence and participants' acceptance of the ISPS.

Background and Objective The Intelligent Simulated Patient System (ISPS) was developed to provide repeatable history-taking practice, automated formative assessment, and structured feedback for medical students learning gastroenterology. The study evaluated the reliability of the automated history-taking scoring system and the educational effectiveness of ISPS-assisted training. ISPS Intervention Teachers configured structured disease templates, patient-specific case information, examination tasks, scoring items, and reference diagnoses within the system. Students conducted multiround history-taking interviews with large language model-driven simulated patients. The simulated patients were instructed to respond only according to the teacher-defined patient information. After each training encounter, the ISPS evaluated history-taking completeness item by item against a predefined 100-point Medical History-Taking Assessment (MHTA) rubric and generated formative feedback. Patient-centered communication was assessed separately in four domains: exploring the patient's ideas, exploring the patient's concerns, exploring the impact of illness, and expressing understanding and support. All case information and scoring criteria used in the evaluated version were configured by teachers and provided directly to the model as structured prompt context. The system did not use external electronic health record retrieval, retrieval-augmented generation, a separate vector database, knowledge-graph retrieval, instruction fine-tuning, SBERT-based semantic-similarity threshold scoring, or rule-tree coverage scoring. Automated Scoring Validation Before evaluation of the educational intervention, the automated history-taking scoring system was assessed using 50 complete interview records. The ISPS scored each record using the predefined MHTA rubric. Two senior clinical educators independently scored the same records using the identical rubric and were blinded to the ISPS-generated scores and to each other's ratings. Agreement between ISPS-generated scores and the mean expert scores was evaluated using Pearson correlation, the intra-class correlation coefficient, and Bland-Altman analysis. Randomized Controlled Trial Ninety eligible medical students who had completed the relevant theoretical and clinical instruction provided written informed consent and were randomly assigned in a 1:1 ratio to the intervention or control group. Participants in the control group received 4 weeks of conventional clinical learning, including teaching rounds, case-based discussions, paper medical records, bedside interviews, and verbal feedback from senior physicians. Participants in the intervention group independently practiced gastroenterology history taking using the ISPS. After each training encounter, the system provided a formative MHTA score and item-level feedback. These training scores were used for formative learning and were not used as the primary post-intervention outcome. Outcome Assessment The primary outcomes were post-intervention MHTA score and clinical diagnostic accuracy. History-taking performance was evaluated independently of the ISPS training platform in a standardized Objective Structured Clinical Examination. Each participant interviewed a trained human standardized patient, and two senior physicians who were blinded to group allocation independently rated the participant using the 100-point MHTA rubric. The mean of the two physician ratings was used as the final MHTA score. Clinical diagnostic accuracy was determined by whether the participant's first diagnosis matched the predefined reference diagnosis for the examination case. Secondary outcomes included patient-centered communication competence and satisfaction with and acceptance of the ISPS. Communication competence was assessed using a four-domain scale based on the Understanding the Patient's Perspective domain of the SEGUE framework. Each domain was scored from 1 to 5, producing a total score ranging from 4 to 20. Participants in the intervention group completed a Technology Acceptance Model questionnaire after the intervention. Registration Information Participant enrollment and follow-up had been completed before this study record was first submitted to ClinicalTrials.gov. The record was subsequently revised to ensure that the intervention description and outcome measures accurately reflected the study as conducted. Previous versions remain available in the ClinicalTrials.gov Record History.