Effectiveness of an Artificial Intelligent Tutoring System in Simulation Training

Overview

Brief Summary: Background: Although surgical experience and technical skill are associated with better patient outcomes, quantitating surgical ability in the operating room is challenging. In surgical education, large datasets generated by high-fidelity virtual reality simulators can be employed by machine learning algorithms to objectively measure trainee performance and competence on expert benchmarks. This allows repetitive practice of surgical skills in safe and risk-free environments with immediate feedback. Our group developed and has a patent pending for an intelligent tutoring system called the Virtual Operative Assistant (VOA). Utilizing an Artificial Intelligence (AI) support vector machine algorithm, the VOA assesses data derived from the NeuroVR (CAE Healthcare) simulator platform and provides individualized audiovisual feedback to improve learner performance during simulated brain tumor resections. The effectiveness of intelligent tutoring systems such as the VOA to the human surgical apprenticeship pedagogy remains to be elucidated. The aim of this study is to compare the effectiveness and educational impact of personalized VOA feedback to expert instruction on medical student's technical skills learning of a virtual reality tumor resection procedure. Specific Aims: 1) To assess if medical students receiving personalized VOA feedback statistically improve their surgical performance when compared to those having (a) no expert instructor feedback or (b) expert instructor-mediated feedback. 2) To outline if different emotions are elicited by the VOA intelligent tutoring system in medical students while performing this achievement task as compared to human instruction

Design: A three-arm partially blinded randomized controlled trial of VOA training versus remote-based expert instruction versus control. Setting: Neurosurgical Simulation and Artificial Intelligence Learning Centre, Montreal Neurological Institute. Participants: Eligible first- and second-year medical students from across the province of Quebec. Task: Complete removal of a simulated tumour - distinguishable by colour and haptic properties - with minimal bleeding and damage to surrounding healthy brain using two surgical instruments (Cavitron Ultrasonic Aspirator and Bipolar pincers) of the NeuroVR (CAE Healthcare) surgical simulator. Intervention: A single 75-minute training session, including six virtual subpial tumour resection attempts (five simple practice scenarios and one complex realistic scenario) with assessment and feedback from either: 1. the VOA intelligent tutoring system (Group 2) or 2. a remote-based expert instructor (Group 3) Both compared to: 3. control group (Group 1) that receives no assessment or performance feedback. To our knowledge this will be the first study to compare the effectiveness of an AI-powered intelligent tutoring system to expert instruction in the context of medical and surgical virtual reality simulation and assess the emotional response to such instruction. This study aims to begin to identify successful approaches to use this innovative technology in the medical educational curriculum and improve patient outcomes by augmenting safety, efficiency and competency of surgeons and other healthcare providers.