Development and Validation of Delirium Recognition Using Computer Vision in Neuro-critical Patients

Overview

This research project employs machine learning algorithms integrated with computer vision, image processing, and pattern recognition technologies to perform digital analysis of facial expression behaviors in neurocritical care patients with delirium. By constructing multidimensional high-level features of delirium, the investigators have established a classification model based on behavioral. The primary objective of this study is to address the critical challenge of achieving precise and efficient delirium diagnosis in neurologically critically ill patients through automated facial expression behavior recognition.

This study is a prospective cohort study approved by the Ethics Committee of Beijing Tiantan Hospital. It aims to support the accurate and efficient diagnosis of delirium in neurocritical patients through a facial expression recognition system. A mobile application was developed for this study, collaboratively designed by senior clinicians and engineers from the Institute of Computing Technology, Chinese Academy of Sciences. The application is based on a stimulus paradigm designed using CAM-ICU (Confusion Assessment Method for the Intensive Care Unit) questions to record dynamic facial videos of neurocritical patients following delirium evaluation based on the DSM-V criteria. Patients were assessed for delirium and facial expression behavior data were collected twice daily during ICU admission, in two time slots: 8:00-10:00 AM and 8:00-10:00 PM, following the study's inclusion and exclusion criteria. A trained and experienced specialist used the gold standard DSM-V to diagnose delirium. Within five minutes after completing the assessment, dynamic facial behavior video data were collected to prepare images for subsequent model development. Various image preprocessing and data augmentation techniques were employed to prepare the images for the VGG16 model. These techniques are standard for running convolutional neural network (CNN) models. Using the "preprocess\_input"function from the Keras VGGFace module, the investigators standardized image color and size to ensure that each image met the expected input requirements for model training. For data augmentation, the investigators applied TensorFlow's "ImageDataGenerator" function to perform horizontal flipping, rotation, scaling, width and height shifting, and shearing. These augmentation techniques created a more diverse dataset, helping to prevent overfitting and improving the model's generalizability to new faces. The investigators developed a binary classification model to identify delirium using a CNN with a pretrained backbone. The VGG16 model, based on deep learning, was adopted, leveraging transfer learning from VGGFace2, which possesses pre-existing facial feature recognition capabilities. Transfer learning allowed us to utilize prior knowledge to detect features more quickly, accurately, and with lower computational cost. The VGGFace2 model was employed for training. Model performance was evaluated through internal validation at Beijing Tiantan Hospital and external validation at Guiyang Second People's Hospital, with metrics including accuracy, sensitivity, specificity, and F1 score. Additionally, to address the "black box" issue of machine learning, occlusion heatmap techniques were used to identify the most critical facial regions for delirium assessment, with the results visualized on a virtual face. This model aims to support precise and efficient identification of delirium in neurocritical care units.