Validate: Trustworthy AI to Improve Acute Stroke Outcomes

Overview

Artificial intelligence (AI)-powered prognostic tools and clinical decision support systems can predict the outcome of certain diseases based on a multitude of patient data at high speed, facilitating decisions by healthcare professionals. In acute ischemic stroke, the overall treatment effect and population-wide outcome benefit of treatments such as IV thrombolysis and mechanical thrombectomy are well established. However, in individual patients it is difficult to predict the prognosis in the acute phase of stroke: some patients are candidates for these treatments, but may have poor clinical outcomes (no improvement of stroke or even worsening) Our aim in this study is to validate an artificial intelligence (AI)-based prognostic tool to provide accurate real-time outcome prediction in patients with acute ischemic stroke. During the study, all patients admitted to the emergency room with an acute ischemic stroke will receive the usual treatment for acute stroke in accordance with the stroke neurologists in charge. A "shadow" clinical researcher, without interaction with treating physicians, will collect the data required by the AI model in vivo. These data will be obtained by filling in clinical data through an App on a hospital mobile/tablet, and by a connection with your electronic medical record. The AI models will estimate the outcome of the acute stroke patient, and this prediction will be compared with the real outcome of the patient after 3 months of follow-up.

Artificial intelligence (AI)-powered prognostic tools and clinical decision support systems have the ability to predict the outcome of certain diseases based on a multitude of patient data at high speed, facilitating decisions by healthcare professionals. In acute ischemic stroke, the overall treatment effect and population-wide outcome benefit of IV thrombolysis and mechanical thrombectomy are well established. However, the outcome still differs significantly for individual patients, where some are eligible for treatment but may have catastrophic outcomes. Multiple prognostic variables and their combination in a single patient make it difficult to predict individual outcomes after stroke treatment. We aim to validate an AI prognostic tool to provide accurate outcome prediction in patients with acute ischemic stroke in a prospective observational follow-up study. Hypothesis AI-based models can be applied in real-time in acute stroke patients and provide an early accurate prediction of their outcome. Methodology The study complies two phases. Phase 1: retrospective study. While technological readiness will be achieved for the clinical validation study further model refinement on heterogeneous data will be performed based on existing models that have been developed on extensive high-quality medical data. VALIDATE will analyse retrospective databases from the 3 clinical sites involved in the study to test and validate the previously generated AI models. Encrypted data of all acute ischemic stroke patients admitted to the centres during 2018-2021 period will fed the AI models to validate the model's accuracy comparing the outcomes predicted by the AI modelling with those of the actual patients. The dataset includes demographics, baseline clinical characteristics, risk factors, neuroimaging data, acute treatments, clinical evaluation (National Institute of Health Stroke Scale (NIHSS)), functional evaluation at 3-6 months (mRS), patient reported outcome measures (PROMs), etc. The interaction between these data sets and the AI models will be done through a federated learning procedure, that is, the data will be analyzed on our hospital servers, and they will not be transferred to any other center. Grading the contribution of the progressively complex diagnostic procedures to the AI models and establishing a set of the minimum relevant variables for the AI model able to accurately predict functional outcome will also be achieved. Phase 3: Prospective multicenter observational shadowing study. consecutive acute stroke patients admitted to 3 high-volume comprehensive stroke centres will be evaluated. All patients will receive the usual stroke work-up and standard of care treatment according to the treating physicians. A shadow clinical researcher with no interaction with the treating physicians will recollect in vivo the data required by the AI modelling. These data will be obtained by filling of clinical data through an app and by connection with the electronical medical record of the patient to obtain additional baseline and neuroimaging data. The real outcomes of the patients will be measured through clinician and patient reported outcomes measurements (CROMs and PROMs), and they will be compared with the estimated outcomes according to the AI model. PROMS after 7 days, 1 and 3 months will be obtained through the implementation of an innovative nudging-based digital platform (NORA) to improve patient-clinician communication and follow-up. An electronic case report form (eCRF) will be designed to recollect key process indicators (KPI) and CROMs, which will be integrated in the NORA platform. The sample size calculation has been based on the results of a clinical dataset of consecutive code stroke patients admitted to Hospital Vall d'Hebron during the year 2020. It has been used as an example of the usual mRS distribution at 3 months in a cohort of consecutive acute stroke patients. In a test for agreement between two raters using the Kappa statistic, a sample size of 182 subjects achieves 95% power to detect a true Kappa value of 0,7 in a test of H0: Kappa = κ0 vs. H1: Kappa ≠ κ0 when there are three categories with frequencies equal to 0,58, 0,34, and 0,08. This power calculation is based on a significance level of 0,05 and a minimum acceptable kappa (κ0) of 0.6 (moderate agreement) and an expected kappa (κ1) of 0.8 (substantial agreement). Assuming a drop-out rate of 20% for the 90-day follow-up the Dropout-Inflated Expected Enrolment (DIEE) Number would be 218 patients with acute ischemic stroke.