STOP-stroke: STroke Outcome Prediction in the Acute Treatment Setting

Overview

The STOP-stroke project aims at improving prediction of outcome early after stroke. In order to achieve this, we need to understand reasons (important variables) for prediction in a real clinical prognostication process. We aim to: 1. Test the predictive performance of stroke neurologists for outcome prediction (NIHSS at 24 hours and 3 months and mRS at 3 months after stroke onset) prospectively and in a real clinical setting, and to explore the most important baseline variables in their prognostication process. 2. Test the prediction performance of our DL models when being provided with structured clinical and/or imaging information from the same patients as the neurologists; and to discover most relevant features of the input data. 3. Use the information gained from our experiments for improving our DL algorithm. This will include an error analysis on the missclassifications of models and neurologists to understand the pitfalls of both approaches. We anticipate to develop a robust, reliable and clinically feasible application ready for testing in a prospective, observational trial.

To avoid irreversible brain damage in acute stroke, neurologists must make treatment decisions under immense time-pressure. In current clinical practice, neurologists decide using visual inspection of brain scans and established clinical parameters. Despite the large quantity of data, statistical or machine learning models have not yet reached clinical practice to guide decision-making. This is in large part because doctors cannot assess the trustworthiness of these models, and because current models cannot handle multimodal input data. Since the field of acute stroke treatments is constantly evolving, there is an urgent need to improve our understanding of the factors determining stroke patient outcome and response to treatment. We are convinced that the project will provide new insights into stroke outcome prediction and help to integrate data from machine learning algorithms into clinical routine. Neurologist's prediction of stroke outcome When a patient arrives at the stroke unit, the treating team continuously integrates information as to what the best therapeutic options or risk of complications may be. To predict patient outcomes, neurologists rely on structured tabular data, such as age, sex and blood pressure, and unstructured data, such as medical image data. Current prediction of therapeutic success in stroke patients outside the therapeutic time window is often based on brain images (Joundi et al. Neurology 97:S68-S78). Diffusion and perfusion weighted imaging (DWI/ PWI) are used to identify the infarct core and hypoperfused region, resulting in an estimate of the tissue at risk referred to as "mismatch"(Heiss WD et al. Int J Stroke 14:351-8). Large mismatch and small infarct core are considered surrogate markers for treatment success, which is why clinical trials have often pre-selected such patients (Albers GW et al. N Engl J Med 378:708-18). However, recent studies question the validity of such a pre-selection. For instance, novel, pooled data analyses indicate that patients with large infarct core benefit from endovascular treatment as well (Campbell BCW el a. Lancet Neurol 18:46-55). Moreover, studies with the pre-selected patients do not allow drawing any conclusions about the effect of treatment in patients with a lack of mismatch or large cores. It becomes increasingly clear that patients with a large core or lack of mismatch could benefit from endovascular thrombectomy treatment as well (Karamchandani RR et al. J Stroke Cerebrovasc Dis 31:106548). We expect that a refined modelling approach and the integration of imaging and clinical parameters will yield better prognostic factors and more reliable outcome predictions. Computer aided prediction of stroke outcome For computer aided outcome prediction in stroke patients, scores were developed based on a few preselected tabular features, such as age and sex, and an underlying logistic regression model. While these methods slightly outperform the score-based methods, they lack interpretability. Therefore, currently there are no computer aided outcome prediction tools used in clinical routine. Previous preparatory work In a previously conducted recent project, weanalyzed imaging features from DWI and perfusion imaging data such as TTP, CBF,CBV and TMAX in a group of stroke patients with LVO, all treated with MT at the InselSpital Berne. We tested if the generally accepted mismatch concept, derived from routine clinical software, indeed enhances outcome prediction for individual patients. We demonstrated that neither the core nor the mismatch volume significantly improved prediction of outcome when used in addition to clinical parameters (Hamann J et al. Eur J Neurol 28:1234-43). To implement a stroke outcome prediction model that is trustworthy for clinicians, we developed deep learning (DL) based models that can not only integrate structured and unstructured data, but also yield interpretable parameter estimates for clinical parameters like odds ratios (https://www.sciencedirect.com/science/article/abs/pii/S003132032100443X). We applied them to data from ischemic stroke patients treated at the University Hospital Berne (https://arxiv.org/abs/2206.13302) and investigated if our models can compete with experienced neurologists by performing a blinded prediction challenge, where five neurologists and our trained DL model were provided with the same structured clinical data of stroke patients. In this experiment, our model tended to show a better performance, particularly when image data were provided (Herzog L et al. Stroke. 2023 Jul;54(7):1761-1769). Accordingly, the present STOP-stroke study aims to improve outcome prediction early after stroke by assessing both the clinician's outcome estimation as well as our trained DL model. Thereby, we try to implement a new prognostication tool, which can be helpful in supporting treatment decision and individualized patient care. Primary and secondary endpoints in STOP-stroke: The primary endpoint of the the study will be the actual stroke related disability assessed by the mRS at 3 months as well as the NIHSS (stroke severity) after 24 hours and 3 months after acute ischemic stroke (AIS). Baseline factors that may have an influence on the primary endpoint are age, sex, vascular risk factors, treatment times, independence before stroke etc. Secondary endpoints are the predicted mRS scores of the treating neurologists on the stroke unit one side and the predicted mRS by the DL model on the other. Project design: This is a single-center project conducted in the Department of Neurology at the USZ. The study design will be exploratory. Recruitment, screening and informed consent procedure: The location of recruitment will be the Department of Neurology at the USZ. We plan to perform a single-center study. All patients referred to the stroke unit of the USZ with suspicion of AIS from 7 am until 5 pm will be screened by the study team. These patients will be recruited according to a consecutive ongoing recruitment through the study team in daily clinical practice. The screening process will take place as described in the following section. Patients with suspicion of AIS arriving at the stroke unit of the USZ will be screened if there is no refusal of data use already documented (prior objected GC). If yes, these patients will not be considered to take part in the study. If not, an independent physician will be asked to consent for the patient in order not to delay the acute treatment. The independent physician, which can be any physician not part of the study team, will be informed that their participation is voluntary and can be withdrawn at any time. Patients will be asked for their consent as soon as they are able to consent in the course of the hospitalisation or at least until the routine 3-month follow up visit. If until then, the patient is still not able to consent, we will ask the patient's next of kin. If the patient has no next of kin, there is no contact to the next of kin or the patient is lost to follow-up, or the patient has died, data will be used if no objection against use of data for research is documented. If patients and/or next of kin are not able to appear in person for the routine visit, we will send them a short letter of request of study participation together with the informed consent via mail. Within the letter of request, we will explain the informed consent procedure and ask patients either to send back the signed informed consent in case of study participation approval or to give a written statement of denyal of study participation. We will inform patients within the letter of request that if there is no reply within four weeks, we will use the obtained de-identified data if no prior objection against use of data for research is documented (the document of the letter of request can be found as a separate attachment). If patients reject to take part in the study, the collected coded data will not be used for further data analysis. We developed an informed consent for patients, next of kin and the independent physician specifically for the purpose of this study, which is attached among the study documents submitted. Study procedures: The study is planned to start immediately after approval by the ethics committee and will continue for an overall project duration of approximately three years. We plan a recruitment period of approximately one to two years. The project duration for each patient will be three months as we assess the primary outcome parameter (mRS) three months after AIS. As the study consists of a collection of personal health data of patients referred with suspicion of AIS to the USZ. These clinical and imaging data will be de-identified and stored on a password secured USZ server. We will not collect any biological patient material or samples (blood or tissue). Clinical and imaging variables: We plan to collect routine laboratory and imaging data, routine clinical scores of stroke severity and functional disability and other clinical routine information. The treating attending physician (board-certified neurologist) on the stroke unit at the USZ will be asked on the time points N1 and N2 and the treating attending physician during the hospitalisation period on N3 to complete a questionnaire (see attached at the end of this manuscript) accessible only via personal login to a specifically for this project developed website (planned with the secure web application REDCap). The clinical and imaging parameters will be obtained from the treating physician on the stroke unit or the neurological ward of the USZ or can be extracted from the electronic patient chart (KISIM). We do not plan to extract data from the Swiss Stroke Registry (SSR) as the data from the acute setting will not be documented here at that time point. The imaging data will be assessed at N2 from the clinically indicated first brain imaging. Additionally, we will assess the following parameters by the treating physician on the stroke unit based on the medical history and clinical examination, which we determined to be essential for the estimation of outcome: frailty, multimorbidity, uncontrolled cancer, pre-existing cognitive decline, polypharmacy (≥ 4 medications), institutional housing, weak or no social support (family or close friends) and an uncategorized slot for up to three other variables not previously listed. These parameters will be assessed categorically (yes/no option) from the study team members upon questioning the treating attending physician or via assessment of the electronic patient chart. The treating physician will assess the parameters upon clinical examination and/or questioning the patient or next of kin, if available in the acute setting. A list of these parameters is attached at the end of this manuscript. The web application REDCap will be designed by the Clinical Trial Center for the purpose of the project. All treating attending physicians will receive a personalized login in order to complete the questionnaires for the different study time points directly in REDCap. We plan a time-lock for the data entry to fix the data and secure the input of the data within reasonable time. Access to the data in REDCap will have only the treating attending physician as well as the study team members. The application is password secured for the purpose secure storage of study data. Outcome variables: NIHSS: The National Institutes of Health Stroke Scale is a 42-point neurological exam used to quantify the severity of an acute stroke. The NIHSS does assess the level of consciousness, eye move-ments, visual fields, facial muscle function, extremity strength, sensory function, coordination, language, speech, and neglect. The score will be collected as a standard procedure in AIS patients on admission to hospital at stroke onset and will represent one of the clinical parameters upon which the outcome prediction has to be made. The NIHSS will be also a primary outcome parameter (at 24 hours and 3 months after stroke onset), which has to be predicted by the physicians on N1-3, and will be prospectively asked from the treating physicians on the stroke unit, who are performing the score as part of the examination on admission routinely on N1, and asked from the treating neurologists on the ward on N3, which is part of the examination of discharge, as well as at the routine 3-months control at the outpatient clinic of the USZ The study team will assess the score via the patient chart or directly from the treating physician of the outpatient clinic of the department of Neurology at the USZ. If the patient is not able to present in person at the 3-month control, the study team can contact the next of kin, general practicioner, treating physician or nurse in the rehabilitation center or nursing home in order to assess the score. mRS: The modified Rankin Scale is a widely used scale of measuring the degree of disability of people who have suffered a stroke. The mRS is used in clinical practice, as well as in stroke studies to describe the clinical outcome after stroke. The mRS runs from 0 to 6, with 0 being a patient without any disability and 6 indicating for death of the patient. The score will be collected as a standard procedure in AIS patients on admission to hospital at stroke onset including the score before stroke onset as well as after three months as an outcome parameter assessed by physicians of the Department of Neurology. The score will be prospectively asked on N1-3 as it also represents a primary outcome parameter as well as at the routine 3-months control at the outpatient clinic of the USZ. The study team will assess the score at 3 months via the patient chart or from the treating physician of the outpatient clinic of the department of Neurology at the USZ at that time point. If the patient is not able to present in person at the 3-month control, the study team can contact the next of kin, general practicioner, treating physician or nurse in the rehabilitation center or nursing home in order to assess the score. Imaging data: We plan to store the imaging data (CT and MR images of the brain including CT- and MR-angiography as well as perfusion images of the brain) collected on admission to USZ or the hospital of the initial hospitalization as part of the clinically indicated first neuroimaging performed after admission to hospital with suspicion of acute ischemic stroke. The imaging data will be taken from the imaging software system used at USZ. The de-identification process will be performed according to USZ internal standards. A password protected master file will be stored by the data manager on USZ servers, which are secured according to USZ guidelines. Images will be stored in a de-identified form in a project specific area with restricted access. Access will be granted to all project members according to the staff list. As data will be archived in a structured way and stored in a coded form, imaging data can be made available on reasonable request for other research projects from other academic institutions. Project visits: After hospital discharge there will be only one regular clinical visit 3 months after stroke onset in our neuroangiology ambulatorium with assessment of the NIHSS and mRS. If a patient has deceased or is not able to come in person due to severe health restrictions, the next of kin or caregiver in charge will be contacted to assess the degree of disability post stroke. There will be no further clinical visits of telephone interviews planned within the study. Model predictions: The clinical and image data collected within this study will be transformed such that they can be fed into the DL models. The DL models will already be trained, that is, they already learned the imaging features, which are relevant for outcome prediction based on observational data that is not part of this study. In the publication of the previously conducted research project, in which we already used the method of the DL model prediction, we describe in detail how the clinical and imaging data will be entered in the model and how the model itself is then run. When all the data is collected for the planned interim analysis or final analysis, the models will be used to predict the outcome using the exact same patients to make the predictions comparable. The treating physicians involved in the prediction part can have insight in the final results after data analysis and evaluation.