Passive Evaluation in Operational Environment of the AI Clinician Decision Support System for Sepsis Treatment

Overview

Sepsis, or systemic infection, is a common reason for ICU admission and death throughout the world. Despite advances in the way we treat this condition, it remains a significant economic and healthcare burden. A key part of the treatment of sepsis is the administration of IV fluids and blood pressure medication. However, there is huge uncertainty around dosing of these drugs in an individual patient. A tool to personalise these medications could improve patient survival. The study team has developed a new method to automatically and continuously review and recommend the correct medication doses to doctors, which was created using artificial intelligence (AI) techniques applied to large medical databases. The researchers' previous work has shown it has the potential to improve patient survival rates. The tool will be capable of processing patient data within the electronic patient record of NHS hospitals in real-time to suggest a course of action. This tool will be evaluated and refined in simulation studies and then be tested in two NHS Trusts in "shadow mode" (results not provided to duty clinicians). This will allow comparison of actual decisions made and recommended decisions from the AI system. The second stage of this clinical evaluation will display the recommendations to clinicians to assess the acceptability of the tool and confirm technical feasibility to inform future clinical trials. The long-term expected benefits of this project are numerous: improved patient survival, reduced use of precious intensive care resources and reduction in healthcare costs.

Sepsis is life-threatening organ dysfunction due to severe infection and affects 250,000 patients annually in the UK (pre-COVID-19), of whom 48,000 die. In addition, virtually all COVID-19 intensive care unit (ICU) deaths had sepsis. It is a leading cause of death and the most expensive condition treated in hospitals. It was recognised as a top research priority by the James Lind Alliance, a partnership of patients and clinicians to prioritise the most pressing unanswered questions facing the NHS. The cornerstone of sepsis resuscitation is the administration of intravenous fluids and/or vasopressors (drugs that squeeze the blood vessels to increase blood pressure) to maintain blood flow to prevent organ failure. However, there is huge uncertainty around the individual dosing of these drugs in an individual patient, partially due to high sepsis heterogeneity. The current guidelines provide recommendations at a population-level but fail to individualise the decisions. Wrong decisions lead to poorer outcomes and increased ICU-resource use. A tool to personalise these medications could improve patient survival. The study team has developed a new method to automatically and continuously review and recommend the correct dose of these medications to doctors, which was created using artificial intelligence (AI) techniques applied to large medical databases. The method used is called reinforcement learning. In this framework, the study models patients with sepsis in the ICU as belonging to a large number of possible disease states, and analyses what interventions are likely to help them transition to healthier states, and eventually to survival. The researchers demonstrated in their initial publication that the value of the AI selected strategy was on average reliably higher than human clinicians. In a large validation cohort independent from the training data, mortality was lowest in patients where clinicians' actual doses matched the AI decisions: mortality rates rose, in a dose dependent manner, as the clinicians' actual decisions diverged from the AI decisions. The study team has estimated that their AI algorithm could reduce mortality by 10% (in relative terms), which represents over 1,000 lives saved annually in the UK and would scale to hundreds of thousands of lives worldwide. Now, the study team intends to start clinical testing of this AI technology in the UK. The envisioned end-product will be a piece of software that will be accessible by clinicians (ICU doctors initially, then eventually to ICU nurses as well) at the bedside in intensive care. This software will be connected to the electronic patient record, which will be fed to the AI algorithm. In return, the AI will identify where the patient sits in the array of possible disease states, and which actions (a dose of intravenous fluids and vasopressors) are most likely to be beneficial. First, the study team will develop this software tool, capable of processing patient data within the electronic patient record of NHS hospitals in real-time to suggest a course of action. The study will start by evaluating and refining this tool in simulation studies. The study team will then test the AI tool in two NHS Trusts in a "shadow mode" when the result is not provided to duty clinicians in charge of patient care. This will allow comparison of actual decisions made and recommended decisions from the AI system. In the second stage of the clinical evaluation, the study team will display the recommendations to clinicians to assess the acceptability of the tool to clinicians and also confirm the technical feasibility to inform future large scale clinical trials. The long-term expected benefits of this project are numerous: improved patient survival, reduced use of precious intensive care resources and reduction in healthcare costs.