Development of a Prediction Model for Intraoperative Blood Pressure Variability

Overview

Objective: The aim of this study was to use machine learning to predict and interpret intraoperative high blood pressure variability（IHBPV）. Design: Retrospective cohort study. Setting: Beijing Tsinghua Chang Gung Hospital . Data resources: 47520 operations performed under general anesthesia in the central operating room from March 2016 to April 2022. Interventions: None. Measurements: investigators collected data on preoperative baseline information and intraoperative variables. The model was constructed with python and run using the following models: XGBoost, random forest, LGBoost, and logistic regression.

1. Introduction Blood pressure variability (BPV) is a description of the degree of fluctuation of a patient's blood pressure， such as standard deviation、average real variability, and variability independent of the mean， The above terms all describe the situation of BPV to varying degrees。BPV is divided into different types, and the short-term BPV of the patient is related to other terms BPV。In addition to the numerical value of blood pressure itself, high blood pressure variability will bring a series of complications, such as Target organ damage、cardiovascular events, and even death. Relevant literature and guidelines also recommend that the increase or decrease of blood pressure should not exceed 20% of the basic blood pressure value, and reducing the risk of intraoperative high blood pressure variability (IHBPV) has become one of the important tasks of anesthesiologists。 The regular vital signs recorded during surgery include blood pressure, heart rate, and respiratory rate, which may cause BPV to be overlooked by anesthesiologists； The factors that cause IHBPV are complex, and the patient's own vascular status,sympathetic nervous excitability,drug use,fluid balance and surgical stimulation can cause IHBPV and increase its perioperative risk, affecting the patient's prognosis. Multidimensional, time-varying changes make it difficult for anesthesiologists to make judgments, and there is currently a lack of effective prediction and explanation of IHBPV. Machine learning（ML）is an interdisciplinary field that studies how computers simulate human learning behavior and reorganize existing knowledge structures to improve their performance and acquire new knowledge and skills. Recent research has found that ML has advantages in predicting and explaining complex systems, ML is suitable for analyzing complex real-world data and processing time series data. The purpose of this study was to construct machine learning models to predict the occurrence of IHBPV during overall surgery in patients, and to clarify the baseline information and intraoperative factors that lead to IHBPV, in order to assist anesthesiologists in improving the management of BPV during surgery. 2. Materials and methods 2.1 Study Design This retrospective study collected data from the Beijing Tsinghua Chang Gung Hospital affiliated with Tsinghua University from March 2016 to April 2022, all patients who underwent surgery at the center operating room with the approval of the ethics committee were included in the study. The study mainly obtained patients' information from the HIS system and the Medtronic system and aimed to predict the possibility of IHBPV in patients based on baseline information and intraoperative factors through ML models and to explain the increase in the risk of IHBPV among patients through interpretable ML models. The original database consisted of 52250 cases of general anesthesia surgery. The sample size was based on the scale of the existing database, of which 47520 surgeries met the criteria of this study. The inclusion criteria were patients who received general anesthesia, intravenous anesthesia, or intravenous-inhalation anesthesia, and ASA1-5 grade. And the exclusion criteria were surgeries with missing key information and surgeries that were not monitored for blood pressure throughout the operation. For surgeries with missing or ambiguous information, investigators used a deletion method to ensure the authenticity of the data. Important variables included time records, vital signs, and fluid balance. The Ethics Committee of Beijing Tsinghua Chang Gung Hospital has approved the study and waived the requirement for personal informed consent,as it is a retrospective study,this research has also been approved by the China Clinical Trial Registration Center.This study followed the relevant part of the "Transparent Reportingof a Multivariable Prediction Model for Individual Prognosis or Diagnosis" (TRIPOD) reporting guidelines for clinical prediction models, which includes commonly used reference items for building clinical prediction models. 2.2 Data collection This study mainly obtained the baseline information of patients from the HIS system, including height, weight, gender, disease diagnosis, and surgical method, and extracted intraoperative factors from the Medtronic surgical anesthesia system as independent variables, including intraoperative blood pressure, intraoperative drug data, fluid balance, and key surgical operations recorded during surgery. investigators selected some commonly used drugs, and the criteria for commonly used drugs were that the usage frequency of the drug in various surgeries exceeded one-tenth. The commonly used drugs are sevoflurane, propofol, dexmedetomidine, midazolam, remifentanil, sufentanil, methoxamine, and rocuronium; For preoperative diagnoses, considering their impact on blood pressure variability, investigators prioritized the extraction of binary variables related to blood pressure, such as cardiac dysfunction, and renal dysfunction. 2.3 Diagnostic criteria for IHBPV The primary endpoint of this study is blood pressure variability (BPV) which has been studied in several previous papers， BPV has different definitions and calculation methods， each with its own advantages. The coefficient of variation (CV) of intraoperative mean arterial pressure (MAP) was used as a quantitative measure of BPV, It takes into account the patient's blood pressure fluctuations while also considering their baseline blood pressure level. In this study, investigators calculated the coefficient of variation (CV) of the intraoperative MAP as a quantitative indicator of blood pressure variability (BPV), CV of MAP is calculated by dividing the standard deviation of MAP by the mean value of MAP。 According to previous literature and guideline recommendations, the increase and decrease in patient blood pressure during surgery should not exceed 20% of the baseline blood pressure value. Therefore, investigators define cases of IHBPV as situations where the CV of MAP exceeds 20% during the surgery, regardless of whether the increase in blood pressure variability is due to a decrease or an increase in blood pressure. The blood pressure recorded every five minutes from the beginning of anesthesia is used to calculate BPV, and the primary source of blood pressure information is the invasive arterial line, and if that information is not available, non-invasive cuff measurements are used. Due to the different monitoring situations during surgery, invasive arterial pressure can be recorded multiple times per minute, while non-invasive cuff measurements are recorded at least every five minutes. All recorded blood pressure values are Used to judge whether the patient complies with IHBPV. 2.3 Predicting Models Prediction is a supervised machine learning (ML) technology that is very effective in predicting data analysis. It is based on training data to map new input records into specific dependent output variables based on the relevant independent variable values. Because of its ability to handle complex correlated variables and its effectiveness in handling mutually correlated variables, it is crucial to use novel prediction algorithms to achieve optimal accuracy. In this study, investigators developed four machine learning (ML) prediction models, extreme gradient boosting (XGBoost) 、Light Gradient Boosting（LGBoost）、 random forest (RF), and logistic regression（LR）to predict the likelihood of intraoperative hypertensive blood pressure variability (IHBPV) using baseline information and intraoperative factors. The models were built using the python 3.9 version, with the pandas 1.3.4 library for data cleaning, the Scikit-Learn 0.24.2 library for model creation and hyperparameter optimization, and the shap 0.41.0 library for interpretability. The training of the prediction models was performed using a dataset of 47520 patients, which was split into training (70%) and testing (30%) sets based on the distribution of the outcome variable (Fig. 1). For validation purposes, a cross-validation approach was used for the training set. When verifying such datasets, a hierarchical k-fold cross-validation was used due to its efficiency and smoothness. Each dataset was randomly divided into k folds, with k 1 folds used for training purposes and the rest used as a validation set. To evaluate the performance of the three prediction models was independently and iteratively validated using k-fold cross-validation from k = 5. The best accuracy of all three algorithms was achieved by 5-fold cross-validation. Our main indicator is to compare the area under the curve（AUC）, the area under the receiver operating characteristic（ROC）curve. Confidence intervals of the AUC are estimated using an estimated indicator of 100 bootstrapped samples. The metrics investigators used for comparison include accuracy, precision, sensitivity, specificity, and F1 score, which are all used to measure the performance of a classification model. The comparison of multiple metrics allows for a balanced assessment of the strengths and weaknesses of the model. The Permutation Importance method is used to analyze the sensitivity of the variables model. Permutation Importance is suitable for tabular data, and the evaluation of the importance of a feature depends on the decline in the performance of the model after the feature is randomly rearranged. It is also not limited to specific model categories and has a wide range of applications. Permutation feature importance is a model verification technique that arranges the feature importance of the estimated values of the given dataset, where importance is defined as the decline in the model score when a single feature value is shuffled randomly. The change in the model's prediction ability is used to determine whether it is necessary to delete the included variables. This technique benefits from model unknowability and can be calculated multiple times with different feature arrangements. 2.5 Interpreting Methods Explanatory methods in this study are based on XGBoost, as XGBoost yielded the best results in prediction and has been shown to have good compatibility with the SHAP method in previous studies. XGBoost is a recently developed machine learning technology that has been widely used in many fields. As a portable and flexible method is suitable for a variety of applications. It is based on Cause Based Decision Tree and Gradient Boosting Machine algorithm, mainly to reduce the error of the learner model. In each lift iteration, the first and second-order gradients of the objective function "squared error" were calculated for each training case. So it is through multiple simple base learners, that constantly reduce the difference between model values and actual values. It gains the ability to enhance tree-lifting methods to quickly and accurately process almost all data types. With these unique functions, the algorithm can be effectively used in regression prediction models. XGBoost Is also used to handle large datasets with a large number of properties and classifications and provides practical and proficient solutions when considering efficiency and accuracy trade-offs. investigators applied interpretability to the ML model, and interpretable ML models include Local Interpretable methods, Global Interpretable methods, and Interactive Interpretable methods. Local Interpretable methods can specifically identify the impact of individual predictors on the outcome index for a single sample, i.e. exploring the perioperative predictors that have the greatest impact on IHBPV. The final output of Local Interpretable methods is the size and proportion of changes in SHAP values for each predictor for a specific patient. Global Interpretable methods can specifically identify the overall impact of predictors on outcomes for all samples, i.e. exploring perioperative predictors that have a comprehensive impact on blood pressure variability during surgery for all patients. Finally, Interactive Interpretable methods explore the modifying effects among multiple predictors, i.e. the phenomenon of a predictor's impact on the outcome changing with the level of other predictors at different levels. Its existence indicates that the effects of several predictors studied simultaneously are not independent. The interpretation of interaction in this study can specifically identify the joint impact of different predictors on IHBPV.