Predicted Prognosis in Heart Failure

Overview

The purpose of the project is to conduct a physician-initiated Canadian multicentre observational research study that compares physician judgement and model prediction to estimate one-year survival in ambulatory heart failure (HF) patients and evaluate the use of resources according to physician intuitive risk. This study will evaluate the accuracy and impact of physician intuition and predictive models in the assessment of prognosis in ambulatory HF patients by: comparing 1-year physician predicted survival and 1-year observed survival to 1-year model predicted survival; evaluating whether model performance could be enhanced by incorporating physician intuition; evaluating the accuracy of physician intuition according to level of confidence in physician intuition (very low, low, moderate, high or very high); evaluating whether physician expertise impacts accuracy of physician intuition; and evaluating patient management and use of resources according to physician estimated survival.

Heart failure (HF) is a large and growing medical and economic problem, with over 26 million people suffering worldwide. The prognosis associated with HF is poor. Optimal management of patients with HF, crucial to improve outcomes and minimize costs, depends on adequate assessment of prognosis to decide on appropriate testing and intervention. There are predictive models, with satisfactory discrimination and calibration, that can accurately estimate mortality in HF patients. Despite their availability, physicians seldom use these models, instead relying on their informed intuition, which has proven to be limited. No studies have compared physician intuition (standard practice) and model predicted survival in patients with HF. The investigators therefore propose a Canadian multicentre study comparing physician intuition and model prediction to estimate one-year survival in ambulatory HF patients and secondly to assess the possible impact of intuition versus model prediction on the use of resources. Evaluating whether predictive models are more accurate than physician intuition will inform the best strategy to assess patient prognosis in HF. More accurate prognostic estimates will facilitate patient management by matching the need for further therapy or testing to patient risk thus offering greater clinical benefit and improving utilization of resources. This study will evaluate the accuracy and impact of physician intuition and predictive models in the assessment of prognosis in ambulatory HF patients by: i. comparing the accuracy of 1-year physician predicted survival and 1-year model predicted survival to the true (observed) 1-year survival; ii. evaluating the accuracy of physician intuition according to physician's level of confidence in their intuition (very low, low, moderate, high or very high); iii. evaluating whether physician expertise impacts accuracy of physician intuition; iv. evaluating whether physician gender, patient gender and physician-patient gender concordance impact accuracy of physician intuition; and v. studying the association between physician estimated survival and resource use and related cost. The investigators hypothesize that predictive models will more accurately predict mortality than physicians, with more marked differences in less experienced physicians or when physician confidence in estimate is low. Physicians will use more resources when they consider patients to be high risk. If these hypotheses prove correct, incorporation of user-friendly systems to estimate prognosis into clinical practice can offer clinical benefit, facilitate patient management and improve utilization of resources. This is a Canadian multicentre prospective cohort study of consecutive consenting ambulatory adult heart failure (HF) patients followed in a HF clinic. Participating centers include tertiary care hospitals with dedicated HF clinics in British Columbia (St. Paul's Hospital, Providence Health Care Centre); Manitoba (St. Boniface General Hospital); Ontario (Toronto General Hospital, St. Michael's Hospital, Ottawa Heart Institute, Sunnybrook Hospital, Mount Sinai Hospital, Hamilton Health Science, Southlake Regional Health Centre); Quebec (McGill University Health Centre); and Nova Scotia (Nova Scotia Health Authority). These clinics attend to different HF populations permitting a wide representation of patient profiles. After obtaining written informed consent, research assistants will collect clinical and laboratory data from electronic records and paper charts necessary to describe the patient population and to calculate predictive model survival. These constitute part of the routine clinical assessment and will include demographic characteristics (age, sex, race), co-morbidities (diabetes, hypertension, smoking, peripheral vascular disease, chronic lung disease), HF characteristics and history (underlying cause, LVEF by echocardiogram, last HF hospital admission, medications and use of ICD and/or cardiac resynchronization therapy) and physical examination (body mass index (BMI), current NYHA class, heart rate, and blood pressure at rest). Laboratory values will include hemoglobin, leucocytes, lymphocytes, electrolytes, BUN (blood urea nitrogen), serum creatinine, total cholesterol and uric acid. Brain natriuretic peptide (BNP) or N-terminal pro-BNP and peak oxygen consumption (peak VO2) will be collected when available. Cardiac rhythm and QRS duration will be collected by electrocardiography and peak oxygen consumption by cardiopulmonary exercise study. This study will include physicians with different level of expertise in HF (HF cardiologists and Family Physicians). Physicians will be asked to provide their intuitive estimates of the likelihood of survival at one year following the patient baseline clinic visit. Physicians will be blinded to the predicted model survival. In the survey, the physician will: (1) estimate patient 1-year survival in absolute terms (from 0% to 100%); (2) rate the confidence in their prediction (from 1 - no confident at all to 5- very confident); (3) collect their impression about the possibility of initiating assessment to evaluate candidacy for advanced heart failure therapies including heart transplant or mechanical circulatory support in the next year (1- not a candidate, 2- patient is already listed, 3- too early, and 4-patient is a candidate); and (4) record status of optimization of medical management (1- beginning optimization, 2- early in the process, 3- late in the process, and 4- completed). Patient candidacy to advanced HF therapies and status of optimization can influence the use of resources. If a patient is not a candidate for advanced HF therapies or is already under optimal medical therapy, the use of resources will be lower compared to candidate patients or patients under medical therapy optimization. This will be considered in the analysis of the impact of physician intuitive risk on resource utilization. There are many predictive models in HF. Of these, the investigators have chosen three models based on comparably acceptable performance: the Seattle Heart Failure Model (SHFM), the HF Meta-Score and the Meta-Analysis Global Group in Chronic Heart Failure (MAGGIC) score. These models include a set of different variables, they have been validated in contemporary cohorts of HF patients and have demonstrated excellent calibration and discrimination with a c-statistic \>0.70. Patients will be followed until the last recruited patient is followed for a minimum of 1-year and the following outcomes will be collected: i. Death, urgent VAD implant and urgent heart transplant. Urgent VAD implant or heart transplant will be defined by the use of intravenous inotropic support at the time of the surgery. The investigators will collect this information from electronic records and by linking with the administrative databases housed at the Canadian Institute for Health Information (CIHI) via OHIP number or name and date of birth (retrieved from the patients' medical records) using deterministic/probabilistic linkage. CIHI is an independent, not-for-profit organization that provides essential information on Canada's health systems. This entity houses 28 pan-Canadian databases across various health sectors. CIHI holds ISO/IEC 27001:2005 certification for information security management. ii. Utilization of health resources and related costs will be collected by linking the project database to the administrative databases, administered by the CIHI Discharge Abstract Database (CIHI DAD) and CIHI National Ambulatory Case Reporting System (CIHI NACRS). Cost information will be obtained and estimated from the HF clinic at Toronto General Hospital, which is representative of other HF clinics. The cost associated to resources in other provinces will be adjusted by a calculated coefficient using publicly provincial health cost information. The investigators will collect inpatient and outpatient costs and resources from clinic visits, imaging and laboratory tests, cardiac rehabilitation, hospitalization and visits to an emergency department. Continuous variables will be expressed as a mean and standard deviation (SD) or median and interquartile ranges for variables with non-Gaussian distributions. All discrete variables will be expressed as counts (n) and percentages (%) of the study population. The statistical analysis will be performed using SAS 9.4 (North Carolina, USA). The investigators will assess physician intuition in comparison to the performance of predictive models by comparing their discrimination, calibration and risk reclassification. Physician intuition and model prediction accuracy will be evaluated separately by physician expertise: HF cardiologists and family physicians. Kaplan-Meier analysis and Cox proportional hazards model will be used to predict 1-year event-free survival with the score from each model and physician intuition to assess calibration and discrimination, respectively. HF clinics may see a set of different HF patients and HF cardiologists practicing in the same HF clinic will potentially have similar practice and intuition. In order to consider this potential confounding effect, the analysis will be adjusted for HF clinic region (Greater Toronto Area, Ontario, Quebec, Manitoba, Nova Scotia and British Columbia). Follow up will be censored as alive at the time of non-urgent VAD or heart transplant or last clinic visit after a year follow up. Observed versus predicted survival will be used to assess calibration illustrating the relationship in a scatter plot and will assess and compare intuition and predictive models' discrimination using Harrell's c-statistic. The investigators will then use risk reclassification analysis (reclassification tables and reclassification calibration test) and absolute net reclassification improvement (NRI) to assess global model performance of physician intuition in comparison to the predictive models. Risk reclassification analysis will be used to show how patients classified by physician intuition are reclassified by the predictive models and will compare the observed and predicted survival in each cross-classified category. The absolute NRI represents the net proportion of patients correctly or incorrectly classified assessing if patients were reclassified in the correct direction, i.e. if survivors are reclassified as having better survival and deceased patients are reclassified as having lower survival. For this analysis, patients will be classified based on deciles of 1-year predicted survival (100-90%, 90-80%, 80-70% and \<70%). The association between physician intuition and utilization of health resources will be evaluated by categorizing patients in the pre-defined risk categories (low, medium, high or very high risk). Multivariate logistic regression will be used to evaluate the association between physician intuition and resources measured as binary variables (i.e. referral to specialists or palliative care) and multivariable Poisson regression to evaluate the association between physician intuition and resources measured as count variables (i.e. clinic visits) adjusted for region. Cost analysis will be used to evaluate the association between total annual cost and physician intuition. Cost will be described using median and interquartile range due to expected positively skewed distribution and evaluate its association with physician intuition using generalized linear models with a gamma distribution adding region as a fixed effect. The investigators will evaluate the impact of model predictive survival using the results from the risk reclassification analysis (e.g. if the predictive model better reclassified 10% of high-risk patients, 10% of the increased cost of treating high-risk patients may be saved by using the model).