Regression Discontinuity Design to Evaluate of Drotrecogin Alpha Effectiveness

Overview

Many health care interventions and medications found to have benefits ("efficacy") in experimental, tightly-controlled, human research trials are later found to lack real-world health benefits ("effectiveness"). Inadequate surveillance of real-world clinical effectiveness may falsely reassure clinicians and those who monitor healthcare quality, propagating unrecognized ineffective or harmful treatments at high costs to patients and society. The failure to translate potential health benefits into realized gains, or to detect unexpected harms in healthcare delivery, stems from a lack of methods with which to robustly measure real-world (in)effectiveness. Current methods to detect changes in outcomes 'before and after' implementation may be biased by secular trends in healthcare practice and outcomes; other methods to compare outcomes for treated and untreated patients may be biased by unmeasured factors. The current project aims to develop and demonstrate - as a proof-of-concept - the use of a quasi-experimental research method called 'regression discontinuity design (RDD)' in surveillance of real-world clinical effectiveness. RDD had previously found use in the evaluation of educational programs in which students scoring below a threshold were assigned an intervention. The US Department of Education considers RDD designs to have quality similar to randomized trials.

Many health care interventions and medications found to have benefits ("efficacy") in experimental, tightly-controlled, human research trials are later found to lack real-world health benefits ("effectiveness"). Inadequate surveillance of real-world clinical effectiveness may falsely reassure clinicians and those who monitor healthcare quality, propagating unrecognized ineffective or harmful treatments at high costs to patients and society. The failure to translate potential health benefits into realized gains, or to detect unexpected harms in healthcare delivery, stems from a lack of methods with which to robustly measure real-world (in)effectiveness. Current methods to detect changes in outcomes 'before and after' implementation may be biased by secular trends in healthcare practice and outcomes; other methods to compare outcomes for treated and untreated patients may be biased by unmeasured factors. The current project aims to develop and demonstrate - as a proof-of-concept - the use of a quasi-experimental research method called 'regression discontinuity design (RDD)' in surveillance of real-world clinical effectiveness. RDD had previously found use in the evaluation of educational programs in which students scoring below a threshold were assigned an intervention. The US Department of Education considers RDD designs to have quality similar to randomized trials. However, RDD has not been rigorously evaluated in the context of evaluating clinical effectiveness. RDD can be used whenever an intervention is given to patients scoring above a threshold on a continuous biomarker or risk score. This scenario often arises in clinical practice, in which thresholds are used to identify and treat 'high risk' patients. In RDD, outcomes are compared for patients just above and just below the threshold, who are similar, but receive different treatments. The project will study the use of RDD in evaluating the real-world effectiveness of drotrecogin alpha, a medication that was recommended by the FDA to be given to critically ill patients with severe sepsis at high risk for mortality (APACHE score \> 25). Drotrecogin alpha was shown to potentially have "effectiveness" using traditional methods of real-world research, but was eventually shown to not be clinically efficacious in subsequent large randomized trials. The present proposal is a 'proof-of-concept' study that will allow evaluation of effect estimates derived from RDD methods to those of gold standard, pooled randomized trial results. The demonstration of feasibility for a new research method, such as RDD, to evaluate real-world clinical effectiveness would be a major leap forward in the ability monitor for potential real world benefits and harms of new treatments.