The goal of this blinded randomized controlled trial is to assess whether AI-assisted TTE interpretation can improve cardiologist efficiency and reduce interpretation variability while preserving accuracy compared to current methods in adult patients whose historical echocardiogram images were performed. The main questions it aims to answer is to: 1. Assess overall transthoracic echocardiogram (TTE) interpretation accuracy of an AI-assisted workflow compared to standard cardiologist-only and sonographer-assisted workflows to hypothesize if AI-assisted workflow for echocardiography interpretation is non-inferior in accuracy compared to standard workflows. 2. Evaluate cardiologist interpretation time using an AI-assisted workflow compared to standard cardiologist-only and sonographer-assisted workflows to determine if an AI-assisted workflow will result in a decrease in interpretation time compared to the cardiologist-only workflow. 3. To compare interpretation consistency and reliability compared to historical reports as well as consensus reports of all finalized cardiologist reports.
Study Type
INTERVENTIONAL
Allocation
RANDOMIZED
Purpose
DIAGNOSTIC
Masking
SINGLE
Enrollment
1,200
Echocardiogram studies are pre-interpreted by the AI software and finalized by a blinded cardiologist.
(Standard of Care 1): Echocardiogram studies are pre-interpreted by a sonographer and finalized by a blinded cardiologist.
(Standard of Care 2): Echocardiogram studies are interpreted solely by two cardiologists (one provides a preliminary interpretation another provides a blinded final interpretation).
Kaiser Permanente Northern California Division of Research
Pleasanton, California, United States
Accuracy of AI-Assisted Echocardiogram Interpretation Compared to Standard Workflows.
The accuracy rate is defined as the proportion of transthoracic echocardiogram (TTE) studies for which the final cardiologist interpretation is not significantly different from the preliminary interpretation. We will assess the accuracy rate in the AI-assisted arm for non-inferiority versus the pooled accuracy rate across both the sonographer-assisted and cardiologist-only arms. The investigators will measure the proportion of TTE studies for which the final cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventriuclar function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure).
Time frame: 6 months
Secondary Measures of Accuracy (Superiority Analyses)
Accuracy rates will also be analyzed for superiority rather than non-inferiority. The proportion of TTE studies for which the final cardiologist interpretation is not significantly different from the preliminary interpretation will be compared individually between the AI-assisted arm and each comparator arm (sonographer-assisted and cardiologist-only). 1. AI-prelim workflow vs. pooled sonographer-prelim and cardiologist-prelim workflow with a superiority test 2. AI-prelim workflow vs. sonographer-prelim workflow with a superiority test 3. AI-prelim workflow vs. cardiologist-prelim workflow with a superiority test
Time frame: 6 months
Accuracy Compared with Historical Cardiologist Interpretations
Accuracy is defined as the proportion of TTE studies for which the final cardiologist report is substantially different from the historical report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventriuclar function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure).
Time frame: 6 months
Accuracy Compared with Consensus Cardiologist Interpretations
Accuracy is defined as the proportion of TTE studies for which the consensus cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventriuclar function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure).
Time frame: 6 months
Evaluation requiring two-grade change in valve disease severity as substantial change
Accuracy is defined asthe proportion of TTE studies for which the final cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change). We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventricular function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure). 1. AI-prelim workflow vs. pooled sonographer-prelim and cardiologist-prelim workflow with a superiority test 2. AI-prelim workflow vs. sonographer-prelim workflow with a superiority test 3. AI-prelim workflow vs. cardiologist-prelim workflow with a superiority test
Time frame: 6 months
Cardiologist Overread Time (Efficiency)
To assess whether AI-assisted interpretation saves time, the investigators will compare the mean time in seconds required for the cardiologist to complete the overreading process across the three arms (i.e., TTE interpretation workflows) in our trial. Specifically, we will perform the following comparisons: 1. The mean cardiologist overread time in the AI-assisted arm versus the pooled results of the sonographer-assisted and cardiologist-only arms 2. The mean cardiologist overread time in the AI-assisted arm versus the sonographer-assisted arm 3. The mean cardiologist overread time in the AI-assisted arm versus the cardiologist-only arm
Time frame: 6 months
Comparison of Existing Standards of Care (Sonographer-Assisted vs Cardiologist-Only)
The investigators will measure the proportion of TTE studies for which the final cardiologist report is substantially different from the preliminary report (e.g., the rate of substantial change) in sonographer-prelim and cardiologist-prelim arms. We define a TTE report to be substantially different from another TTE interpretation if any of its key elements are substantially different (LVEF, right ventricular function, chamber size, valvular regurgitation severity, valvular stenosis severity, and right ventricular systolic pressure).
Time frame: 6 months
Blinding
In each of the arms, at the completion of each study for the final cardiologist assessment, the cardiologist will be asked to guess whether the preliminary assessment was AI or human. This will be used to calculate a blinding index comparing the AI-prelim arm vs. a composite of the two standard of care arms, as well as comparing the AI-prelim arm with the sonographer-prelim arm individually and comparing the AI-prelim arm with the cardiologist-prelim arm individually.
Time frame: 6 months
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.