This study evaluates the impact of a novel computer-aided prognostic prediction tool for colorectal liver metastases (CRLM) on clinician performance. Colorectal cancer is a leading cause of cancer-related mortality worldwide, with 20-30% of patients presenting synchronous liver metastases, which are associated with poor prognosis and high postoperative recurrence rates. Simultaneous resection of primary tumor and liver metastases is a preferred treatment for selected patients but outcomes vary significantly. The latest web-based tool uses Random Forest models integrating demographic, clinical, laboratory, and genetic data to predict postoperative recurrence and mortality specifically for CRLM patients undergoing simultaneous resection. This multiple-reader, multiple-case (MRMC) study will assess 12 physicians who will predict 1-, 3-, and 5-year recurrence and mortality risks in 166 retrospective cases, with and without the tool's aid, separated by a washout period. The primary focus is to determine whether the tool improves prediction accuracy for 3-year postoperative mortality, measured by AUC-ROC. Secondary and exploratory endpoints include other time points, sensitivity, specificity, inter-rater reliability, decision-making confidence, and evaluation time. By enabling individualized risk assessment, this tool aims to support optimized clinical decision-making and tailored treatment strategies for CRLM patients undergoing simultaneous resection.
This study aims to evaluate the impact of a novel computer-aided prognostic prediction tool on clinician performance in managing patients with colorectal liver metastases (CRLM). Colorectal cancer remains one of the leading causes of cancer-related mortality worldwide, with approximately 20-30% of patients presenting synchronous liver metastases at diagnosis. These metastases are associated with poor prognosis and a high rate of postoperative recurrence. For selected patients, simultaneous resection of the primary colorectal tumor and liver metastases is the preferred treatment approach, though clinical outcomes vary widely. To address this variability, the latest web-based prediction tool employs Random Forest machine learning models that integrate comprehensive demographic, clinical, laboratory, and genetic data. This tool is specifically designed to predict postoperative recurrence and mortality for CRLM patients undergoing simultaneous resection, enabling individualized risk assessment. In this multiple-reader, multiple-case (MRMC) study, 12 physicians will independently evaluate 166 retrospective patient cases. Each physician will estimate the risk of disease recurrence and mortality at 1-, 3-, and 5-year time points, both with and without access to the prediction tool. These two assessment phases will be separated by a washout period to minimize bias. The primary objective is to determine whether use of the tool improves the accuracy of predicting 3-year postoperative mortality, quantified by the area under the receiver operating characteristic curve (AUC-ROC). Secondary and exploratory endpoints include prediction accuracy at other time points, sensitivity, specificity, inter-rater reliability, clinician confidence in decision-making, and time required for evaluation. By providing specific, data-driven risk estimates, this computer-aided prognostic tool aims to enhance clinical decision-making and support personalized treatment planning for CRLM patients undergoing simultaneous resection, ultimately striving to improve patient outcomes.
Study Type
OBSERVATIONAL
Enrollment
166
No. 17, South Panjiayuan, Chaoyang District, Beijing, Cancer Hospital, Chinese Academy of Medical Sciences, China
Beijing, Beijing Municipality, China
RECRUITINGAUCs: Area Under the Receiver Operating Characteristic Curve (AUC-ROC)
The comparative accuracy of model aided versus unaided risk prediction of post-operative 3-year mortality assessed by readers. Under model aided condition, prediction model will estimate the overall survival (OS) rate at Year 3 for each individual patient. OS was defined as the time from the date of simultaneous resection to death due to any cause. Patients without death were censored at their last known alive date. The mortality data were retrospectively collected and the event status were known. Patients were followed up to approximately 120 months after simultaneous resection.
Time frame: Up to approximately 120 months
AUCs: Area Under the Receiver Operating Characteristic Curve (AUC-ROC)
The comparative accuracy of model aided versus unaided risk prediction of post-operative 1-year recurrence, 3-year recurrence, 5-year recurrence, 1-year mortality, and 5-year mortality made by readers. PFS was defined as the time from the date of simultaneous colorectal and liver resection to first documented disease progression or death due to any cause, whichever occurs first. Patients without PFS event were censored at their last assessment follow-up. OS was defined as the time from the date of simultaneous resection to death due to any cause. Patients without death were censored at their last known alive date. Patients were followed up to approximately 120 months after simultaneous resection.
Time frame: Up to approximately 120 months
Sensitivity: the ratio of true positives to total (actual) positives.
To evaluate aided versus unaided with respect to (improved) sensitivity for prediction of post-operative 1-year, 3-year, and 5-year recurrence and mortality made by readers.
Time frame: Up to approximately 120 months
Specificity: the ratio of true negatives to total (actual) negatives.
To evaluate aided versus unaided with respect to (improved) specificity for prediction of post-operative 1-year, 3-year, and 5-year recurrence and mortality made by readers.
Time frame: Up to approximately 120 months
Inter-rater reliability: the consistency of ratings made by readers on the cases
To evaluate aided versus unaided with respect to (improved) inter-rater reliability for rating on the likelihood of post-operative 1-year, 3-year, and 5-year recurrence and mortality made by readers.
Time frame: Up to approximately 120 months
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.