COLORS-Validate Study of Decision-Support Software for Surgical Sequencing in Colorectal Liver Metastases

Overview

This study evaluates a surgical decision-support software designed to assist physicians in selecting the optimal surgical sequence for patients with colorectal cancer liver metastases. In this multicenter prospective simulation study, participating surgeons from multiple centers will review standardized clinical case scenarios. For each case, physicians will first make a surgical sequencing decision based on their own clinical judgment. They will then review the recommendation provided by the decision-support software and make a second decision if they choose to revise their initial plan. The study will assess whether the software influences clinical decision-making, including changes in surgical strategy, decision confidence, decision time, and agreement with software recommendations. Physician user experience will also be evaluated using standardized questionnaires, including system usability and cognitive workload scales. The goal of this study is to determine the clinical utility and usability of the decision-support software in improving surgical decision-making consistency and supporting clinical reasoning in colorectal liver metastases cases.

The selection of surgical sequencing-whether to resect the primary tumor first or the liver metastases first-in patients with colorectal cancer liver metastases (CRLM) undergoing synchronous resection represents a critical and highly complex decision in current clinical practice. With the advancement of systemic therapies and surgical techniques, an increasing number of patients with initially unresectable CRLM have gained the opportunity for surgical treatment. However, due to substantial heterogeneity among patients in terms of tumor burden, biological behavior of the disease, and overall physical condition, no unified standard currently exists for determining the optimal surgical sequence. Existing studies comparing different surgical strategies have yielded inconsistent prognostic results, and most are based on retrospective analyses lacking consistent conclusions and clear stratification criteria. Consequently, in real-world clinical practice, such decisions rely heavily on physician experience and institutional preference, resulting in considerable inter-physician variability and uncertainty. This scenario, characterized by the absence of a clearly optimal strategy, renders the decision-making process inherently a form of decision-making under uncertainty. Based on this context, we previously developed, using multicenter retrospective data, two multi-outcome predictive models corresponding to different surgical strategies. These models integrate postoperative major complications, Comprehensive Complication Index (CCI), progression-free survival (PFS), and overall survival (OS), and have been externally validated. However, in real-world clinical decision-making, the central challenge faced by surgeons is not merely the prediction of a single outcome, but rather the need to balance multiple competing outcomes. For example, one surgical strategy may be associated with a lower risk of postoperative complications but limited long-term survival benefit, whereas another may involve higher perioperative risk in exchange for improved oncological outcomes. This inherent tension among multidimensional outcomes transforms clinical decision-making into a multi-objective decision-making problem, rather than a simple comparison of a single endpoint. In current practice, such trade-offs across outcomes are typically based on physicians' experiential judgment, whereby the relative importance of different outcomes is implicitly weighted. However, this weighting process lacks explicit representation and standardized criteria, and is prone to influence by individual experience, risk preference, and institutional norms, thereby leading to substantial variability and inconsistency in decision-making. To address this key limitation, the present study further constructs a structured multi-outcome weighting framework using the Delphi method combined with the analytic hierarchy process (AHP). This approach establishes the relative importance of different clinical outcomes through expert consensus and translates it into a quantifiable weighting system, thereby enabling the integration of multidimensional predictive results into a comparable composite score. Based on this framework, we developed a decision-support software system capable of simultaneously providing individualized multi-outcome predictions and composite reference scores, transforming the previously experience-dependent implicit trade-off process into a transparent, interpretable, and standardized decision-support process. Nevertheless, the establishment of predictive models and weighting frameworks does not necessarily translate into practical clinical value. For decision-support systems, focusing solely on predictive performance is insufficient to reflect their real-world clinical utility. More importantly, it is necessary to evaluate whether such systems can influence physician decision-making behavior under uncertainty, reduce unnecessary variability, and provide a structured reference framework for decision-making. In addition, because clinical decision-making involves counterfactual comparisons (i.e., potential outcomes under alternative strategies for the same patient cannot be simultaneously observed), traditional outcome-based evaluation methods have inherent limitations. Therefore, it is necessary to adopt a model-informed standardized reference framework to evaluate the impact of AI assistance at the level of decision behavior. Based on the above considerations, this study adopts a multicenter, prospective crossover simulation design to systematically evaluate the clinical utility of the decision-support system in a standardized setting. The evaluation focuses on the following two dimensions: 1. From a scientific decision perspective: whether the system alters physician decision behavior, improves decision consistency, and reduces variability; 2. From a human factors perspective: whether the system affects cognitive load, decision confidence, and user experience. This study extends AI evaluation beyond predictive performance toward its influence on decision-making behavior, with a particular focus on the human-AI collaborative decision-making process in uncertain clinical scenarios.