Computer-aided diagnosis (CADx) for colonoscopy aims to enhance optical diagnosis but often underperforms when used alongside humans due to under-reliance on AI. Psychological interventions like cognitive forcing, such as delaying CADx suggestions, may improve human-AI interaction by fostering critical assessment. However, their impact on patient-important outcomes remains unexplored. The investigators will conduct an ex-vivo randomized study with 70 endoscopists assessing 100 polyp videos (≤5 mm) using a CADx tool (GI Genius, Medtronic). Participants will be randomized to either: * Intervention group: CADx suggestions will be shown in the last 3 seconds of the 15 second polyp video. * Control group: CADx suggestions will be shown in real-time throughout the playback of the 15 second polyp video. The primary endpoint is sensitivity for high-confidence neoplasia detection, with secondary endpoints assessing endoscopists' reliance on AI. CADx systems on the market function in various ways, such as real-time, delayed, or on-demand diagnosis. Our study aims to inform users and manufacturers whether cognitive forcing through delayed CADx suggestions enhances human-AI interaction, leading to improved clinical outcomes.
Computer-Aided Diagnosis and cognitive forcing Computer-aided diagnosis (CADx) for colonoscopy is expected to improve physicians' ability to predict colorectal polyp pathologies (optical diagnosis). However, recent randomized trials indicate that collaboration between humans and CADx yields lower performance than CADx alone. This suggests suboptimal human-AI interaction due to users' under-reliance on CADx advice, favoring their inherent biases over critically assessing AI suggestions. Psychological interventions, such as cognitive forcing, aim to address this by encouraging crucial assessment of AI suggestions. One example, delayed display of CADx suggestions may promote proactive thinking by giving endoscopists enough time to consider polyp pathologies before receiving CADx suggestions, potentially leading to critical and optimal human-AI interaction. The effectiveness of such cognitive forcing was observed in experimental studies of AI for mammography reading. However, despite its potential, no studies have evaluated the impact of such interventions on optical diagnosis accuracy in colonoscopy. To investigate the value of psychological intervention in optical diagnosis, the investigators will conduct an ex-vivo randomised controlled study. Study aim: This is an ex-vivo randomised controlled study. The hypothesis of our study is that cognitive forcing by delaying display of CADx suggestions facilitates physicians' critical thinking, leading to better clinical outcomes in optical diagnosis in colonoscopy. Comercially CADx device and video details: In this study, the investigators are going to use the comercially available CADx system (GI Genius CADx, manufactured by Cosmo Intelligent Medical Devices and distributed by Medtronic Corp). The GI Genius CADx highlights the suspicious area for polyps on-screen with bounding boxes and provides optical diagnosis prediction (i.e. adenoma, non-adenoma). All the endoscopists will be given the information that the CADx tool used in the study is GI Genius with a link to their product overview. The investigators will collect 100 colonoscopy videos of 100 different diminutive polyps from the Polyp Image BAnk database (PIBAdb). In accordance with the real-world prevalence, 65 polyps will be neoplastic while the remaining 35 will be non-neoplastic. The duration of each video will be adjusted to contain 15 second appearance of the lesion including wite light (WL) and narrow band imaging (NBI). All the polyps should be \< = 5 mm. The original videos were recorded without having any CADx interaction. PIBAdb contains 507 videos of diminutive colorectal polyps with WL and NBI, of which 231 have a duration of 15 seconds or more. All the videos contain polyps with their histopathology available (i.e. adenoma, sessile serrated lesions, traditional serrated adenomas, invasive, hyperplastic and non-neoplastic). The polyps that have no histology category are going to be excluded from the study. The investigators will use this database as a pool to select the 100 study videos. First, the investigators are going to split the pool of data into two groups according to polyp histology: one pool containing neoplasia (i.e. adenoma, sessile serrated lesions and traditional serrated adenoma) and the other containing non-neoplasia (i.e. hyperplastic and non-epithelial neoplastic). In each pool, 20 videos will be randomly selected as a first step. These 20 videos will be assessed if they meet our inclusion and exclusion criteria (see below), leaving only eligible videos. This selection process will be repeatedly done until the investigators collect 65 videos of polyps with neoplasia and 35 videos of polyps with non-neoplasia. The figure below shows how the video selection process takes place. GI Genius´s specification The GI Genius system processes the videos at 50-60 frames per second and in high-definition format to have a good quality of the videos. To process the videos, the GI Genius splits them into two different streams by dedicated video card. One stream is transmitted to the first path to the output without any processing of the AI algorithm. The second stream is sent to the AI algorithm (i.e. second path) and after appropriate computation, if there is a polyp present on-screen appears an overlay in the output highlighting the polyp. The system was designed to work on unaltered WL video streams, but it works under both WL and NBI similarly.The transmission of the video stream to GI Genius is through a Serial Digital Interface (SDI) cable, acquiring the output stream from the video displaying in a computer. Finally, the SDI output stream is transmitted to the monitor containing the original video stream with additional markers superimposed on it. How to make CADx overlaid videos In the present study, the selected videos of colorectal lesions will be stored in our high-spec computer system first. Then, these videos will be transmitted to GI Genius with an SDI cable. A capture card (DeckLink 8K Pro Mini, Blackmagic) is integrated into the computer system that converts the recorded video output to SDI signal, which allows GI Genius to process the transmitted videos. All 100 selected videos will be processed by GI Genius in two different ways. In the first set, only the frames from the last 3 seconds of each 15-second video will be processed, and CADx suggestions will appear only in that final 3-second segment. This first set will be shown to endoscopists who are allocated to the intervention arm. In the second set, all frames from the entire 15-second duration will be processed, and CADx suggestions will be overlaid on every frame. This first set will be shown to endoscopists who are allocated to the control arm.
Study Type
INTERVENTIONAL
Allocation
RANDOMIZED
Purpose
DIAGNOSTIC
Masking
SINGLE
Enrollment
70
The investigators showed the CADx suggestion during the 15-second playback of the video
During the 15-seconds polyp video the CADx suggestion appear only in the last 3 seconds of the video
Clinical Effectiveness Research Group
Oslo, Norway
Research Group in Gastrointestinal Oncology Ourense
Ourense, Spain
Sensitivity of the optical diagnosis of neoplastic lesions.
\- Sensitivity of each endoscopist and in each arm in the optical diagnosis of neoplastic or non-neoplastic lesions with high confidence level
Time frame: Through study completion, an average of 1 year
The reliance level on artificial intelligence, measured using the C value of the signal detection theory
Measured by the C value of the signal detection theory
Time frame: Through study completion, an average of 1 year
Discrimination (d´) level of neoplastic lesions based on the signal detection theory
Measured by the d' value of the signal detection theory
Time frame: Through study completion, an average of 1 year
Receiver Operating characteristic (ROC) curve to determine overall discrimination in the signal detection theory
Overall discrimination assessment by the d' measured by the Receiver Operating characteristic Curve
Time frame: Through study completion, an average of 1 year
Proportion of high confidence diagnosis
Proportion of lesions classified as neoplastic or non-neoplastic with high confidence diagnosis
Time frame: Through study completion, an average of 1 year
Association between the reliance level (C value) on AI and endoscopists age,sex, level of expertise in colonoscopy or CADx, confidence level and area of procedence
Association between the reliance level (i.e. the response bias (c) that measures the shift regards to the ideal observer or criteria of the responses) on AI and endoscopists' age, sex, level of expertise in colonoscopy, CADx, confidence level and area of procedence in optical diagnosis.
Time frame: Through study completion, an average of 1 year
Specificity of the optical diagnosis for neoplastic lesions.
We are going to calculate the specificity of each endoscopist in each arm for the optical diagnosis of neoplastic o non-neoplastic lesions with high confidence level
Time frame: Through study completion, an average of 1 year
Positive and negative predictive values and accuracy of the optical diagnosis for neoplastic lesions.
We are going to calculate the positive, negative and accuracy of the optical diagnosis of neoplastic and non-neoplastic lesions for each endoscopist and in each arm
Time frame: Through study completion, an average of 1 year
Sensitivity of the optical diagnosis for neoplastic lesions by endoscopists' age, sex, level of expertise in colonoscopy and CADx and area of procedence.
Sensitivity sub analysis by endoscopist age, sex, level of expertise in colonoscopy and CADx,and area of procedence
Time frame: Through study completion, an average of 1 year
Specificity of the optical diagnosis for neoplastic lesions by endoscopists' age, sex, level of expertise in colonoscopy and CADx and area of procedence.
Specificity sub-analysis by the endoscopist age, sex, level of expertise in colonoscopy and CADx and area of procedence
Time frame: Through study completion, an average of 1 year
Positive and negative predictive values and accuracy of the optical diagnosis for neoplastic lesions by endoscopists' age, sex, level of expertise in colonoscopy and CADx and area of procedence.
Sub-analysis by positive predictive value, negative predictive value and accuracy for the optical diagnosis of neoplasia or non-neoplasia by endoscopists age, sex, level of expertise in colonoscopy and CADx and area of procedence
Time frame: Through study completion, an average of 1 year
Association between endoscopists' sensitivity in optical diagnosis of neoplasia and their reliance level on CADx.suggestions
Time frame: Through study completion, an average of 1 year
Survey responses.
The endoscopists of both arms will answer the following three questions in the RedCap application with yes / no response to measure how psychological interventions affect their behaviors: * Did you have enough time to critically assess CADx suggestions? * Did you get exhausted by this task? * Do you think the timing of presenting CADx you experienced today is optimal in real clinical practice?
Time frame: Through study completion, an average of 1 year
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.