This is a prospective observational clinical study designed to evaluate the performance of artificial intelligence (AI) algorithms applied to upper aerodigestive tract (UADT) video-endoscopy. The study assesses three main tasks: lesion detection (localization), classification (benign vs malignant), and segmentation of tumor margins. AI algorithms will be applied to endoscopic video data acquired during routine clinical practice without influencing clinical decision-making. The system will process images in real time and store data for subsequent analysis. AI outputs will be compared with physician assessment and reference standard histopathology to evaluate diagnostic performance.
The artificial intelligence algorithms developed will be employed in the analysis of laryngeal lesions for 3 tasks: * Task 1: Computer aided diagnosis (CADx): the algorithm provides a differential diagnosis between benign and malignant neoplasms (binary classification) and the exact histology (multiclass classification). During the UADT video-endoscopy in the outpatient clinic, the physician performs the video-endoscopy and selects and captures n.3 WL and n.3 NBI significant frames of the lesion. The AI model records the classification output of the algorithm that the physician cannot access. The predicted pathologic results will be finally displayed as two different classifications along with the probability of each prediction (0% to 100%) as estimated by the AI algorithm: a first binary classification "neoplastic" or "non-neoplastic," and a second multiclass classification with the exact histology. The physician subsequently, based on the endoscopic examination, will write the suspected diagnosis (benign vs. malignant lesion and the actual histology) in the appropriate patient chart. Next, the physician reviews the screenshot taken and makes sure the lesion is visible in every one of them. Retrospectively, an investigator (blinded to the physician's assessment) will review the AI processed frames with the resulting CADx classifications and mark the AI-processed diagnosis in the patient chart. Once biopsied, the final histology of the lesion analyzed by definitive histopathological examination is recorded in the patient chart by the investigator. The investigators will finally compare the two recorded diagnoses (CADx and physician) with the definitive histology. * Task 2: Computer aided detection (CADe): the algorithm, through the representation of a rectangle (bounding box), localizes the lesion during the video-endoscopy in the outpatient clinic in real-time. During the UADT video-endoscopy, the physician performs the video-endoscopy as for standard-of-care procedure. In parallel, the AI model processes in real-time the endoscopic video and records the output of the algorithm (which the physician cannot access). The physician captures n.3 WL and n.3 NBI significant frames of the lesion. Moreover, n.3 frames where no lesions are visible are captured as negative controls. Later, the physician reviews the screenshot taken and makes sure to label the frames where the lesion is visible as "positive cases" and the frame where the lesion is not visible as "negative cases". The investigators will finally assess if the lesion was detected by the CADe system to define a "true positive". Similarly, to define a true negative, the CADe system should have not output a bounding box in the majority of the "negative cases" frames. * Task 3: Computer aided segmentation (CASe): the algorithm analyzes the neoplasm margins and provides a delineation mask. In the operating room setting, once the lesion to be resected is identified with a 0° telescope, the surgeon captures n.1 WL and n.1 NBI close-up photographs that exemplify the superficial lesion margins. The same procedure is repeated with a 70° optics and other two photographs are acquired. The frames taken are then saved and analyzed by the AI algorithm, which will perform the segmentation task. The surgeon will be blinded to the AI prediction. Later, the surgeon will draw the margins of the lesion according to her/his evaluation of each captured frame. The annotated frame will be saved so that it can be analyzed at a later time. Afterwards, in cases where positive superficial margins are identified by histopathologic examination, the surgeon-designed margins and the AI model ones will be compared to see if there was any difference in the suggested margin.
Study Type
OBSERVATIONAL
Enrollment
283
UZ Leuven
Leuven, Flemish Brabant, Belgium
IRCCS Ospedale Policlinico San Martino
Genova, GE, Italy
Hospital Clínic de Barcelona
Barcelona, Barcelona, Spain
Negative Predictive Value of the CADx Algorithm for Malignant or Premalignant Upper Aerodigestive Tract Lesions
Negative Predictive Value (NPV) of the computer-aided diagnosis (CADx) algorithm for classifying UADT lesions as malignant/premalignant versus benign/non-neoplastic, using definitive histopathology as the reference standard. The CADx final classification will be based on the majority rule across selected white-light and narrow-band imaging frames. NPV = true negatives / (true negatives + false negatives). The pre-specified performance target is NPV ≥ 90%.
Time frame: From index outpatient UADT video-endoscopy until definitive histopathology result is available, assessed up to 60 days after endoscopy.
Sensitivity of the CADe Algorithm for Localization of Upper Aerodigestive Tract Lesions
Sensitivity of the computer-aided detection (CADe) algorithm for localizing UADT lesions with a bounding box. A true positive is defined as localization of the lesion area by a bounding box in the majority of physician-labeled lesion-positive captured frames. Sensitivity = true positives / (true positives + false negatives).
Time frame: At index outpatient UADT video-endoscopy, with blinded post-processing assessment performed up to 30 days after endoscopy.
Median Intersection Over Union Between CASe Segmentation and Surgeon-Drawn Lesion Margins
Median overlap between the AI-generated segmentation mask and the lesion margin area drawn by the surgeon on intraoperative endoscopic images. Intersection over Union (IoU) = area of overlap / area of union. Values range from 0 to 1; higher values indicate greater agreement.
Time frame: At intraoperative pre-resection endoscopy, with assessment performed after image annotation up to 30 days after surgery.
Median Dice Similarity Coefficient Between CASe Segmentation and Surgeon-Drawn Lesion Margins
Median Dice Similarity Coefficient (DSC) between the AI-generated segmentation mask and the lesion margin area drawn by the surgeon on intraoperative endoscopic images. Dice Similarity Coefficient = 2 × area of overlap / (AI segmented area + surgeon-drawn area). Values range from 0 to 1; higher values indicate greater agreement.
Time frame: At intraoperative pre-resection endoscopy, with assessment performed after image annotation up to 30 days after surgery.
WL-NPV vs. NBI-NPV of CADx classification
Negative Predictive Value (NPV) of CADx classification calculated using only the three selected white-light frames, compared with definitive histopathology, vs. NPV of CADx classification calculated using only the three selected narrow-band imaging frames, compared with definitive histopathology. The final AI-result will be calculated based on the majority rule of the 3 WL and 3 NBI frames computed separately.
Time frame: From index outpatient UADT video-endoscopy until definitive histopathology result is available, assessed up to 60 days after endoscopy.
Clinician-Reported Usability Score for the AI Endoscopy System
Usability of the AI endoscopy system assessed using standardized usability questionnaires administered to clinicians after use of the AI system. Questionnaire scoring will be interpreted according to the selected questionnaire manual, with higher scores indicating greater usability.
Time frame: Assessed after clinician use of the AI system during study procedures, up to 20 months after study initiation.
Sensitivity, Specificity and Accuracy of CADx histology prediction
Sensitivity, Specificity and Accuracy of the CADx algorithm for histology prediction, compared with definitive histopathology.
Time frame: From index outpatient UADT video-endoscopy until definitive histopathology result is available, assessed up to 60 days after endoscopy.
F1 Score of CADx Classification
F1 score of the CADx classification output compared with definitive histopathology.
Time frame: From index outpatient UADT video-endoscopy until definitive histopathology result is available, assessed up to 60 days after endoscopy.
Area Under the Receiver Operating Characteristic Curve of CADx Classification
AUC of the ROC curve for CADx classification of UADT lesions compared with definitive histopathology.
Time frame: From index outpatient UADT video-endoscopy until definitive histopathology result is available, assessed up to 60 days after endoscopy.
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.
Sensitivity, Specificity and Accuracy of human physician histology prediction
Sensitivity, Specificity an Accuracy of the treating physician's suspected diagnosis compared with definitive histopathology.
Time frame: From index outpatient UADT video-endoscopy until definitive histopathology result is available, assessed up to 60 days after endoscopy.
Specificity of the CADe Algorithm for Localization of Upper Aerodigestive Tract Lesions
Proportion of physician-labeled lesion-negative frames/cases in which the CADe algorithm does not output a bounding box in the majority of negative-control frames.
Time frame: At index outpatient UADT video-endoscopy, with blinded post-processing assessment performed up to 30 days after endoscopy.
Accuracy of the CADe Algorithm for Localization of Upper Aerodigestive Tract Lesions
Overall proportion of correctly classified lesion-positive and lesion-negative cases/frames by the CADe algorithm.
Time frame: At index outpatient UADT video-endoscopy, with blinded post-processing assessment performed up to 30 days after endoscopy.
Positive Predictive Value of the CADe Algorithm for Localization of Upper Aerodigestive Tract Lesions
Positive predictive value of CADe bounding-box output for lesion localization. PPV = true positives / (true positives + false positives).
Time frame: At index outpatient UADT video-endoscopy, with blinded post-processing assessment performed up to 30 days after endoscopy.
Negative Predictive Value of the CADe Algorithm for Localization of Upper Aerodigestive Tract Lesions
Negative predictive value of CADe absence of bounding-box output for lesion localization. NPV = true negatives / (true negatives + false negatives).
Time frame: At index outpatient UADT video-endoscopy, with blinded post-processing assessment performed up to 30 days after endoscopy.
Percentage of Positive Superficial Margin Cases in Which the AI-Predicted Tumor Area Is Wider Than the Surgeon-Drawn Area
Among cases with positive superficial margins on final histopathology, percentage of cases in which the AI-predicted tumor area extends beyond the surgeon-drawn margin at the affected margin.
Time frame: At intraoperative pre-resection endoscopy, with assessment performed after image annotation up to 30 days after surgery.