AI-based System for Assessing Suspected Viral Pneumonia Related Lung Changes

Overview

The AI-based system designed to process chest computed tomography (CT) aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4). The retrospective study aims to demonstrate the clinical validation of the AI-based system. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined to provide evidence about the clinical efficacy of the AI-based system. The hypothesis is that the measures of clinical validation of the AI-based system differ by no more than 8% from those declared by the manufacturer.

The AI-based system designed to process chest CT aims to 1) detect the presence of pathologic patterns associated with interstitial changes in pneumonia; 2) highlight areas on the images with the probable presence of pathologies; 3) provide the physician with the results of image processing, including quantitative indicators of suspected viral pneumonia related lung changes according to visual pulmonary lesion grading system (CT0-4). This retrospective clinical study will provide the clinical validation of the AI-based system to analyze chest CT images and identify pathological patterns associated with interstitial changes in pneumonia. Clinical validation measures (sensitivity, specificity, accuracy, and area under the ROC curve) will be determined and compared with values declared by the manufacturer to provide evidence about the clinical efficacy of the AI-based system. The first stage of clinical validation is the collection of a verified labeled dataset. For this purpose, the dataset is collected, labeled, and verified by a research group. The verified dataset should include chest CT images without infiltrative and interstitial lung changes characteristic of viral pneumonia, including COVID-19-associated (CT-0) and chest CT images of all degrees of lung involvement CT-1 (≤25%), CT-2 (25-50%), CT-3 (50-75%), CT-4 (≥75%) \[1\]. Forming the verified dataset will allow reliable conclusions to be drawn upon completion of the clinical validation. The verified dataset must include a sufficient volume of chest CT images. The verified dataset must be de-identified to ensure the safety of patient personal data. The second stage of the clinical validation is assessing AI-based system performance by experts. For that purpose, the AI software is analyzed to identify radiological signs of viral pneumonia. Then an examination is made of the correctness of the quantitative assessment of lung damage associated with interstitial changes in pneumonia. The evaluation of both the ability to correctly identify signs of lung damage and to quantify the identified changes is carried out on the same verified dataset. The third stage of clinical validation is the calculation of clinical efficacy metrics (accuracy, sensitivity, specificity, area under the ROC-curve (AUROC) of the AI-based system by testing it on a verified data set. Testing of the hypothesis to verify the main diagnostic characteristics (sensitivity and specificity) declared by the manufacturer is planned by constructing a two-sided 95% confidence interval (CI), which should not differ by more than 8% from the declared values of 95% and 97%, respectively. Those. the lower limit of the 95% CI for sensitivity should not cross the 87% threshold, and the lower limit of the 95% CI for specificity should not cross the 89% threshold. All stages of the clinical trial must be under the control of the Principal Investigator. Randomization of images is not provided in this clinical study, because All CT images will be assessed by the research group and AI software. Also, this design does not involve blinding or masking of the research team. The evaluation of CT images by experts and the software is carried out independently, i.e. the results of each party's assessment are not known to the other party in advance.