Prospective User Study and Multicenter Validation of Multimodal Medical Imaging Large Models

The Third Affiliated Hospital of Southern Medical University310 enrolled

Overview

Following model development and locking, the fixed model is evaluated in prospectively collected CT cohorts from two centers. The study is observational and does not affect clinical care. A subset of cases is used in a randomized crossover reader study.

After model locking, CT data are prospectively collected at two centers for observational validation without retraining or parameter adjustment. Model outputs do not influence patient management. A subset of eligible cases is selected for the randomized crossover reader study.

Study Type

OBSERVATIONAL

Enrollment

310

Conditions

Diagnostic Imaging Common Systemic Diseases Artificial Intelligence (AI)

Interventions

Standalone Radiologist InterpretationOTHER

Radiologists interpret the medical images independently without any assistance from the AI model to establish a baseline performance.

AI-assisted Radiologist InterpretationOTHER

Radiologists interpret the same set of medical images with the assistance of the multimodal medical imaging large model to evaluate the improvement in diagnostic performance.

Eligibility

Sex: ALLMin age: 18 YearsHealthy volunteers:

Medical Language ↔ Plain English

Inclusion Criteria: * Patients who underwent CT examinations for common systemic diseases. * Imaging data must have confirmed clinical reference standards, expert consensus, or pathological diagnosis. * Availability of complete DICOM format images with standard acquisition protocols. Exclusion Criteria: * Poor image quality (e.g., severe motion or metal artifacts) that precludes definitive diagnosis. * Cases with incomplete clinical or pathological reference standards. * Corrupted image files or duplicate cases.

Locations (1)

The Third Affiliated Hospital of Southern Medical University

Guangzhou, Guangdong, China

Outcomes

Primary Outcomes

Case-level Diagnostic Accuracy and Area Under the ROC Curve (AUC)

Evaluation of case-level diagnostic accuracy (defined as the proportion of diagnostic decisions matching the clinical ground-truth label) and discrimination performance (measured by AUC) to compare unaided radiologist performance versus AI-assisted performance.

Time frame: Up to 1 week per evaluation period

Secondary Outcomes

Diagnostic Efficiency (Reading and Reporting Time)

Measurement of diagnostic efficiency recorded as the time (in seconds) taken by radiologists to complete the case review and generate findings, with and without AI assistance.

Time frame: Up to 1 week per evaluation period

Inter-rater Agreement (Fleiss' Kappa)

Assessment of diagnostic consensus and inter-rater consistency among participating radiologists measured using Fleiss' kappa (κ).

Time frame: Up to 1 week per evaluation period

Clinical Report Quality and Semantic Accuracy Score

Assessment of AI-generated draft report quality evaluated by senior experts on a 5-point Likert scale (focusing on semantic accuracy and clinical relevance) and automated metrics (GREEN and ROUGE-L).

Time frame: Up to 1 week per evaluation period

Data from ClinicalTrials.gov

This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.