Artificial Intelligence-assisted HER2 Expression Assessment in Urothelial Carcinoma Based on Imaging-pathology Omics

Overview

This study aims to build upon previous research by using artificial intelligence methods to fuse multimodal data from imaging and pathology to construct a predictive model for HER2 expression in urothelial carcinoma. The model's performance will be validated and optimized using a multicenter cohort study, ultimately achieving accurate and rapid prediction of HER2 expression. This will guide precise decision-making for further HER2-targeted therapy and improve patient prognosis. Big data analysis and deep learning will also assist physicians in more accurately diagnosing the disease and developing personalized treatment plans. The research findings will promote the integration and development of artificial intelligence technology with the healthcare industry in the application of multimodal data from clinical, imaging, and pathology perspectives.

1. Patient Data Collection and Construction of a Multimodal Dataset and Sample Library This study will construct a standardized, high-quality multimodal urothelial carcinoma data warehouse. The core is to collect patient data from those pathologically diagnosed with urothelial carcinoma who possess preoperative multiparametric MRI (T1WI, T2WI, DWI/ADC) and paired digital H\&E whole slide images (WSI). All cases must use expert-reviewed immunohistochemical results as the gold standard label for HER2 status, and be accompanied by complete clinical data. We will establish strict inclusion and exclusion criteria to ensure data quality, and utilize a professional platform to de-identify, standardize, and correlate clinical, pathological, and imaging data for storage, laying a solid data foundation for subsequent analysis. 2. Extraction and Screening of Imaging/Pathological Features of Urothelial Carcinoma Patients This stage aims to extract quantitative features from macroscopic imaging and microscopic pathological images. For MRI, radiologists will manually delineate the three-dimensional region of the tumor (VOI), and then use radiomics tools to extract a large number of quantitative features describing tumor intensity, shape, and texture heterogeneity. For H\&E pathological sections, a deep learning/machine learning model was used to automatically segment the tumor region. High-dimensional deep features from millions of image patches were extracted using pre-trained convolutional neural networks, and then aggregated into a feature vector for the entire section using a multi-instance learning framework. Finally, statistical tests and learning methods such as LASSO were combined to select the most relevant and stable feature subset for HER2 status assessment, eliminating redundancy and providing input for model construction. 3. Constructing an AI-Assisted HER2 Assessment and Prediction Model for Urothelial Carcinoma The core task is to integrate the selected multimodal features and construct a high-precision prediction model. We will explore various machine learning algorithms (such as XGBoost and Random Forest) and deep learning architectures, focusing on multimodal fusion strategies: including late-stage fusion that simply splices together radiomics, pathological deep features, and clinical features, and mid-stage fusion (such as using attention mechanisms) that involves feature interaction in the middle layers of the model, to capture deep complementary information between macroscopic imaging and microscopic pathology. The final output of the model is not a simple binary division, but rather provides a continuous probability value (0-1) for HER2 positivity, providing more decision-making basis for clinicians, thereby achieving non-invasive preoperative prediction. (4) Model Validation and Optimization To rigorously evaluate the model's generalization ability and clinical applicability, we performed cross-validation and hyperparameter optimization on an internal dataset, and conducted blinded testing. Evaluation metrics included statistical indicators such as sensitivity, specificity, positive predictive value, and negative predictive value. Simultaneously, interpretability technologies such as Grad-CAM were used to generate heatmaps to locate key imaging areas and pathological morphological features, enhancing physicians' understanding and trust in the model, and potentially uncovering new biological insights.