Validation and Standardization of Paramagnetic Rim Lesion (PRL) Identification in Multiple Sclerosis Across MAGNIMS Centers

Overview

Multiple sclerosis (MS) is a chronic inflammatory and neurodegenerative disease of the central nervous system. Paramagnetic rim lesions (PRLs) are a subtype of chronic active MS lesions showing an iron-laden inflammatory rim on susceptibility-sensitive MRI. PRLs mark a more aggressive disease course and have recently been incorporated, together with the central vein sign, into the updated diagnostic criteria for MS. Before PRLs can be used reliably in multicenter clinical practice and trials, the way they are identified must be shown to be consistent across different centers, raters, and scanners. This is a retrospective, multicenter, observational validation study. Using previously acquired 3-Tesla brain MRI scans from MS patients across 15 MAGNIMS centers, expert raters count PRLs according to the NAIMS consensus definitions. The study measures how consistently PRLs are identified between centers, raters, and scanners (inter-rater, inter-center, and inter-scanner agreement), using statistical methods such as the intraclass correlation coefficient (ICC) and Bland-Altman analysis. Where systematic sources of disagreement are found, the PRL definitions are refined through a structured Delphi consensus process and re-tested on an independent set of scans. The goal is to standardize PRL identification and provide an evidence-based basis for consistent, reproducible PRL assessment to support future multicenter studies. No new patients are recruited and no new scans or biological samples are collected; only pre-existing, de-identified imaging and clinical data are used.

Design: Retrospective, multicenter, observational validation study coordinated by IRCCS Ospedale San Raffaele (Milan, Italy) within the MAGNIMS network. Data source: A pool of previously acquired 3T brain MRI scans fulfilling NAIMS-PRL acquisition criteria (3D-FLAIR, SWI, post-gadolinium 2D/3D T1-weighted; submillimeter in-plane resolution) is available across 15 MAGNIMS centers. A preliminary survey identified 1402 eligible scans. To participate in the study, each MAGNIMS center is required to contribute scans from a minimum of 30 patients and to have at least one expert rater with clinical or research experience in PRL evaluation. From this pool, two selected datasets of 150 scans each, corresponding to 10 randomly selected scans per center per scanner, are drawn for evaluation. The selected datasets are balanced across MS phenotype, age, and lesion load. Evaluation procedure: Phase 1 - First evaluation: Selected Dataset 1 is rated for PRLs by at least one expert rater from each participating MAGNIMS center, applying the NAIMS-PRL criteria. Inter-rater, inter-center, and intra-center reproducibility of PRL counts is assessed using intraclass correlation coefficient and Bland-Altman analysis by the coordinating center together with the Department of Neurology, Medical University of Vienna. Phase 2 - Disagreement analysis: Sources of disagreement between expert raters are identified at the level of individual PRL designations on the MRI images. Phase 3 - Refinement and Delphi consensus: If consistent sources of disagreement are identified, proposed refinements to the PRL definitions are translated into operationalized consensus statements and ratified through an electronic Delphi survey using a 5-point Likert scale. Consensus requires at least 75% agreement, with a minimum 80% response rate per round. Phase 4 - Second evaluation: The refined criteria are re-tested on Selected Dataset 2 by the same expert raters, with repeated ICC and Bland-Altman analysis. If excellent reproducibility, defined as ICC ≥0.90, is reached, the process is complete; otherwise, Phases 2 to 4 are repeated. Clinical correlations: Mandatory clinical data, including demographics, MS phenotype, EDSS, and disease duration, and, where available, optional measures, including Timed 25-Foot Walk and 9-Hole Peg Test, are integrated to explore associations between PRL burden and clinical disability. Data handling: All imaging and clinical data are de-identified, either anonymized or, where agreed in the Data Transfer Agreement, pseudonymized, before transfer to the coordinating center. MRI data in NIfTI format undergo defacing using fsl\_deface from the FMRIB Software Library. No new recruitment, MRI scans, biological samples, or study-specific interventions are performed. Retrospective data acquired between January 2017 and November 2025 are used.