LLM-Generated Lay Summaries for Brain MRI Reports

Overview

The goal of this clinical trial is to learn if a summary written by artificial intelligence (AI) helps adults understand brain MRI reports for headaches. The main question it aims to answer is: "Does adding a simple summary help readers correctly understand if a cause for the headache was found in the report?" Researchers will compare standard MRI reports to reports that include an AI-generated explanation to see if the extra summary improves understanding. Participants will: Read 6 fictional brain MRI reports online. Answer questions to check if they understood the results. Rate their satisfaction and if they feel they would need to ask a doctor for help.

Background Headaches account for approximately 2% to 4% of emergency department visits, representing about 450,000 consultations annually in France. While 95% of these cases are benign primary headaches, identifying secondary causes requiring urgent management is critical, often leading to increased use of neuroimaging such as MRI. However, radiology reports often contain complex medical jargon that can be difficult for patients and non-specialist physicians to understand, potentially causing confusion or anxiety. Large Language Models (LLMs) have demonstrated the potential to simplify complex medical text. While commercial models exist, open-weights models (which can be deployed locally to ensure data security) offer a promising avenue for clinical integration. This study aims to evaluate the efficacy of an AI-generated plain-language summary in improving patient understanding of brain MRI reports. Study Design This is a randomized, controlled, single-blind trial nested within the COMPARE e-cohort. The study uses a parallel-group design with a 1:1 allocation ratio. The entire study is conducted remotely via secure online forms. Participants The study recruits adult volunteers already enrolled in the COMPARE e-cohort. Participants must have sufficient proficiency in written French to read the reports and complete the questionnaires. No specific medical condition is required for inclusion, as the study uses fictional case scenarios. Intervention and Procedures Participants are randomized to one of two groups via a minimization procedure balancing history of brain MRI and known neurological pathology. Each participant is asked to read six fictional brain MRI reports simulating common emergency headache scenarios. The six reports cover three clinical situations: two with normal results, two with incidental findings not explaining the headache, and two with abnormalities explaining the headache. In the experimental group, participants receive the standard MRI report enriched with a structured summary paragraph generated by an open-weights LLM, inserted under the section Synthesis for the patient and non-radiologist physician. In the control group, participants receive the standard MRI report in its native version without the AI-generated summary. Outcome Measures Immediately after reading each report, participants complete a standardized questionnaire. The primary outcome is the comprehension of the report, measured by the accuracy of the response to the binary question: Is a probable explanation for the headache found in this report? Secondary outcomes include participant satisfaction measured on a Likert scale, perceived need for professional clarification, perceived ability to explain results to a relative, and projected anxiety levels. Statistical Analysis The primary analysis will compare the proportion of correct responses between groups using a mixed logistic regression model. This model will include the intervention group as a fixed effect and account for crossed random effects (participant and report) to manage intra-individual correlation and variability between clinical cases. The sample size is calculated to be 412 participants (206 per group) to detect a 10% difference in understanding with 95% power.