Large Language Models To Improve the Quality of Care of Cardiology Patients

Overview

This study evaluates the impact of large language models (LLMs) versus traditional decision support tools on clinical decision-making in cardiology. General cardiologists will be randomized to manage real patient cases from a cardiovascular genetic cardiomyopathy clinic, with or without AI assistance. Each case will be assessed by two cardiologists, and their responses will be graded by blinded subspecialty experts using a standardized evaluation rubric.

Large language models have been shown to improve physician performance in simulated settings. Large language models have demonstrated promise in various healthcare contexts, including medical note-writing, addressing patient inquiries, and facilitating medical consultation. However, it remains uncertain whether large language models improve clinical reasoning of clinicians using real world cases. Clinicians dedicate years of training to develop expertise, with clinical knowledge a key component. Clinicians have different areas of expertise, from generalists spanning diseases of all organ systems and patients of all ages, to subspecialists dedicated to often a handful of diseases effecting a specific organ. Both skill sets are vital to a well-functioning medical system, as generalists generally care for patients and refer to specialists when dedicated, specialty knowledge is required. There is a paucity of specialists, and thus the quality of triaging and referral to specialists is of upmost importance. We hypothesis that large language models may be able help generalists management complex patients, and improve their triage to specialists and subspecialists. The scarcity of subspecialist medical expertise, particularly in rare, complex and life-threatening diseases, poses a significant challenge for healthcare delivery. This issue is particularly acute in cardiology where timely, accurate management determines outcomes. In this study, we will recruit General Cardiologists as participants who will be randomized to answer clinical management cases with or without access to a large language model. Each case is a real patient case of a patient referred to a subspeciality cardiovascular genetic cardiomyopathy clinic. Each case will be performed by two general cardiologists (one with access to a large language model and one without access). Each case has multiple components, and the participants will be asked to answer questions related to the management. Answers will be graded by independent, blinded subspeciality Cardiologists with expertise and training in genetic cardiomyopathies. An evaluation rubric was developed by 10 expert discussants.