Assessment of Artificial Intelligence Algorithms for ROTEM

Overview

The goal of this observational validation study is to evaluate whether artificial intelligence (AI) models can accurately interpret ROTEM (Rotational Thromboelastometry) data and provide appropriate treatment recommendations in adult patients undergoing elective cardiac or liver transplantation surgery. The main questions it aims to answer are: Can AI models (e.g., ChatGPT and Gemini ) accurately determine whether treatment is indicated based on ROTEM parameters? Can AI models correctly identify the type of coagulopathy (e.g., fibrinogen deficiency, platelet dysfunction)? Are the treatment recommendations from AI models concordant with expert clinical consensus? Researchers will compare the decisions made by AI models to a gold standard expert panel to see if AI models can match or approximate expert-level decision-making in interpreting ROTEM outputs. Participants will: Undergo elective cardiac or liver transplant surgery. Have standard ROTEM tests performed intraoperatively. Have their anonymized ROTEM data reviewed independently by: A panel of 3 clinical experts. AI models (ChatGPT and Gemini) using standardized prompts and ROTEM interpretation guidelines.

Rotational thromboelastometry (ROTEM) is a point-of-care viscoelastic testing method used to assess the coagulation status of patients undergoing high-risk surgical procedures, such as cardiac surgery and liver transplantation. While ROTEM-guided transfusion algorithms have improved clinical outcomes, the accurate interpretation of ROTEM results remains complex and heavily dependent on clinical experience. This prospective observational validation study aims to assess the accuracy and clinical decision-making performance of artificial intelligence (AI)-based language models in interpreting ROTEM findings. The study will compare the AI-based evaluations to expert consensus in terms of both diagnostic accuracy and treatment recommendations. De-identified ROTEM case data will be converted into structured clinical vignettes, which will be independently interpreted by at least three experienced clinicians (serving as the gold standard) and AI models. Each AI system will be prompted with ROTEM parameters in a standardized format and asked to assess coagulopathy type and suggest appropriate treatment options. ROTEM interpretation algorithms, such as Görlinger's protocol, will be provided as background context to ensure consistent guidance. The study will include adult patients (≥18 years) undergoing elective cardiac surgery or liver transplantation, provided complete and technically valid ROTEM results are available. The main outcome of the study is the agreement between AI-based and expert decisions regarding the need for treatment. Secondary outcomes include diagnostic classification of coagulopathy, concordance in treatment recommendations, and standard accuracy metrics (sensitivity, specificity, PPV, NPV, overall accuracy, and Cohen's Kappa). This study does not involve any direct patient interventions or changes in treatment based on AI output. All data will be anonymized before analysis, and informed consent will be obtained from all participants. As part of the AI evaluation and expert comparison, each ROTEM clinical scenario will be assessed based on a standardized set of 14 structured clinical questions. These questions are designed to determine both the presence and type of coagulopathy, as well as the appropriate treatment recommendations. The specific questions are: Is there evidence of coagulopathy based on the ROTEM findings? Do the ROTEM results indicate hyperfibrinolysis? Do the findings suggest the presence of residual heparin effect? Is there evidence of fibrinogen deficiency? Is there evidence of thrombocytopenia or platelet dysfunction? Do the results indicate coagulation factor deficiency? Do the findings suggest protamine overdose? Is treatment not required at this time? Is antifibrinolytic therapy indicated? Should protamine be administered? Should fibrinogen or fibrinogen-containing products be administered? Should platelet transfusion be performed? Should prothrombin complex concentrate (PCC) or fresh frozen plasma (FFP) be administered? If bleeding continues, should reassessment be done after 10-15 minutes? Each question will be answered as "Yes" or "No" by both the AI system and the expert panel, and the responses will be used to calculate diagnostic and treatment concordance metrics.