Clinical Validation of RBfracture for Diagnosing Trauma-related Musculoskeletal Injuries

Overview

The goal of this study is to determine if the computer software, RBfracture, developed by Radiobotics, helps primary care, emergency, and radiology clinicians more easily identify bone injuries caused by a traumatic impact (such as a fall or car collision). RBfracture uses artificial intelligence (AI) to analyze X-ray images of patients to identify fractures and joint dislocations visible on the X-ray images. RBfracture also identifies fluid buildup in the elbow and knee joints resulting from a fracture or dislocation. Sixteen clinicians will review X-ray images from 415 adult patients, who may have sustained a bone injury, to diagnose any injuries visible on their X-ray images. First, the clinicians will review half of the images with and half of the images without the help of the RBfracture software. After a 4-week break, the clinicians will once again review the same images. This time, the software's help will be switched, so it is unavailable for the images the clinicians previously reviewed with it, and available for the images they reviewed without it. The number of correct and incorrect diagnoses made by the clinicians when they were helped by the software will be compared to the number of correct and incorrect diagnoses made by the clinicians when they did not receive any help from the software. This comparison will reveal if using the software helps clinicians to diagnose more injuries and miss less injuries.

RBfracture, developed by Radiobotics (Copenhagen, Denmark), is a decision-support software solution for computerized detection and diagnosis of acute fractures, joint dislocations, joint effusion, and lipohemarthrosis. To demonstrate improved diagnostic performance of the intended users when assisted by RBfracture, a fully-crossed multi-reader multi-case (MRMC) study will be conducted. 415 study exams will be retrospectively obtained through stratified sampling with enrichment to represent all anatomical regions supported by the software. Sixteen independent clinicians representing various clinical roles and years of experience will participate as study readers. The reference standard will be established by American Board of Radiology (ABR)-certified radiologists with a minimum of 3 years of experience post certification. All exams will be reviewed and annotated by two musculoskeletal (MSK) radiologists, independently. Annotations will be done in a medical-grade reading environment, using diagnostic quality monitors. If required, any disagreement will be adjudicated by a third specialized radiologist. In both assisted and unassisted sessions, readers will localize every unique fracture, joint dislocation, joint effusion, and lipohemarthrosis. For each finding, readers will provide a bounding box and a confidence score (1-5). Additionally, at the exam level, readers will indicate the presence or absence of each injury type along with a confidence score (0-100). At the case level, ROC curves will be generated for each reader under both device-assisted and unassisted reading scenarios. The AUC of the ROC curve per reader in each reading scenario, and the difference in the reader-averaged AUC between device-assisted and unassisted reading scenarios will be calculated. The difference in reader-averaged AUC between the two reading scenarios will be reported as the primary outcome to demonstrate improved diagnostic performance of the intended users. Exam-level sensitivity, specificity, and average false positives, and injury-level sensitivity and AUC-AFROC will be presented as secondary outcomes.