Testing an AI Tool to Help Primary Care Clinicians With Specialty Consultation Questions

N/ANot Yet RecruitingNCT07706920

Stanford University15 enrolled

Overview

The goal of this study is to test an artificial intelligence (AI) tool called SAGE. SAGE helps primary care doctors with questions that often need a specialist. Primary care doctors are the doctors people usually see first. SAGE reviews a case and suggests what a specialist might advise. The main questions it aims to answer are: * Do doctors make sound, timely care decisions when they use SAGE? * Do those decisions match what a specialist would advise? Researchers will compare the decisions doctors make with and without SAGE. Doctors in the study will: * Review made-up patient cases (these are not real patients) * Make a decision for each case, their usual way and with SAGE

Study Type

INTERVENTIONAL

Allocation

Purpose

HEALTH_SERVICES_RESEARCH

Masking

NONE

Enrollment

Conditions

Decision Support Systems, Clinical Referral and Consultation

Interventions

Eligibility

Sex: ALLMin age: 18 YearsHealthy volunteers:

Medical Language ↔ Plain English

Inclusion Criteria: * Physician practicing in primary care internal medicine and/or family medicine * Attending physician, or resident physician at postgraduate year 2 (PGY-2) or higher * Stanford-affiliated (faculty, staff, or trainee) * Able to complete a single, approximately 60-minute remote (Zoom) session conducted in English Exclusion Criteria: * Not currently practicing in primary care internal medicine or family medicine * Member of the study team or otherwise involved in the development of SAGE * Unable to complete the remote study session

Outcomes

Primary Outcomes

Clinical actionability of the clinician's decision

For each case, independent reviewers judge whether the physician's decision is clinically actionable, meaning the physician can act on it now rather than wait for a specialist. The outcome is the proportion of cases judged actionable, measured separately for decisions made with SAGE and for decisions made the standard way.

Time frame: Post-session independent review, through study completion (up to approximately 2 months)

Concordance of the clinician's decision with the specialist reference answer

Agreement between the physician's management decision and a specialist reference answer, rated by independent reviewers, with inter-rater agreement summarized. Assessed for decisions made with SAGE and for decisions made the standard way.

Time frame: Post-session independent review, through study completion (up to approximately 2 months)

Secondary Outcomes

System usability (System Usability Scale, SUS)

Perceived usability of SAGE measured with the 10-item System Usability Scale (SUS), scored from 0 to 100.

Time frame: End of study session (Day 1, approximately 60 minutes)

Order acceptance in the SAGE condition

Proportion of SAGE-suggested orders that the clinician accepts, assessed in the SAGE condition.

Time frame: During study session (Day 1)

Time to complete case review

Time (in minutes) taken to review each case and reach a management decision, compared between the SAGE and standard eConsult conditions.