This study looks at whether a clinician-facing artificial intelligence (AI) tool can help outpatient doctors spend less time reviewing medical records and documenting care. The tool creates brief summaries of existing medical record information to support routine clinical work. This study examines how use of this tool affects clinicians' workload, time spent in the electronic health record, and overall experience with documentation. The goal is to better understand whether AI documentation support tools can improve efficiency and reduce burden for clinicians in outpatient specialty practice. Eligible UNC outpatient specialists may be invited to complete surveys and, if they qualify, are randomly assigned either to receive access to the tool (Evidently) at the beginning of the study period or to continue their usual workflow for eight weeks before receiving access. The study does not recruit patients and does not change medical care for patients.
Background and aim: This is a behavioral and health-information-technology study focused on clinical workflow. It does not target a specific patient disease or condition; it evaluates whether a clinician-facing artificial intelligence (AI) tool (Evidently) that produces brief summaries of existing electronic health record information can reduce documentation burden and improve efficiency and experience for outpatient specialists. Design: Interventional, randomized, parallel-group trial. Eligible clinicians are randomly assigned either to receive access to Evidently at the start of the study period or to continue usual documentation workflow for eight weeks before receiving access. The primary comparison window is eight weeks per arm description in the protocol. Setting and population: Recruitment targets outpatient attending physicians affiliated with UNC Faculty Practice (UNCFP) or UNC Medical Group (UNCMG) in the United States who can consent and complete study activities in English, with age limits and inclusion of birth-assigned sex categories as specified in the approved IRB application. Participants are recruited by invitation through professional clinical channels; the study does not recruit the general public. Eligibility (summary): Screening confirms attending physician role; UNCFP or UNCMG affiliation; primary specialty in the protocol-eligible list (for example cardiology, pulmonology, gastroenterology, neurology, oncology, surgery, or anesthesiology); at least two half-day outpatient sessions per week (each at least 3.5 hours); fewer than 25 percent of outpatient sessions jointly staffed with trainees or advanced practice providers at or above the protocol threshold; no participation in a prior Evidently pilot; and no anticipated absence from work longer than one week during the eight-week study period. Individuals who do not meet these criteria are not enrolled. Procedures for participants: Participation includes an eligibility and baseline survey (approximately 10 to 15 minutes), randomization, use or non-use of Evidently according to assignment during the randomized period, and a brief follow-up survey. There is no monetary compensation. The consent materials describe voluntary participation, confidentiality protections, and that participation is not reported to supervisors or leadership and does not affect employment status, evaluation, or clinical responsibilities. Outcomes (high level): Outcomes emphasize clinician-reported workload; time and effort related to reviewing and summarizing medical records and using the electronic health record; task-load and related experience measures; and, for those assigned to early access, usability and perceived impact of the tool. The study does not enroll patients and does not assign interventions to patients; any linkage of survey responses to workflow data for analysis follows the approved consent and privacy plan. Operational identifiers needed to link survey responses to workflow systems, if collected, are described in the approved consent and are handled per institutional policies and the IRB-approved protocol.
Study Type
INTERVENTIONAL
Allocation
RANDOMIZED
Purpose
HEALTH_SERVICES_RESEARCH
Masking
NONE
Enrollment
150
Clinicians randomized to the intervention arm will receive access to the AI clinical summarization tool following randomization and completion of baseline study procedures. Clinicians randomized to the control arm will continue usual chart review and documentation practices for the duration of the study. Participation in the study and use of the AI clinical summarization tool are voluntary. Clinicians assigned to the intervention arm are not required to use the tool as part of routine care. Use of the AI clinical summarization tool will be characterized using system-generated utilization metrics provided by the vendor. These metrics may include measures such as the number of sessions (i.e., the number of times the tool is invoked), the number of patients for whom the tool is used, and, where available, the number of customized summaries generated. All utilization measures will be aggregated at the clinician level over the study period.
UNC Health
Chapel Hill, North Carolina, United States
Change in Clinician Cognitive Load
Cognitive load measured using the Physician Task Load (PTL) scale, adapted from the National Aeronautics and Space Administration (NASA) Task Load Index. The PTL includes four items-mental demand, physical demand, effort, and time pressure-each rated on a 0-100 scale with reference to a typical clinical workday during the past 1-2 weeks. Item scores will be summed to produce a continuous composite score ranging from 0 to 400, with higher scores indicating greater cognitive workload. The primary outcome is the clinician-level change in PTL score from baseline to the end of Study Month 2, analyzed under an intention-to-treat framework.
Time frame: Baseline to end of Study Month 2
Change in time in clinical review
Epic Signal-derived time spent in clinical review, capturing time spent reviewing patient charts and clinical information.
Time frame: Baseline to Study Month 2
Change in time outside work hours
Epic Signal-derived time spent outside scheduled work hours, including evenings and unscheduled days where available.
Time frame: Baseline to Study Month 2
Change in clinic preparation time
Self-reported time spent preparing for clinic, during scheduled and outside scheduled work hours (minutes per half-day clinic).
Time frame: Baseline to Study Month 2
Change in new patient record review time
Self-reported time spent reviewing and summarizing a new patient's record prior to clinic.
Time frame: Baseline to Study Month 2
Change in burnout
Change in clinician burnout assessed using a validated single-item self-report measure. The item asks clinicians to rate their current level of burnout on a 5-point ordinal scale ranging from 1 (no symptoms of burnout) to 5 (completely burned out). Higher scores indicate greater burnout. The outcome is the clinician-level change in this single-item burnout rating from baseline to the end of Study Month 2.
Time frame: Baseline to Study Month 2
Number of tool sessions invoked
Total number of times the AI clinical summarization tool (Evidently) was invoked by each clinician during the study period, as recorded by vendor-provided system logs. Reported as a count aggregated at the clinician level (intervention arm only).
Time frame: Study Month 2
Number of patients with tool use
Total number of unique patients for whom the AI clinical summarization tool (Evidently) was used by each clinician during the study period, as recorded by vendor-provided system logs. Reported as a count aggregated at the clinician level (intervention arm only).
Time frame: Study Month 2
Proportion of eligible visits with tool use
Proportion of eligible outpatient visits during the study period for which the AI clinical summarization tool (Evidently) was used by each clinician, as derived from vendor-provided system logs and scheduling data. Reported as a percentage aggregated at the clinician level (intervention arm only).
Time frame: Study Month 2
Context of use
System-generated data describing clinical contexts in which Evidently was used (e.g., new versus return visits, specialty workflows), supplemented by structured survey items where necessary (intervention arm only).
Time frame: Study Month 2
Usability and fit
Usability and fit are assessed using a study-specific Likert agreement questionnaire comprising three statements about Evidently, each rated on a 5-point Likert agreement scale scored from 1 (Strongly disagree) to 5 (Strongly agree); higher scores indicate more favorable usability and fit. The statements are: Evidently is easy to use; Evidently was easy to learn; Evidently fits the chart review and summarization needs of my specialty. Item-level distributions and/or summary scores (e.g., mean or median per item) will be reported at Study Month 2 (intervention arm only).
Time frame: Study Month 2
Accuracy and trust
Perceived accuracy and trust are assessed using a study-specific Likert agreement questionnaire comprising four statements, each rated on a 5-point Likert agreement scale scored from 1 (Strongly disagree) to 5 (Strongly agree); higher scores indicate more favorable perceptions of accuracy and trust. The statements address whether Evidently-generated summaries are generally accurate; whether Evidently highlights clinically relevant information; whether Evidently surfaces relevant information the respondent may not have found independently; and whether summaries from Evidently are at least as good as summaries the respondent would otherwise prepare. Item-level distributions and/or summary scores will be reported at Study Month 2 (intervention arm only).
Time frame: Study Month 2
Impact on day
Single-item measure assessed using a 5-point Likert agreement scale scored from 1 (Strongly disagree) to 5 (Strongly agree); higher scores indicate stronger agreement that Evidently helps the clinician get through their clinical day more easily. The item prompt is: Evidently helps me get through my clinical day more easily. Distribution of responses will be reported at Study Month 2 (intervention arm only).
Time frame: Study Month 2
Time savings
Self-reported perceived time savings per half-day clinic attributable to Evidently (intervention arm only).
Time frame: Study Month 2
Impact on care
Single-item measure assessed using a 5-point Likert agreement scale scored from 1 (Strongly disagree) to 5 (Strongly agree); higher scores indicate stronger agreement that Evidently helps the clinician provide better patient care. The item prompt is: Using Evidently helps me provide better patient care. Distribution of responses will be reported at Study Month 2 (intervention arm only).
Time frame: Study Month 2
Overall satisfaction with AI clinical summarization tool assessed by single-item Likert agreement scale
Overall satisfaction with the AI clinical summarization tool (Evidently) is assessed using a single-item measure on a 5-point Likert agreement scale scored from 1 (Strongly disagree) to 5 (Strongly agree); higher scores indicate greater overall satisfaction. The item prompt is: Overall, I am satisfied with Evidently. The outcome is reported as the distribution of responses (frequency and percentage per category) at Study Month 2 (intervention arm only).
Time frame: Study Month 2
Open-ended feedback
Qualitative open-ended feedback collected via a single optional free-text item on the end-of-study questionnaire (no numeric scale is used for this measure). The item invites additional narrative feedback about Evidently, including what worked well, where Evidently was most helpful, what was frustrating or less useful than expected, and suggestions for improvement or new functionality. Responses will be analyzed using qualitative content or thematic analysis. Results will be reported as major themes with counts or proportions of respondents whose responses were assigned to each theme where appropriate, plus representative anonymized quotations. If concise enough, a tabulation of the most common paraphrased response categories may be presented; lengthy narrative responses will be summarized thematically rather than as a ranked list of full verbatim answers (intervention arm only).
Time frame: Study Month 2
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.