This study evaluates the safety, accuracy, and impact of an artificial intelligence (AI) tool designed to support patient education in breast cancer care for breast oncology patients under selected physicians care within the University of California, San Francisco breast cancer center and affiliate sites.
PRIMARY OBJECTIVES: I. Lead-in period: To evaluate the safety of the OpenEvidence CareConnect a Retrieval-Augmented Generation (RAG) Large Language Model (LLM) tool during the initial lead-in phase in patients with metastatic breast cancer. II. To evaluate the clinical accuracy and appropriateness of the OpenEvidence CareConnect RAG LLM tool in delivering general health and breast cancer-specific information. SECONDARY OBJECTIVES: I. To assess patient perceptions of the AI tool's helpfulness, alignment with other information sources, and its impact on communication with their oncology care team. II. To evaluate provider perceptions of the tool's usefulness and impact on communication and the quality of patient education. III. To compare patient experiences and perceptions between immediate access and delayed access to the AI educational tool. EXPLORATORY OBJECTIVES: I. To evaluate the changes in inter-visit communication between patient and care team with and without access to the AI tool. II. To understand patient concerns and educational needs related to breast cancer diagnosis and treatment. OUTLINE: The first 5 participants will be assigned to the Lead-In Phase. After which, the next 30 participants will be randomized to either an Intervention arm or a Waitlist Control arm. Participants on the Waitlist Control arm will cross over after 3 months to the Intervention arm. Surveys and provider assessments occur throughout, and adherence and safety are monitored during the lead-in. Each participant will be on study for approximately 6 months.
Study Type
INTERVENTIONAL
Allocation
RANDOMIZED
Purpose
HEALTH_SERVICES_RESEARCH
Masking
NONE
Enrollment
35
Participants will complete surveys to evaluate their experience using OpenEvidence CareConnect RAG LLM tool.
Retrieval-Augmented Generation (RAG) Large Language Model (LLM) tool
The provider will complete a survey to assess OpenEvidence CareConnect RAG LLM tool's usefulness and the quality of its patient education communication.
University of California, San Francisco
San Francisco, California, United States
Mean score of Safety and Risk Assessment (Lead-in and Provider)
Safety and Risk Assessment evaluates safety concerns based on provider responses using a 5-point Likert scale.
Time frame: Up to 90 months
Mean score of Provider Assessment of AI patient Education Tool Survey (Provider)
AI patient Education Tool Survey evaluates clinical accuracy and appropriateness of the OpenEvidence CareConnect tool based on provider responses using a 5-point Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree).
Time frame: Up to 180 days
Mean score of Assessment of Answer Accuracy and Content Appropriateness (Provider)
Provider will evaluate participant questions, answer accuracy and content provided within the OpenEvidence using 3-point scales.
Time frame: Uup to 180 days
Mean score of AI Tool Experience survey (Participant)
The AI Tool Experience Survey is a 40-question instrument to assess participants' perceptions of usability, usefulness, clinical integration, and support for shared decision-making. Each item is rated on a 5-point Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree).
Time frame: Up to 180 days
Mean score of Medical Information Experience Survey (Participant)
Medical Information Experience Survey is a 13-question instrument to evaluate how participants access and engage with medical information. Questions are measured using a 5-point Likert scale.
Time frame: up to 180 days
Mean Change of Score of Patient AI Tool Experience Survey (Participant)
The AI Tool Experience Survey is a 40-question instrument to assess participants' perceptions of usability, usefulness, clinical integration, and support for shared decision-making. Each item is rated on a 5-point Likert scale ranging from 1 (Strongly Disagree) to 5 (Strongly Agree). Comparisons of survey responses between the immediate and delayed access groups will be reported.
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.
Time frame: up to 180 days
Mean Change of Score of Medical Information Experience Survey (Participant)
Medical Information Experience Survey is a 13-question instrument evaluates how participants access and engage with medical information. Questions are measured using a 5-point Likert scale. Comparisons of survey responses between the immediate and delayed access groups will be reported.
Time frame: Up to 180 days