The rapid advancement of artificial intelligence (AI) has expanded its applications in healthcare, particularly in diagnostic assistance, intelligent triage, and patient interaction. Hepatobiliary and pancreatic diseases (such as liver cancer, pancreatic cancer, cirrhosis) are characterized by insidious onset, rapid progression, low early-diagnosis rates, and poor prognosis. However, grassroots medical institutions in China face challenges including physician shortages, variable patient health literacy, and incomplete initial information collection, leading to high misdiagnosis/missed diagnosis risks. Recent breakthroughs in large language models (LLMs) and multi-agent systems (MAS) offer new solutions. LLMs enable advanced natural language processing, while MAS coordinates specialized agents for complex decision-making. Integrating MAS with medical LLMs could create intelligent pre-consultation systems that systematically collect patient symptoms, risk factors, family history, and lifestyle data to enhance diagnostic efficiency. This study aims to develop a MAS-based pre-consultation system for hepatobiliary-pancreatic diseases featuring four specialized agents ("guidance agent," "medical history agent," "risk assessment agent," and "summary generation agent"). The system will simulate clinical reasoning to generate structured diagnostic reports for physicians. Research Objectives: Develop a specialized multi-agent framework combining LLMs to simulate clinical diagnostic logic and standardize symptom collection Enhance pre-consultation data integrity through intelligent dialogue focusing on key disease indicators Generate structured diagnostic summaries highlighting critical symptoms and risk factors Establish foundation for clinical validation and application through expert evaluation and user feedback This pre-diagnostic tool will assist physicians rather than replace clinical judgment, promoting safe, effective AI applications in early disease screening and tiered healthcare systems.
With the rapid development of artificial intelligence (AI) technologies, their applications in the healthcare field have become increasingly widespread, particularly demonstrating significant potential in diagnostic assistance, intelligent triage, and patient interaction. Hepatobiliary and pancreatic diseases (e.g., liver cancer, pancreatic cancer, bile duct cancer, cirrhosis) are characterized by insidious onset, rapid progression, low early diagnostic rates, and poor prognosis. Clinical diagnosis and treatment demand high requirements for early recognition and precise evaluation. However, grassroots medical institutions in China commonly face challenges such as insufficient specialist physicians, heterogeneous patient health literacy, and inadequate initial information collection during first visits, leading to a high risk of missed or incorrect diagnoses and delays in optimal intervention timing. In recent years, large language models (LLMs) have achieved breakthrough advancements in natural language understanding and generation, providing a technical foundation for constructing intelligent, personalized medical dialogue systems. Multi-agent systems (MAS) simulate collaborative workflows among multiple specialized roles, enabling more complex and structured decision-making processes with notable advantages in task decomposition, knowledge integration, and dynamic reasoning. Integrating multi-agent architectures with medical LLMs offers the potential to develop intelligent pre-consultation systems with domain-specific expertise, enabling systematic collection and preliminary analysis of critical patient information-including symptoms, risk factors, family history, and lifestyle-to enhance the efficiency and completeness of medical consultations. This study aims to develop an intelligent pre-consultation LLM system for hepatobiliary and pancreatic diseases based on a multi-agent architecture. By establishing clearly defined "guidance agents," "medical history collection agents," "risk assessment agents," and "physician summary generation agents," the system will simulate clinical diagnostic logic, proactively guide patients through structured disease presentations, and automatically generate initial diagnostic reports compliant with clinical standards for physician reference. This system seeks to assist grassroots physicians in improving early recognition of hepatobiliary and pancreatic diseases, optimize outpatient resource allocation, and enhance patient healthcare experiences. Research Objectives: This study aims to design, develop, and preliminarily validate an intelligent pre-consultation LLM system for hepatobiliary and pancreatic diseases based on a multi-agent architecture, exploring its feasibility and potential value in clinical pre-consultation scenarios. Specific objectives include: Establishing a specialized multi-agent collaborative framework: Leveraging LLM technology, design a multi-agent system capable of division of labor and collaboration, incorporating modules such as guidance, medical history collection, preliminary risk screening, and report generation. By simulating the diagnostic logic and reasoning pathways of clinical physicians, the system will achieve systematic, structured information collection for hepatobiliary and pancreatic disease-related symptoms. Enhancing the completeness and accuracy of pre-consultation information: Through intelligent dialogue, proactively guide patients to review and articulate their medical history, prioritizing key risk factors for hepatobiliary and pancreatic diseases-including jaundice, abdominal pain, weight loss, alcohol consumption history, history of viral hepatitis, and family history of cancer-to improve patient self-reporting completeness and reduce information omissions caused by memory biases or unclear expression. Generating structured initial diagnostic assistance reports: Automatically integrate patient-reported symptoms and medical history into pre-consultation summaries adhering to clinical documentation standards, highlighting critical physical signs and risk alerts. This will assist physicians in rapidly grasping patient profiles, shortening outpatient consultation times, and improving diagnostic efficiency. Laying the foundation for subsequent clinical validation and application: Through small-scale simulation testing and expert evaluation, preliminarily verify the system's usability and clinical alignment, collect feedback from physicians and patients, and optimize interactive workflows. This will provide scientific evidence and ethical compliance support for future prospective clinical studies and product development. This research is not intended for direct clinical diagnosis or treatment decisions but rather as an information-assistance tool for physicians prior to consultations. It aims to promote safe, effective, and equitable applications of AI in the early screening of hepatobiliary and pancreatic diseases and tiered healthcare delivery systems.
Study Type
OBSERVATIONAL
Enrollment
400
Patients first complete a full interaction with the multi-agent system until the Arbiter confirms the medical record is error-free. The system then generates a structured "Case Characteristics" (CC) summary and preliminary diagnostic recommendations via the Oracle Agent. However, the complete AI output is not displayed to the subsequent attending physician. The physician conducts an independent consultation following standard clinical protocols, and their medical records are solely used to maintain clinical workflow integrity and are not evaluated as part of this study. The core assessment objective for this group is the concordance between the AI-generated final medical records and the gold-standard reference.
After patients finalize the CC through interaction with the multi-agent system, the structured "Case Characteristics" (excluding Oracle-generated diagnostic advice to avoid over-guidance) are pushed in a standardized format to the corresponding physician's electronic workstation. Physicians may reference this summary before formal consultation to adjust their questioning focus, verify information accuracy, or supplement missing details. The physician's final written medical record serves as the primary evaluation object for this group.
Concordance Rate Between AI-Generated Medical Records and Gold Standard
Proportion of AI-generated "Case Characteristics" summaries that match the gold standard (established by expert panels) in key diagnostic elements (e.g., symptom description, risk factors, physical findings).
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
Diagnostic Accuracy of Physician Documentation With AI Summary Reference
Proportion of physician-completed medical records in Experimental Group 2 that meet predefined quality criteria (completeness, diagnostic relevance, alignment with gold standard).
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
Proportion of physician medical records meeting predefined quality criteria without AI support
Proportion of physician-completed medical records that meet predefined quality criteria (completeness, diagnostic relevance, alignment with gold standard) in the absence of AI assistance.
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
Average physician consultation duration
Comparison of average consultation times across study groups.
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
Average pre-consultation preparation time for physicians
Time spent by physicians reviewing AI-generated case summaries before consultation.
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
Patient satisfaction measured by validated questionnaires
Patient-reported experience measured via validated satisfaction questionnaires. Scores range from 0 to 100; higher scores indicate greater satisfaction, assessing ease of AI interaction and perceived care quality.
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
Physician satisfaction with AI workflow
Physician survey evaluating perceived efficiency, usability, and clinical utility of AI tools. Survey scores range from 0 to 100; higher survey scores represent better satisfaction with the AI system.
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
Frequency of AI-related adverse events and workflow disruptions
Documentation of AI-related errors, workflow disruptions, or safety concerns. Count of participants experiencing safety or workflow issues.
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
System Usability Scale (SUS) scores for the AI interface
System Usability Scale (SUS) is a standardized usability questionnaire; total score ranges from 0 to 100. Higher SUS scores correspond to better perceived usability of the AI interface, completed by physicians and patients.
Time frame: From enrollment through completion of the index outpatient visit and chart assessment, assessed up to 7 days
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.