Background: The establishment of neuroinformatics as a distinct field has enabled the integration of computational biology and informatics to improve neurological research. This interdisciplinary approach enhances the capacity to integrate diverse datasets, unravel complex neural networks, and develop computational models that can improve clinical management. The investigators aim to evaluate whether an artificial-intelligence-based tool is effective in non-English-speaking regions. Hypothesis: Integrating a language model-based clinical assistance system within the neurology ward will significantly enhance the efficiency and accuracy of patient care by leveraging neuroinformatics principles. The investigators hypothesize that combining natural language processing and data analytics will improve diagnostic and treatment processes.
Research Design: A pre-post-intervention design will be used, measuring outcomes before and after the implementation of a neuroinformatics-driven clinical assistance system. Changes in diagnostic accuracy, treatment decisions, and workflow efficiency will be quantified. Workflow: 1. A neurologist evaluates each patient, and the patient signs informed consent. 2. A medical student manually uploads de-identified clinical information to a secure interface. 3. Data are analyzed by a large language model (LLM) system through a hospital-approved application. 4. A senior physician must approve any decision based on the LLM's recommendation. 5. For half of the prospectively enrolled participants, the LLM's recommendation is presented to the resident. Retrospective Data: 1. The investigators will extract data from 10,000 patients who were evaluated by a neurologist in the Emergency Department. 2. Clinical information (without patient identifiers) will be uploaded to a secure, hospital-approved LLM with a security key. 3. The model's output will be compared with actual clinical decisions and patient outcomes (e.g., mortality, discharge status).
Study Type
OBSERVATIONAL
Enrollment
1,000
Artificial Intelligence as a decision making tool
This intervention involves collecting and de-identifying clinical data from 10,000 patients previously evaluated by neurologists in the Emergency Department. The data are then used retrospectively to refine and fine-tune a large language model (LLM). No real-time clinical decisions or recommendations are made for these patients, as the purpose is to improve the model's accuracy and relevance for future use.
Rambam healthcare campus
Haifa, Israel
Proportion of correct diagnoses generated by the LLM compared to the final attending physician diagnosis.
* Description: Measures the percentage of times the LLM's recommended diagnosis matches the final attending physician's diagnosis. * Unit of Measure: Proportion (0.0-1.0) or Percent (0-100%) * How Assessed: Each LLM recommendation is compared to the final documented diagnosis in the medical record. The number of correct diagnoses is divided by the total number of cases.
Time frame: 3 years
Accuracy of Next-Step Recommendations
Proportion of correct or appropriate next-step management decisions recommended by the LLM. Description: Proportion of correct next-step management decisions recommended by the LLM, compared to either the attending physician's final plan or standard-of-care guidelines. Unit of Measure: Proportion (0.0-1.0) or Percent (0-100%) How Assessed: For each encounter, the LLM's recommended "next step" is recorded and deemed correct if it aligns with the attending plan or guideline.
Time frame: Up to 3 years.
Cost of Running the LLM
Average daily cost (in USD) to operate the large language model. Description: This outcome will measure the direct computational and licensing costs incurred by running the LLM in the clinical setting. Unit of Measure: US Dollars (per day). How It Is Assessed: The total daily spend for the LLM (e.g., cloud-compute fees, licensing fees) will be recorded and divided by the total number of patient encounters that day to derive an average cost.
Time frame: Up to 3 years
Staff Compliance With AI
This outcome will assess how often (compliance) and how willingly (acceptance) physicians, residents, and nurses use the LLM in eligible clinical scenarios. Description: Determines how often physicians, residents, and nurses use the LLM for eligible patient encounters. Unit of Measure: Proportion (0.0-1.0) or Percent (0-100%) How Assessed: Tracks the number of encounters in which the LLM is actually used, divided by the total number of eligible encounters. Higher percentages indicate greater compliance.
Time frame: Up to 3 years
Transparency/Explainability
Description: Evaluates how clearly the LLM's reasoning is communicated to clinical staff. Unit of Measure: Score on a 5-point Likert scale (1 = not at all clear, 5 = extremely clear) How Assessed: After each LLM-guided decision, the resident or attending physician rates the clarity of the model's explanation. A mean score is reported. Higher scores indicate better clarity.
Time frame: 3 years
Quality of LLM-Generated Clinical Reports
Accuracy and Completeness Score of the LLM-Generated Clinical Report This outcome evaluates how accurately and comprehensively the LLM summarizes a patient's clinical encounter and recommended plan in a structured report (e.g., discharge instructions, progress notes). Unit(s) of Measure: Quality Score based on a modified SOAP (Subjective, Objective, Assessment, Plan) Note Rating Scale, ranging from 0 to 10, where higher scores indicate better documentation quality. The score reflects the inclusion of required elements and correctness of clinical details. Assessment Method: Each LLM-generated report is independently evaluated by a panel of attending physicians or trained clinical staff. Using a standardized checklist, reviewers assess completeness (e.g., inclusion of symptoms, physical findings, diagnosis, and plan) and accuracy (e.g., correct medications, identifiers, and diagnoses), then assign a score based on the predefined rubric.
Time frame: Up to 3 years
Staff Acceptance of AI
Description: Evaluates how willingly staff integrate the LLM into their clinical workflow. Unit of Measure: 5-point Likert scale (1 = strongly disagree, 5 = strongly agree) How Assessed: After each LLM-assisted encounter, staff complete a brief survey about perceived helpfulness and willingness to reuse the LLM. Higher scores indicate greater acceptance.
Time frame: Up to 3 years
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.