This study protocol describes a type 1 hybrid effectiveness-implementation study evaluating a clinical decision support dashboard for neonatal pain management in a tertiary NICU. Using a pre- and post-implementation design, the study assesses whether the dashboard reduces the duration of inadequately managed neonatal discomfort during the first seven days of admission, while concurrently evaluating implementation outcomes such as acceptability, appropriateness, fidelity, and penetration. Quantitative EHR- and dashboard-derived data are complemented by qualitative interviews and observations to provide a comprehensive evaluation of both clinical impact and implementation processes.
This protocol outlines a single-center, type 1 hybrid effectiveness-implementation study evaluating an in-house developed clinical decision support dashboard designed to optimize pain management in the Neonatal Intensive Care Unit (NICU). The study is conducted at a level III-IV NICU and follows a pre-implementation (retrospective), mid-implementation, buffer, and post-implementation (prospective) design over approximately 26 months. The primary effectiveness objective is to determine whether implementation of the dashboard reduces the duration of inadequately managed neonatal discomfort during the first seven days of admission. Discomfort is operationalized as the cumulative duration a neonate is classified as "red" by the dashboard prioritization algorithm, indicating the need for immediate clinical attention. Pre-implementation "red" classifications are generated retrospectively from electronic health record data using the same algorithmic criteria applied prospectively post-implementation. The secondary objective is to evaluate the implementation process of the dashboard within routine NICU workflows. Implementation outcomes, based on the framework of Proctor et al., include acceptability, appropriateness, fidelity (implementation and effectiveness), and penetration. These outcomes are assessed using a mixed-methods approach, combining quantitative measures such as dashboard use metrics, prescribing patterns, and timing of pain assessments with qualitative data from semi-structured interviews and observations involving healthcare professionals and parents. The comfort-dashboard is implemented using a phased ward-based approach supported by targeted training, local champions, and ongoing facilitation. The study poses minimal risk, as the dashboard supports but does not replace clinical decision-making, and no changes are made to core algorithm logic or outcome definitions during the study. By integrating effectiveness and implementation evaluations, this protocol aims to generate clinically relevant evidence while supporting reproducibility, transparency, and future scalability of digital decision support tools in neonatal care.
Study Type
INTERVENTIONAL
Allocation
NON_RANDOMIZED
Purpose
OTHER
Masking
NONE
Enrollment
1,250
The comfort-dashboard designed to assist healthcare professionals in managing neonatal pain within the Neonatal Intensive Care Unit (NICU). The software aggregates and visualizes patient-specific data from connected systems, providing near-real-time metrics, patient prioritization, and clinical decision support.
Erasmus MC-Sophia Children's hospital
Rotterdam, Netherlands
Main study parameter/endpoint - Effectiveness evaluation
For the primary aim of our study, we will assess the effectiveness of the dashboard in improving the comfort of patients admitted to the NICU, defined by the difference in the duration of inadequately managed discomfort in neonates during the first seven days of admission. Here, the duration of inadequately managed discomfort is measured by the duration a patient has a 'red' status, as classified by the classification algorithm.
Time frame: Days 1-7 of NICU admission, assessed through study completion (approximately 13 months post go-live, November 2026).
System Usability Scale (SUS) score among healthcare professionals
The SUS is a 10-item standardized questionnaire administered to healthcare professionals. Each item is rated 1-5. Odd-numbered item scores are reversed; all scores are summed and multiplied by 2.5 to yield a normalized score (range 0-100; higher = better usability). The SUS score is interpreted as a percentile rank.
Time frame: At 6 months post go-live (week 32-35)
Qualitative assessment of acceptability of the dashboard among healthcare professionals and parents
Semi-structured interviews exploring perceived barriers and facilitators to implementation, usefulness, ease of use, and overall satisfaction with TMD-C, conducted with healthcare professionals and parents. Analysis uses direct content analysis guided by the Consolidated Framework for Implementation Research (CFIR).
Time frame: At 6 months post go-live (week 32-35)
Qualitative assessment of appropriateness of the implementation strategies within NICU workflows
Perceived fit, relevance, and compatibility of the chosen implementation strategies, assessed through semi-structured interviews and systematic observations with healthcare professionals, analyzed using CFIR constructs.
Time frame: Through study completion, ~13 months
Proportion of healthcare professionals attending training sessions (%)
Implementation fidelity measure: the proportion of healthcare professionals working in the NICU during the study period who attended at least one TMD-C training session.
Time frame: Weeks 0-7 (mid-implementation phase)
Use of loading doses at therapy initiation and dose-escalation (n; %)
The difference in the proportion of therapy initiations and dose-escalation events accompanied by a loading dose within the protocol-specified timeframe, comparing pre- and post-implementation phases, stratified by moment (initiation vs. escalation).
Time frame: Days 1-7 of admission through study completion, ~13 months
Continuous infusion duration in patients without discomfort (minutes)
The difference in the duration of continuous analgesic infusion in neonates classified as not experiencing discomfort, comparing pre- and post-implementation phases, stratified by drug and relevant patient characteristics.
Time frame: Days 1-7 of admission, through study completion (approximately 13 months post go-live)
Dose-escalation titration step size (mg/kg/hr)
The difference in starting dosage and height of subsequent dose escalation steps surrounding dose-escalation events, comparing pre- and post-implementation phases, stratified by drug and relevant patient characteristics.
Time frame: Days 1-7 of admission, through study completion (approximately 13 months post go-live)
Duration of oversedation (minutes)
The difference in the cumulative duration a patient is classified as oversedated, comparing pre- and post-implementation phases.
Time frame: Days 1-7 of admission, through study completion (approximately 13 months post go-live)
Proportion of timely pain measurements following key therapy events (%)
The difference in the proportion of pain measurements conducted within the protocol-specified timeframe after therapy initiation, dose-escalation, dose-decrease, and therapy discontinuation, comparing pre- and post-implementation phases, stratified by relevant patient characteristics.
Time frame: Days 1-7 of admission, through study completion (approximately 13 months post go-live)
Number of dashboard accesses per morning round (n)
The frequency of TMD-C access during morning rounds, used as a measure of penetration - the extent to which dashboard use has become part of routine NICU workflow.
Time frame: Through study completion, approximately 13 months post go-live
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.