Relational Playbook Pilot Study

Purpose: To implement and evaluate the Relational Playbook, an innovative workforce development intervention developed in Dr. Gilmartin's VA Career Development Award using a pilot, site randomized trial design with a mixed method process evaluation. Aim 1: Test the implementation, feasibility and acceptability of the Relational Playbook intervention, coaching strategy, and study procedures in 6 VA Cardiac Catheterization Labs (CCL) and follow-them for 1 year. All 6 sites will implement the Playbook. CCLs will be randomized to enhanced leadership coaching implementation support (n=3) or standard implementation support (n=3). The enhanced group will receive 6 months of virtual leadership coaching support with Dr. Gilmartin, an associate leadership coach with the VA National Center for Organization Development. The feasibility and acceptability of the Playbook intervention and coaching strategy will be assessed by the VA Collaborative Evaluation Center (VACE) using measures developed by Weiner et al. and select Reach, Effectiveness, Adoption, Implementation and Maintenance (REAIM) measures. Aim 2: VACE will conduct a mixed methods process evaluation of intervention implementation to understand 1) intervention adaptations, ease of use, engagement, usefulness, and 2) implementation speed, costs, barriers, facilitators, and unintended consequences, guided by Realist Evaluation. The hypothesis is that leadership coaching will optimize Playbook implementation compared to standard implementation support. Background and Significance The Veterans Health Administration (VA) is prioritizing employee well-being due to crisis levels of clinician burnout and turnover. The National Academy of Medicine has proposed the creation of supportive work environments to enhance employee well-being. The study team conducted a longitudinal, sequential, mixed methods study that identified supportive learning environments in VA cardiac catheterization laboratories (CCLs). The team identified positive relationships between more supportive learning environments and increased job satisfaction, lower burnout, lower intent to leave, lower CCL turnover in the previous 12 months and increased perceived safety climate. Interviews and literature reviews were conducted to understand the survey findings. These data informed the creation of The Relational Playbook which includes research-based resources and 50 evidence-based practices that have been distilled into short, easily digestible content suitable for busy healthcare teams. The Playbook has been piloted in a single CCL but has not been rigorously implemented and evaluated in diverse clinical settings. The purpose of the proposed study is to implement and evaluate The Relational Playbook in six CCLs and test leadership coaching as a novel implementation strategy. The hypothesis is that leadership coaching will optimize Playbook implementation. Leadership coaching is a novel implementation strategy that enables proactive, knowledgeable, and supportive leadership behaviors. Although coaching has been embraced in the VA, through Whole Health and the VA Leadership Coaching Program, no studies have examined the impact on innovation implementation with clinical managers. All sites will implement the Playbook. CCLs will be randomized to enhanced leadership coaching implementation support (n=3) or standard implementation support (n=3). The enhanced group will receive 6 months of leadership coaching support. The standard implementation group will receive logistical support, but no alternative to coaching, such as advisement. Six sites are anticipated as sufficient to generate data to inform a future trial. The advantage of comparing leadership coaching to standard implementation support (e.g., no coaching), is the ability to rigorously evaluate an enhanced approach compared to "usual care" in a real-world setting. There are no data available that does not support the study hypothesis. There are no similar studies currently under way or completed, indicating the proposed study is unnecessary. Preliminary Studies The study team conducted a longitudinal, sequential, mixed methods study to identify supportive learning environments in VA cardiac catheterization laboratories (CCLs). In 2018, 294 CCL physicians, nurses, and technicians from 68 VA CCLs were surveyed to examine supportive learning environment factors. Respondents identified the presence of training programs, openness to new ideas, and respectful interaction as strengths. The lack of structured knowledge transfer (e.g., huddles, crisis communication) and low use of forums for improvement were areas for improvement. The team also conducted unit-level analysis of 28 CCLs. One ranked as a supportive learning environment, 24 as mixed, and four as low learning environments. In 2020, the study team repeated the survey with 231 clinicians from 67 CCLs and identified positive relationships between more supportive learning environments and increased job satisfaction, lower burnout, lower intent to leave, lower CCL turnover in the previous 12 months and increased perceived safety climate. Based on preliminary work, the team interviewed 13 CCL leaders and staff from 1 supportive, 4 mixed and 1 low ranked CCL to understand how and why these learning environments were created. Five themes emerged. A review of the literature revealed the scientific rationale for these themes, which are conceptually grounded in the fields of positive psychology, team science, servant leadership and the VA Whole Health and Clinical Team Training Models. These data informed the creation of The Relational Playbook which includes research-based resources and 50 evidence-based practices that have been distilled into short, easily digestible content suitable for busy healthcare teams. Six CCL and LHS-HRO experts reviewed the Playbook. The experts found the interventions to be valid, acceptable (4.37/5; 1-5 ascending Likert scale), appropriate (4.28/5) and feasible for implementation (3.94/5). Research Methods Study Design and Research Methods: This study is a pilot, site randomized trial of VA CCLs with an embedded mixed methods process evaluation. Dr. Gilmartin and the implementation team will oversee the launch of the Playbook intervention, then Dr. Gilmartin will deliver the 1:1 leadership coaching implementation strategy over 6 months. Dr. Leonard and the VA Collaborative Center of Evaluation (VACE), an expert group of evaluators independent from Dr. Gilmartin, will collect study measures from CCL managers, physicians and staff that will be aggregated to the CCL level. Study measures will be collected at baseline, 6 and 12 months. The primary aim is to implement and establish the feasibility and acceptability of the Playbook intervention combined with a leadership coaching implementation strategy. The study team will collect baseline and 6-month data from the Learning Environment Survey with employee well-being items to determine power for an effectiveness trial. The secondary objectives are to 1) evaluate adaptations and identify how and why the intervention was or was not feasible and acceptable; 2) if CCL nurse managers integrate the Playbook interventions into daily activities, making it part of the culture; and 3) to gain insight into considerations and methods for a future trial. The central hypothesis is that leadership coaching will be a more feasible and acceptable approach to support Playbook implementation and the cultivation of supportive learning environments than implementation without coaching. The inclusion of a mixed methods process evaluation to evaluate Playbook implementation, but in which data will not be used during the conduct of the study to influence the process, presents a novel contribution to the LHS-HRO literature. The process evaluation is a methodological approach that will provide insight into how LHS-HRO interventions function more generally, beyond their application in CCLs. VA CART, the Office of Cardiology, Office of Nursing Services and REBOOT leaders will act as advisors, increasing the likelihood the study results will impact VA policy and operations. Interventions Relational Playbook. The Playbook consists of 5 chapters and hosts research-based resources and 50 evidence-based interventions that are presented in short, easily digestible content to create supportive learning environments. Playbook implementation is guided by a 4-step process. * Step 1: Assess the current state (10 minutes). The CCL manager and team assesses the current state of their learning environment through completion of an on-line, 13-item Learning Environment Assessment Tool. The results are automatically scored and grouped by Playbook chapters. A summary report is immediately available and includes guidance on how to interpret results. * Step 2: Select areas for improvement (1 meeting). CCL manager works with team to identify areas of strength, where their work is at its best and should be maintained or spread. The CCL manager then will discuss opportunities for improvement and which Playbook chapters and interventions they will adopt. * Step 3: Implement and adapt interventions (standing item in meetings/huddles). CCL manager works with their team to develop implementation and adaptation PDSA cycles for the interventions. Over the 6-month intervention period, CCL managers are required to implement interventions from 2-3 Playbook chapters, depending on the team Tool results. Fidelity to the interventions is required and will be monitored by VACE to ensure the critical aspects of each practice are delivered. However, a CCL team may adapt the delivery method of the interventions and may implement additional interventions during the intervention period. * Step 4: Evaluate the impact (standing item in meetings/huddles). CCL manager and team develop feedback loops to monitor the performance of the interventions. CCL managers reflect on what they are learning, gauge impact while acknowledging any conflict or tension, detect emerging problems, identify, and implement solutions, and adjust the implementation process accordingly. Leadership Coaching. The coaching implementation strategy will provide enhanced implementation support to the three CCL nurse managers randomized to this group. The coaching mindset for this study is that CCL manager's success is due to their own efforts (i.e., self-belief). The coaching approach is being curious, asking powerful questions and actively listening. Leadership coaching was selected due to CCL manager input, prior success in the literature, and perceived feasibility. Dr. Gilmartin will schedule and conduct the 1:1 60-minute coaching sessions every 4 weeks for 6 months on Microsoft Teams. The coaching process is standardized and Dr. Gilmartin will follow the VA National Center for Organization Development (NCOD) Coaching Curriculum, which uses the GROW Framework (Grow, Reality, Options, Will). Dr. Gilmartin will follow the NCOD-GROW framework for every coaching session and document specific details of the coaching conversation for each session in the NCOD Coaching Portal. The documentation will be used by the VACE team to monitor fidelity using the NCOD Coaching Curriculum Fidelity Assessment Guide. The documentation will also facilitate replication of the method in subsequent studies. The literature indicates spacing the sessions over 6 months and following the NCOD GROW manualized curriculum will allow new leadership and team habits to form and fidelity assessment. 1:1 coaching was selected over group coaching to cultivate a psychologically safe atmosphere with the manager. Any adaptations to the coaching schedule or approach will be tracked as part of the coaching records in the NCOD coaching portal to allow for comparison across sites. This study is a pilot, site randomized trial with an embedded mixed methods process evaluation. The team is piloting a randomized trial with embedded evaluation, compared to the one-group, pre-post study design commonly used in feasibility studies, to determine if randomization and the process evaluation is feasible in this context. The study team has enrolled 6 CCLs and will collect staff level data (range: 8-40 per CCL) and follow-them for 1 year. All sites will implement the Playbook. CCLs will be randomized to enhanced leadership coaching implementation support (n=3) or standard implementation support (n=3). The enhanced group will receive 6 months of leadership coaching support. The standard implementation group will receive logistical support, but no alternative to coaching, such as advisement. Six sites are anticipated as sufficient to generate data to inform a future trial. Study Protocol The 6 sites will be stratified into 3 tiers (high, medium, low) based on 2022 percutaneous coronary intervention volume. Within each tier, one site will be randomized to the enhanced implementation group and the other to the standard implementation group. This will ensure equal allocation between groups with different site characteristics. An allocation table will be generated in R statistical software and uploaded to a password-protected folder. All set-up, implementation, and evaluation activities will be scheduled by the research team, at the convenience of participants, to reduce the burden on nurse managers. The nurse managers' responsibility will be to implement the Playbook interventions, participate in coaching (if in the Enhanced Group), and participate in data collection. Once randomization is complete, the Enhanced and Standard groups will be invited to watch at their convenience, brief introductory videos and participate in a site-specific 1-hour study launch meeting to learn about the Playbook intervention and data collection expectations. VACE will collect baseline data, then both groups will be sent a link to the Playbook SharePoint site which includes the 13-item Assessment Tool. CCL teams will complete the Tool as step one of the four-step process outlined above. The Enhanced Group will schedule their initial coaching session. Over the subsequent 6 months, both groups will receive educational nudges to continue Playbook implementation through an automated monthly Qualtrics survey. The survey is part of the Playbook and is designed to collect adoption and fidelity data measures. Aim 1 (intervention phase) will end at 6 months and Aim 2 (evaluation phase) will end at 1 year. Implementation of the interventions and data collection will be deemed part of an employee's tour of duty. However, CCL physicians and staff can elect not to participate in the Playbook interventions at any time. Participating CCLs will be responsible for employee salaries. Dr. Gilmartin and team, plus VACE materials and salaries will be covered by pilot IIR and Dr. Gilmartin's current research funds. Methods Research Aim 1 Data Collection: This study uses a mixed methods approach to evaluate the feasibility and acceptability of the intervention and study procedures. The VACE team will collect data from several sources, guided by the Reach, Effectiveness, Adoption, Implementation and Maintenance (REAIM) framework. At baseline and 6 months (pre-post intervention), trends in learning environment and employee well-being (i.e., job satisfaction, burnout, turnover, and safety climate scores) will be collected from managers, physicians, and staff (range: 8-40 per CCL) through the validated 64-item Learning Environment Survey. At baseline, readiness for change will be collected from the nurse manager through a brief, theory-based measure of organizational readiness to change. At 6 months, feasibility and acceptability will be assessed by nurse managers using the measures developed by Weiner et al. At 6 months, managers will be interviewed regarding their progress on quality improvement goals and manager growth and development. Additional measures will include adoption and fidelity, time spent, and adaptations collected from managers using the monthly Qualtrics survey (3-5 pre-populated items and open text option). Coaching notes will be analyzed for the frequency of and attendance at coaching sessions, plus fidelity to the NCOD GROW Model. Maintenance data will be collected from the manager at 1 year through a one-time Qualtrics survey asking for the number of Playbook interventions still in use. Research Aim 2 Data collection: For the interviews of CCL managers, physicians, and staff is guided by the principles of Realist Evaluation, which seeks to understand what works, for whom, and under what circumstances. The interview guide includes questions that query the experience of learning about and participating in the intervention, coaching, changes to CCL team workflow required to implement the Playbook, organizational barriers, or facilitators to effective implementation. Participants will also be asked about the ease of use, engagement, usefulness, adaptations, speed, and cost of implementation, any unintended consequences, and maintenance. To capture multiple perspectives, VACE will interview the CCL manager, and a sample of CCL staff and VAMC leaders who interact with CCL managers (e.g., Chief Nurse, Chief of Cardiology, perioperative leaders - up to 4 per site). The interview guides will be pilot tested to ensure clarity and relevance to the study questions. Interviews will last between 20-30 minutes. Interviews will be recorded and transcribed verbatim. Data will be analyzed using rapid content analysis to identify themes related to contextual influences and practices related to implementation and evaluation of the intervention. Anticipated Risk \& Potential Benefits This research study is funded by a VA research grant. However, it is also part of CCL quality improvement efforts and takes place in the context of clinical operations. As a result, if individuals choose to participate, their identity and participation will be known by CCL leadership and co-workers. As the Playbook interventions are team-focused, participants will be able to identify each other. Breach of confidentiality is a risk for CCL managers and staff who will be identified by name during study launch and data collection. There is a risk of disclosure of information about workplace relationships and learning environment that could be perceived as sensitive, such as colleagues' attitudes about new CCL procedures, or perceptions of CCL leader engagement or support. Additionally, due to the project's association with VA operations, the following people or groups may know the VA employees participating in the study: I.The CCL manager, director and/or Chief of Nursing or Cardiology or other supervisors and co-workers at participating facilities II.The research and evaluation team members III.VA (to monitor studies) IV.The VA committees that oversee research, including the Institutional Review Board that oversees the safety and ethics of VA studies. Protection Against Risk While the study team will make every effort to keep information confidential, no system for protecting confidentiality can be completely secure in this operational context. Individuals may choose to share with others the fact of their participation or opinions about the Playbook and coaching and process evaluation. However, the study team will not disclose participation in the study or responses to any questions to anyone outside the research team. Interviews and coaching sessions will be conducted via Teams at a time convenient to the participants, and the recording and interview notes will be saved directly to an access-controlled data folder on a secured server at VA Eastern Colorado Healthcare System. The study team will take stringent precautions to protect the confidentiality of subjects' personal information. Informed Consent Procedures This project has received a Waiver of Documentation of Informed Consent and a Waiver of HIPAA Authorization for this subject population. Participating sites (VA employee subjects) have been recruited based on study site selection. Study information statements emphasize that the site may withdraw from the study at any time. Reporting Although methods to enhance relationships, communication, teamwork, and high reliability are available to CCLs and fall within existing standards of care, and although this study is a feasibility and acceptability pilot and not a clinical trial, there is a need to have a system in place for identifying and reporting adverse clinical events that occur during the study period. This will be accomplished through the existing real-time adverse event system built into CART. All procedures done in VA CCLs, are monitored in real-time through CART for adverse events as part of routine clinical care. CART information is available in real-time and as a result, patient safety events are continuously reported and reviewed. All adverse events are reported to the CART clinical committee and local facility representatives. The study team has stringent data safety procedures in place to ensure subject protection. A data safety monitoring plan, even for non-clinical research, is required as per VHA Handbook 1200.05. Additionally, the Denver/Seattle Center of Innovation has an established Data Security Plan and framework the team will work under. Under these circumstances, this study will not have a Data Monitoring Committee. Data Analysis Plan: Data Analysis will be conducted in R statistical software and Atlas.ti. The feasibility, acceptability, adoption, and fidelity data for both groups will be summarized, described, and compared. Analysis of trends in the learning environment and employee well-being over time will be based on a repeated-measures analysis framework. The model for each end point will include a common intercept for both groups, indicator variables by month and interactions between these time variables and an indicator variable for the intervention group. The interview data will be analyzed using qualitative content analysis to pull out major concepts with attention to differences between the two groups. The quantitative data will be triangulated with the qualitative data to assess for points of convergence and help explain how and why changes occurred. Integration of the mixed methods data will be achieved by reporting results together. The quantitative results will be reported first, followed by qualitative quotes or themes that support or refute the quantitative results. Knowledge to be Gained: This study will contribute to the LHS and HRO literature and practice by implementing and establishing the feasibility and acceptability of a Playbook intervention combined with a leadership coaching implementation strategy to cultivate supportive learning environments. To create supportive learning environments and integrate research-based practices into care, frontline leaders must use evidence-based methods. This is especially important as clinical teams are expected to continuously learn, adapt, and maintain high reliability (2023 VA HSR\&D cross-cutting principles and research priorities). The potential impact of the Playbook intervention and coaching strategy includes the creation of supportive learning environments that enhance employee well-being. However, the Playbook has not been implemented in a clinical setting. The study team must establish the feasibility and acceptability of the Playbook combined with the coaching implementation strategy prior to effectiveness testing. This will inform future studies to assess the impacts of the Playbook intervention on the health of VA employees and the Veterans they serve.