ImpleMentAll - Towards Evidence-based Tailored Implementation Strategies for eHealth

Overview

The ImpleMentAll (IMA) project aims to examine the effectiveness of tailored implementation compared to usual implementation of Internet-based Cognitive Behavioural Therapy (iCBT) for patients suffering from common mental disorders in routine practice. Common mental health disorders account for an alarming proportion of the global burden of disease. Being regarded as an evidence-based psychotherapeutic eHealth intervention, Internet- based Cognitive Behavioural Therapy (iCBT), has the potential to answer to this societal challenge by providing an efficacious and efficient treatment from which more people can benefit. ImpleMentAll will develop, apply, and evaluate tailored implementation strategies in the context of on-going eHealth implementation initiatives in the EU and beyond. The objectives are: 1. To develop a generic Integrated Theory-based Framework for Intervention Tailoring Strategies (the ItFits-toolkit) for data-driven tailored implementation of evidence-based eHealth services. 2. To demonstrate the impact of the ItFits toolkit on the implementation of eHealth for common mental disorders. 3. To disseminate the validated toolkit in various healthcare contexts across Europe. Following a stepped-wedge trial design, the ItFits-toolkit will be introduced in twelve implementation sites in nine countries, and evaluated for its effectiveness in obtaining implementation success. An in-depth process evaluation using a realist evaluation methodology will provide information about the particularities of tailored implementation and the application of the ItFits-toolkit in real implementation work. The resulting ItFits-toolkit will enable data driven evaluation of eHealth implementation projects and its methods, materials, and strategies will provide concrete guidance on tuning implementation interventions to local determinant of practice across a variety of health care systems.

ImpleMentAll aims to examine the effectiveness of tailored implementation (i.e. the ItFits-toolkit) compared to usual implementation of Internet-based Cognitive Behavioural Therapy (iCBT) in routine practice in twelve implementation sites in nine countries. The project has a total duration of 51 months divided in three phases: 1) preparation (Jan 2017 - May 2018), 2) trial period (June 2018 - September 2020), and 3) analysis and reporting (October 2020 - March 2021). The preparation phase includes localising the generic study protocol to local settings and for obtaining the required ethical clearances. In the trial period, the ItFits-toolkit will be put into practice and data will be collected in each implementation site. In the final phase, the data will be cleaned and analysed. Depending on the results and conclusions of the study, the ItFits-toolkit will be further improved by adapting it to the findings. iCBT, an innovative psychotherapeutic intervention, has been found to be an efficacious treatment alternative for a range of common mental health disorders. The iCBT service aims to combine innovative Internet technology with a broad evidence-based definition of CBT. The iCBT services that are being implemented apply different delivery formats ranging from self-help, to therapist guided and blended with face-to-face therapy for depression and/or anxiety disorders and/or in comorbidity with somatic symptom disorder. iCBT is currently being implemented in twelve different sites in primary and specialised care in Italy, Spain, Germany, France, the Netherlands, Denmark, Kosovo, Albania and Australia. All these sites are in different phases providing a natural laboratory for the IMA study. A generic 'Integrated Theory-based Framework for Implementation Tailoring Strategies' toolkit (the ItFits-toolkit) is developed and will in introduced in these 12 implementation sites at randomly chosen moments. ItFits functions as an online self-help toolkit by which users are guided through the process of tailoring site-specific implementation strategies. The ItFits-toolkit includes four modules that implementers need to work through: 1) identifying and prioritising implementation goals and determinants of practices, 2) matching up implementation determinants to strategies, 3) designing a plan for carrying out strategies in a local context, and 4) applying strategies, and reviewing progress. In each of these four modules, evidence-informed materials such as iCBT relevant determinants of practices and implementation strategies, are included as well as methods for engaging with stakeholders. By introducing the ItFits-toolkit in settings with ongoing implementation of iCBT services, a comparison of the effectiveness and efficiency of the ItFits-toolkit with Implementation-as-Usual (IAU) is possible. IAU refers to any existing efforts to embed and integrate iCBT within an organisation. The main research hypothesis is that the Itfits-toolkit will lead to better implementation outcomes than IAU does. Reaching better outcomes can be expressed in two dimensions: the capability to establish the desired effects (i.e. implementation effectiveness) and its efficiency (i.e. implementation costs in relation to uptake and normalisation outcomes) by which the effects came about. Implementation effectiveness is defined in terms of uptake and as the extent to which a service is regarded as being a normal part of practice, i.e. normalisation degree. The question is whether the ItFits-toolkit leads to a positive impact on uptake and normalisation. Parallel to the main objective, the efficiency of the ItFits-toolkit in reaching the implementation outcomes as compared to IAU will be explored. Organisational readiness for change will be investigated to corroborate the uptake and normalisation outcomes adding to the development of measurement instruments for implementation processes. In addition, the perceived satisfaction and usability of the ItFits-toolkit will be explored for potential improvement of the toolkit. An in-depth qualitative process evaluation using Realist Evaluation methods will be conducted alongside this implementation effectiveness study. A stepped-wedge trial (SWT) design is applied by which the time points at which implementation sites receive the ItFits-toolkit are randomised. All implementation sites have received the ItFits-toolkit sequentially acting as control and intervention condition at the end of the trial. There are 6 groups and 2 implementation sites per group. Sites will be informed about the group they are allocated to three months prior to cross-over. Every three months, a new group crosses over from IAU to using the ItFits-toolkit. There are two types of study participants: 1) implementers, and 2) staff involved in iCBT service delivery. Implementers are directly involved in the development, coordination and execution of implementation activities. Implementers are represented in the study by an Implementation Lead (IL) who has a coordinating role in the implementation activities. Implementers are first engaged with the Implementation-As-Usual activities and become users of the ItFits-toolkit when the implementation site crosses-over to the ItFits-toolkit condition (i.e. when they receive the ItFits-toolkit). When using the ItFits-toolkit, the IL is supported by an implementation core team (1-3 persons), and a number of relevant stakeholders providing feedback and advice on the implementation work through a process of co-creation. The IL engages with the ItFits-toolkit user-interface directly and collects and provides the data on implementation site level. Service delivery staff are the 'receivers' of the implementation activities and are subject to changes in their daily work to implement iCBT. Staff within the site are eligible for partaking in the study when they are involved in iCBT delivery. Involvement refers to a distinct role in delivering iCBT: mental health specialists, referrers, administration, and ICT support. Next to the study participants, each site has a study team responsible for the preparation, coordination, and execution of the local trials. These persons do not provide data themselves, but are involved in localising the current protocol, obtaining ethical approvals, recruitment of study participants, data collection, analysis, and reporting. Implementation success is defined as a multidimensional concept that includes increases in uptake of the iCBT service on organisational level, normalisation process in staff, and efficiency outcomes on organisational level. Uptake is defined as a combination of referral rates and the extent to which the service completed by patients; normalisation process as the extent to which the new service is considered by staff to be a normal part of daily work; and efficiency as costs of implementation effort per uptake and costs per degree of normalisation. To be able to relate the effects of the ItFits-toolkit to these primary outcomes, Implementation Leads' exposure to, perceived satisfaction with, and usability of the ItFits-toolkit will be assessed. A measure for organisational readiness for change will be used to corroborate the normalisation instrument. The instruments used to measure the primary and exploratory outcomes are based on brief and validated questionnaires. In addition, a number of newly developed measures are constructed for measuring uptake (referral rate, completeness of use), implementation costs (effort, other costs, and overheads), and exposure to the ItFits-toolkit (event-based log files). Uptake, degree of normalisation, implementation costs, and organisational readiness for change are measured at baseline and every three months for the total duration of the study. Data on exposure to the ItFits-toolkit will be collected continuously, whereas satisfaction and usability of the ItFits-toolkit will be assessed at the end of the exposure period in which an implementation site has received and used the toolkit. At local implementation sites, questionnaire data will be gathered using web-based surveys in the ItFits-toolkit platform. Questionnaire measures for staff will be in local language. Quantitative data that is collected will be anonymized at local research sites, and checked, pruned, pooled, and packaged on a secure data server at Central Data Management. No data will be collected or used without the explicit informed consent of the staff that are participating in the IMA study. Study participants can withdraw at any point without disadvantage. As part of the informed consent procedure, informed consent for data transfer to other parties within the IMA consortium, and long-term preservation will also be asked. The main aim of the analysis is to test whether the introduction of the ItFits-toolkit will induce positive chances in the uptake and normalisation of iCBT, and in the efficiency of this process (i.e. uptake and normalisation divided by implementation costs), across and within sites. Generalized linear mixed models (GLMM) will be used to determine the extent to which the total variation observed in uptake and normalisation is a result of the introduction and use of the ItFits-toolkit. To analyse efficiency in terms of cost-effectiveness, the costs for implementing iCBT using the ItFits-toolkit will be compared to the costs for Implementation-as-Usual in relation to the achieved outcomes (i.e. uptake and normalisation). In the analyses, all observed data will be included, in an effort to implement the intention-to-treat principle.