Building Outcomes With Observation-Based Supervision: An FFT Effectiveness Trial

Overview

The proposed effectiveness study examines differences in treatment outcomes of an observation-based supervision (BOOST) versus supervision as usual (SAU). The study will be implemented within 16 teams delivering FFT services at 11 sites in the California Institute of Mental Health (CIMH) system. The 16 FFT therapist teams will be randomly assigned either to BOOST or SAU. Each team will have 3 therapists who will treat 6 families for a total of 18 families per team. Thus, each condition will include 24 therapists who will treat 144 families. The project will be implemented in four staggered waves to establish a more even rate of data collection and treatment implementation to enhance the feasibility of the study by keeping staffing and project costs more constant across the 5-year project. Each wave will involve 4 FFT teams, 2 receiving BOOST and 2 receiving SAU. Teams will be randomized to supervision conditions. Outcome assessments of parents and adolescents will be conducted at baseline and at 4 months and 16 months after treatment initiation.

The proposed study is designed to evaluate two distinct approaches to therapist supervision in the implementation of Functional Family Therapy (FFT), an EBT for adolescent substance abuse and delinquent/behavior problems. The impetus for the proposed research emerged from a large scale dissemination study conducted by the Washington State Institute for Public Policy. The study identified two key findings: (1) Only half the therapists were actually implementing the FFT model in a competent manner, and (2) positive outcomes, i.e. reductions in felony recidivism, were evident only for therapists implementing the model competently. For the subset of competent FFT therapists, the financial benefits of the dissemination were estimated at $7.50 for each dollar of program cost. Based on the accumulated evidence supporting FFT, interest in disseminating the FFT treatment model has skyrocketed. Currently, the dissemination organization, FFT LLC, has trained more than 270 local, state, national and international organizations. Over 1100 FFT therapists have served over 12,000 families around the globe, with large scale multi-site implementations ongoing in California, Washington, Pennsylvania, New York, Florida, and the Netherlands. To address the key issue of enhancing treatment competence, FFT LLC recently developed and implemented a sophisticated web-based application designed to monitor highly structured FFT therapist progress notes, as well as supervisor and client ratings of therapist competence. The process helps to maximize sustainability for community programs by limiting costs. Hence, the supervision process involves feedback to therapists based on the progress notes and therapist-supervisor discussions of therapist performance. The impact of this supervision process on therapist competence and treatment outcomes is unknown. By contrast, Miller and his colleagues found that supervision involving active feedback and/or coaching based on supervisor review of therapy sessions to observe therapist behaviors directly results in improved model fidelity. Such observation-based supervision has been the hallmark in the development of family therapy models and in fidelity monitoring processes in efficacy trials evaluating family therapy and addictions treatments. By eliminating this practice, it is possible that FFT dissemination efforts may unwittingly be omitting one of the most important components needed for effective transfer of treatment. During this early period of research on dissemination, it is vital to identify the specific procedures most likely to maximize effective translation of efficacious treatments into community settings. Observation-based supervision of family therapy has played a critical role both in the development of clinical practices and in efficacy trials which have established positive effects on youth outcomes. Hence, a key first step in effectiveness research is to examine the extent to which supervision practices involving therapist report and observation of therapist behaviors during sessions influence therapist competence and subsequent treatment outcomes. The costs associated with supervision based on a direct review of therapy session recordings are often viewed as prohibitive for community implementation of EBTs. However, the potential sustainability of community implementations must be considered in the context of empirical evidence for the procedures in relation to outcomes achieved. If more intensive supervision practices were to enhance competence and produce superior outcomes, then economic analyses evaluating supervision cost, cost effectiveness, and cost-benefit would be essential. Such analyses would ultimately drive policy considerations with respect to funding EBTs at the community level to maximize the long-term financial and societal benefits. Study Aims The purpose of the proposed study is to examine the effects of observation-based supervision (BOOST) versus the standard supervision as usual (SAU) approach currently used by FFT LLC on therapist competence and adolescent outcomes in a community-based sample of adolescents receiving FFT. The study will be implemented with 16 FFT teams affiliated with the California Institute of Mental Health (CIMH), the coordinating center for stakeholders implementing FFT in community agencies in California. Of these 16 teams, 14 teams will be from 10 community-based agencies affiliated with CIMH and 2 teams will be FFT teams within Los Angeles County Probation. Participants will include three therapists from each of the 16 FFT teams (n=48) and 6 adolescents/families per therapist (n=288). All of the adolescents referred to participate in the study by the FFT teams within Los Angeles County Probation will be youth who are "home on probation." In addition, some of the youth referred to the study by community-based organizations will include "home on probation" youths (e.g., Star View Behavioral Health) and some will not (e.g., Long Beach Adolescent Program). Similarly, youths referred by some community-based organizations to participate in the study may have child welfare involvement. Therapist teams will be randomly assigned to either BOOST or SAU. The aims of the study are as follows: Specific Aim 1: To examine differences between BOOST and SAU in improving treatment outcomes. Specifically, we hypothesize that BOOST will be associated with significantly greater reductions in adolescent substance use (Aim 1a) and conduct disturbance/delinquent behaviors (Aim 1b) than SAU. Specific Aim 2: To examine differences between BOOST and SAU in improving engagement and retention in treatment. Specifically, we hypothesize that BOOST will be associated with significantly higher rates of engagement and retention in treatment. Family therapy interventions are based largely on the premise that dysfunctional family processes play a central role in the development of adolescent problem behaviors such as substance use and delinquency. Family therapy is aimed at mitigating the dysfunctional processes. The success of family therapy in improving family functioning and youth outcomes, as noted above, depends on the fidelity of treatment implementation in general and on therapist competent adherence to the treatment model in particular. Thus: Specific Aim 3: To examine differences between BOOST and SAU in improving therapist competence and family functioning. Specifically, we hypothesize that BOOST will be associated with significantly greater rates of therapist competence (Aim 3a) and improvements in family functioning (Aim 3b) than SAU. Exploratory Aim 4: To explore potential mediating effects of therapist competence and family functioning on treatment outcomes. Specifically, we will explore whether therapist competence mediates (Aim 4a) the association between supervision condition and youth outcomes (substance use, conduct/delinquent behaviors), improvements in family functioning, and greater rates of engagement and retention in treatment. We also will explore whether improvements in family functioning (Aim 4b) mediate the link between supervision condition and youth outcomes (substance use, conduct/delinquent behaviors).