This multi-method, multi-phase pilot trial is designed to explore initial implementation and effectiveness of a theoretically-driven, culturally-tailored, measurement-based care (MBC) approach, Strategic Treatment Assessment for Youth (STAY), for racial and ethnic minoritized (REM) youth with depressive symptoms or suicidal thoughts and behaviors (STB). STAY offers an innovative, culturally-tailored approach to retain REM youth with depressive symptoms with or without STB in mental health treatment. In this R34 study, the investigators will first refine the STAY clinical protocol and implementation plan, develop STAY instrumentation (STAY, Aim 1), and then pilot test the effectiveness of STAY in three community-based mental health clinics (Aim 2). Aim 2 focuses on testing the protocol through a pilot Hybrid Type 2 trial by comparing clinicians randomly assigned to STAY (intervention condition) as compared to those assigned to the MBC As Usual (active comparison condition). Results from this pilot trial will inform optimal study procedures, measures, and site selection for a subsequent, fully-powered Hybrid Type 2 trial to examine STAY implementation, effectiveness, and scalability.
This multi-method, multi-phase pilot trial is designed to explore initial implementation and effectiveness of a theoretically-driven, culturally-tailored, measurement-based care (MBC) approach, Strategic Treatment Assessment for Youth (STAY), for racial and ethnic minoritized (REM) youth with depressive symptoms or suicidal thoughts and behaviors (STB). STAY offers an innovative, culturally-tailored approach to retain REM youth with depressive symptoms with or without STB in mental health treatment. In this R34 study, the investigators will first refine the STAY clinical protocol and implementation plan, develop STAY instrumentation (STAY, Aim 1), and then pilot test the effectiveness of STAY in three community-based mental health clinics (Aim 2). Aim 2 focuses on testing the protocol through a pilot Hybrid Type 2 trial by comparing clinicians randomly assigned to STAY (intervention condition) as compared to those assigned to the MBC As Usual (active comparison condition). Results from this pilot trial will inform optimal study procedures, measures, and site selection for a subsequent, fully-powered Hybrid Type 2 trial to examine STAY implementation, effectiveness, and scalability. In Aim 1 of the research plan, the investigators will refine the STAY protocol and intervention plan based on the study team's preliminary development of the STAY model, protocol, and implementation plan. The STAY protocol includes a clinician manual (including clinician guide for each STAY component, step-by-step instructions, sample scripts, clinical vignettes) and assessment measures. The STAY implementation plan includes the STAY training, goals and objectives, responsibilities, and timelines. The investigators will employ user-centered design (UCD) methods guided by a Discover, Design, Build, and Test (DDBT) Development process to revise the STAY protocol and implementation plan to ensure STAY is aligned with the needs of deployment contexts and end users. A sample size of N=12 users (i.e., clinicians and clinical administrators) divided into three cohorts of four participants each will be used as per recommendations for user-centered design (UCD) testing for complex interventions. Aligned with best practices, sampling will balance both novice and more experienced users. Participants will interact with the STAY protocol and implementation plan prototypes and engage in usability tests, including cognitive walk-throughs and lab-based user testing. Participants will also compete the 10-item Intervention Usability Scale. Iterative data analysis and prototyping between each cohort will be used to rapidly refine the STAY protocol and implementation plan to optimize usability. Then, the STAY Observational Coding System (SOCS) will be developed to assess its feasibility and scalability for future trials, as well as a 10-item STAY Knowledge, Skills and Attitudes (KSAT) Measure to tap core knowledge domains and implementation mechanisms to be assessed in the Aim 2 trial. At the end of Aim 1, the study team will have refined a STAY protocol and implementation plan. The investigators will also have an observational coding system to assess STAY fidelity and a brief, pragmatic measure to assess knowledge, attitudes and practices of clinicians trained in STAY. In Aim 2 of the research plan, the investigators will conduct a pilot effectiveness-implementation Hybrid Type 2 trial of STAY vs MBC As Usual. Using a randomized design, 20 clinicians at three public mental health sites will be stratified by site and randomly assigned to STAY (intervention condition) or to MBC As Usual (active control condition). Clinicians randomized to STAY will receive the STAY protocol, and MBC As Usual clinicians will receive standard MBC educational materials used in the study team's other MBC studies. Clinicians in both conditions will attend a one-day training and monthly, 1-hour, post-training consultation calls for 6 months. Clinicians in both conditions will be asked to implement MBC with REM youth and their families who meet eligibility criteria and facilitate study referrals. All clinicians will complete measures at baseline (pre-training) and post-training and post-implementation (6-month post-training follow-up). Half of the clinicians in each condition will be randomly selected to participate in the SOCS. Youth/caregiver dyads will be recruited on a rolling basis due to enrollment patterns at the clinical sites, which is why the investigators are planning 20 months of data collection to achieve 4-month follow-up measures from all youth. Youth and caregivers will complete measures at baseline (within 30 days of their first contact with the site), 2-months (8 weeks) and 4-months (16 weeks) following the baseline measure. Youth measures capture implementation outcomes (e.g., clinicians' cultural humility and cultural comfort), engagement mechanisms (e.g., treatment relevance and acceptability, therapeutic alliance), service outcomes (e.g., attitudinal engagement), and youth symptom outcomes (e.g., depression symptoms, suicidal ideation and behavior). Caregiver measures capture implementation outcomes (e.g., clinicians' cultural comfort), engagement mechanisms (e.g., treatment relevance and acceptability, therapeutic alliance), and service outcomes (e.g., attitudinal engagement). Youth and caregivers will also be asked to consent to secondary data collection by the study team to obtain medical records information for their attendance, discharge reason and fidelity of MBC. Clinician measures capture implementation outcomes (e.g., feasibility, acceptability, appropriateness). The investigators will assess STAY implementation outcomes of fidelity, feasibility, acceptability, and appropriateness (primary outcomes). Further, the investigators will explore whether STAY, as compared to MBC As Usual, will result in (secondary outcomes): (1) Greater therapeutic alliance and treatment relevance and acceptability (engagement mechanisms); (2) Greater session attendance, attitudinal engagement and treatment completion (service outcomes); and (3) Greater reduction in youth depressive symptoms, suicidal ideation and behavior (treatment outcomes). Appropriateness of Methods Follow up intervals are informed by typical treatment length and retention measures in prior research. Sample size was determined based on the resources and time frame constraints of 3 years; participating clinicians and families will be recruited in year 1, quarter 4 to finish all data collection in year 2. Also, individual clinician patterns in MBC administration and feedback can feasibly be examined with this small sample size, as has been useful in other implementation pilot studies. Knowledge gained from this pilot will be used to refine the research strategy for a fully powered, subsequent R01 with optimal procedures. Piloting STAY with procedures that would be exactly mirrored in a future, multi-site R01 provides numerous learning opportunities about the feasibility and parameters of measures, study recruitment and data collection procedures (e.g., collecting youth- and caregiver-reported data over the phone), clinician engagement in consultation calls, record review procedures to track fidelity, missing data patterns, and range and pattern of session attendance for both the STAY and MBC As Usual groups. Also, site variation in this trial is expected to provide information about how different sites handle STAY implementation, which the investigators can further investigate as site-level variation in a fully-powered R01 trial. Outcomes of this trial could inform potential additions to the R01 design, such as 1) enhancements to STAY to more comprehensively address logistical barriers, 2) testing STAY among REM adolescents with conditions other than depression, and/or 3) a fully-powered Type 2 Hybrid trial to detect clinician effects, moderators, and effectiveness outcomes.
Study Type
INTERVENTIONAL
Allocation
RANDOMIZED
Purpose
TREATMENT
Masking
SINGLE
Enrollment
140
Culturally tailored measurement based care
Measurement based care as usual
Yale University
New Haven, Connecticut, United States
Patient Health Questionnaire-9 Item Version
The Patient Health Questionnaire-9 (PHQ-9) is a 9-item measure for use with individuals over 12 years of age. Items are scored on a 3-point Likert scale (0="Not at all"; 3= "Nearly every day") with greater scores indicative of higher depression. The PHQ-9 has been well established as a brief depression severity measure and used in numerous studies examining the effectiveness of MBC for depression. Internal consistency of the PHQ-9 is high (α\>0.80) and criterion validity was established by conducting 580 structured interviews which showed that individuals who scored high (≥ 10) on the PHQ-9 were 7 to 13.6 times more likely to be diagnosed with depression. Moreover, among a sample of 442 adolescents enrolled in mental health treatment, a PHQ-9 score of ≥11 had a sensitivity of 89.5% and specificity of 77.5% for detecting youth with major depression.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Columbia-Suicide Severity Rating Scale (C-SSRS)
The Columbia-Suicide Severity Rating Scale (C-SSRS) is a semi-structured interview that measures suicide ideation and behavior. It is predictive of lifetime history of non-suicidal self-injury among adolescents and suicide attempts at return visit to an emergency department. The C-SSRS is the leading national suicide assessment measure and has been used in numerous National Institute of Mental Health (NIMH) clinical trials. In three multisite studies with 673 adolescents and adults, internal consistency on the intensity subscale ranged from moderate (α=.73) to high (α=.95) and the C-SSRS demonstrated good convergent and divergent validity with other prominent suicidal ideation/behavior scales (100% specificity and 100% sensitivity in correctly identifying lifetime suicide attempts).
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Session Attendance (Behavioral Treatment Engagement)
Behavioral engagement will be measured by participation in sessions, which is defined as the number of sessions attended divided by number of sessions scheduled over a 4-month period.
Time frame: 4-Month Follow-Up
Patient Activation Measure (Attitudinal Treatment Engagement)
Attitudinal engagement will be measured using the reduced, 13-item version of the Patient Activation Measure (PAM). This measure has strong psychometric properties particularly in relation to the other limited measures of this construct; the theoretical range of items and mean scores is 0-100 but means among a sample of 1,515 individuals with a high school education or less range from 38-51, where higher scores reflect higher levels of activation.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Treatment Retention (Treatment Completion/Continuation)
Treatment retention will be assessed via records review for successful completion or continuation vs. whether they terminated inappropriately (e.g., early) at four months after treatment started. If the patient has terminated services by the 4-month mark, the investigators will code the discharge reason as "successful completion" (i.e., retained) or "premature termination" (i.e., not retained). If the patient is still in services at the 4-month mark, they will be considered "retained" unless they have missed all three of the most recently scheduled consecutive sessions. This is an established treatment retention measure.
Time frame: 4-Month Follow-Up
Treatment Evaluation Inventory - Short Form
Treatment relevance and acceptability will be measured by the caregiver- and youth-reported Treatment Evaluation Inventory (TEI). The 15 items are rated on a 7-point Likert scale (1="strongly disagree"; 7="strongly agree") that load on a unitary factor. Item scores are summed to provide a general treatment acceptability index, with higher values indicating higher rated acceptability. The Treatment Evaluation Inventory-Short Form (TEI-SF) has shown variable internal consistency (α\>0.35 to 0.80) and is considered to be the first and most frequently used instrument of treatment acceptability. The study team will adapt wording in items 1 and 2 to refer to depression instead of problem behavior.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Working Alliance Inventory - Short Revised
Therapeutic alliance will be measured by youth and caregiver report on the Working Alliance Inventory - Short Revised (WAI-SR). This 12-item measure was developed using confirmatory factor analysis and item response theory on the original WAI and includes Goal, Task and Bond constructs. Items are rated on a 5-point Likert scale (1="Seldom"; 5="Always"), with higher scores indicating stronger alliance. Reliability is strong (α\>0.80), convergent validity is adequate r \>0.64), as is model fit.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision (DSM-5-TR) Self-Rated Level 1 Cross-Cutting Symptom Measure Child Age 11-17
This measure consists of 25 questions that assess 12 mental health domains that are important across psychiatric diagnoses. These domains include depression, anger, irritability, mania, anxiety, somatic symptoms, inattention, suicidal ideation/attempt, psychosis, sleep disturbance, repetitive thoughts and behaviors, and substance use. Each item asks the child, age 11-17, to rate how much or how often they have been bothered by each symptom during the past 2 weeks. 19 items are rated on a 5-point scale (0=none or not at all; 1=slight or rare, less than a day or two; 2=mild or several days; 3=moderate or more than half the days; and 4=severe or nearly every day). The suicidal ideation, suicide attempt, and substance abuse items are rated on a "Yes" or "No" scale. This measure was found to have good test-retest reliability in the DSM-5 Field Trials conducted in pediatric clinical samples across the United States.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition, Text Revision (DSM-5-TR) Parent/Guardian-Rated Level 1 Cross-Cutting Symptom Measure-Child Age 6-17
This measure consists of 25 questions that assess 12 mental health domains that are important across psychiatric diagnoses. These domains include depression, anger, irritability, mania, anxiety, somatic symptoms, inattention, suicidal ideation/attempt, psychosis, sleep disturbance, repetitive thoughts and behaviors, and substance use. Each item asks the parent/guardian to rate how much or how often their child has been bothered by each symptom during the past 2 weeks. 19 items are rated on a 5-point scale (0=none or not at all; 1=slight or rare, less than a day or two; 2=mild or several days; 3=moderate or more than half the days; and 4=severe or nearly every day). The suicidal ideation, suicide attempt, and substance abuse items are rated on a "Yes", "No" or "Don't know" scale. This measure was found to have good test-retest reliability in the DSM-5 Field Trials conducted in pediatric clinical samples across the United States.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Revised Child Anxiety and Depression Scale (RCADS-25)
This 25-item measure is a revised shortened version of the original Revised Child Anxiety and Depression Scale (RCADS). Each item is rated on a 4-point scale (0=never; 3=Always). There are two subscales: an Anxiety Total scale and a Depression Total scale, with higher sum scores representing greater severity of general anxiety and depressive symptoms. Both subscales have shown acceptable reliability within clinic-referred and school-based samples.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Strategic Treatment Assessment for Youth (STAY) Observational Coding System
The STAY Observational Coding System (SOCS) will be developed by the investigators to include observational items with anchored ratings. A coder's manual will be developed for each STAY component and reviewed by collaborators and consultants. To optimize likelihood of the SOCS being used in whole or in part by others, the investigators will include recommendations for modified versions (e.g., cultural competence delivering other interventions, standard measurement-based care competence, and use by clinical leaders for adherence monitoring). Clinicians will be asked to audiotape three sessions with two clients and securely transfer them to the study team for coding using the observational coding system.
Time frame: 6 Months Post-Training
Strategic Treatment Assessment for Youth (STAY) Feasibility
Feasibility will be measured using clinician self-report on the Feasibility of Intervention Measure (FIM), collected using a web-based, clinician-report survey. The FIM is a 4-item measure (1= 'Completely Disagree"; 5 = "Completely Agree"), with higher values reflecting higher fidelity, developed based on input from 36 implementation scientists and 27 mental health professionals. Structural validity, test-retest reliability, and sensitivity to change have been established.
Time frame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
Strategic Treatment Assessment for Youth (STAY) Appropriateness
Appropriateness will be measured by clinician self-report on the Intervention Appropriateness Measure (IAM). This 4-item measure (1= 'Completely Disagree"; 5 = "Completely Agree"), with higher values reflecting higher fidelity, developed based on input from 36 implementation scientists and 27 mental health professionals. Structural validity, test-retest reliability, and sensitivity to change have been established.
Time frame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
Strategic Treatment Assessment for Youth (STAY) Acceptability
Acceptability will be measured by clinician self-report on the Acceptability of Intervention Measure (AIM) via web-based, clinician-report survey. This 4-item measure (1= 'Completely Disagree"; 5 = "Completely Agree"), with higher values reflecting higher fidelity, developed based on input from 36 implementation scientists and 27 mental health professionals. Structural validity, test-retest reliability, and sensitivity to change have been established.
This platform is for informational purposes only and does not constitute medical advice. Always consult a qualified healthcare professional.
Time frame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
Strategic Treatment Assessment for Youth (STAY) Knowledge, Skills and Attitudes
Knowledge, skills and attitudes regarding Strategic Treatment Assessment for Youth (STAY) will be evaluated using a 10-item measure developed by the investigators to tap the core knowledge domains and implementation mechanisms potentially influencing STAY implementation. Items will be modeled after the 40-item and more recent 17-item Knowledge of Evidence-Based Services Questionnaire, the 18-item Attitudes toward Standardized Assessment Scale-Monitoring and Feedback and the 4-item Current Assessment Practice Evaluation. The study team will select and/or adapt items most specific to central aspects of the STAY training content and refine and reduce a final set of items based on rapid pilot testing and feedback among the study team, collaborators and contributors. The response scale for the measure will also be developed based on feedback from collaborators and contributors.
Time frame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
Current Assessment Practice Evaluation (CAPE)
Measurement-Based Care practices in general will be assessed by the Current Assessment Practice Evaluation (CAPE). This 4-item measure collects clinician self-reported use of MBC practices across different phases of intervention (e.g., at intake, ongoing during treatment, at termination). Items are reported on a 4-point scale (None, Some \[1-39%\], Half \[40-60%\], Most \[61-100%\]) with higher ratings reflecting more frequent use of MBC practices. Internal consistency is acceptable (alpha = 0.72) and has been replicated in the study team's prior work with a sample of 95 clinicians.
Time frame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
Attitudes toward Standardized Assessment Scale - Monitoring and Feedback
Attitudes toward MBC in general will be assessed via the Attitudes toward Standardized Assessment Scale-Monitoring and Feedback (ASA-MF). This 18-item measure includes three subscales, Clinical Utility, Treatment Planning and Practicality, which have shown acceptable internal consistency (alphas between 0.81 and 0.85) which has been replicated in the study team's prior work. Items are scored on a scale from 1 (Strongly Disagree) to 5 (Strongly Agree), with higher values indicating more positive attitudes towards standardized assessment.
Time frame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
Multicultural Counseling Knowledge and Awareness Scale
Clinician practices in culturally-responsive treatment delivery will be assessed via the clinician self-reported Multicultural Counseling Knowledge and Awareness Scale (MCKAS) and the youth-reported Cultural Humility Scale (CHS). The Multicultural Counseling Knowledge and Awareness Scale (MCKAS) is a 32-item measure (1 = "Not At All True"; 7 = "Totally True") consisting of 20 knowledge items and 12 awareness items, with higher values indicating higher knowledge and awareness. The MCKAS possesses a moderate degree of validity and a moderate to high degree of both internal consistency and test-retest reliability and was found to be the least influenced by social desirability attitudes among respondents on similar measures. The MCKAS was developed by distinguished national experts in multicultural counseling and appears to be the leading measure in this domain.
Time frame: Baseline (pre-training), Immediately Post-Training, 6 Months Post-Training
Cultural Humility Scale (CHS)
Cultural humility, or the degree to which a therapist can create a therapeutic environment that respects cultural differences and an openness to different beliefs, values, and worldviews, will be measured by youth report on the Cultural Humility Scale (CHS). The 12 items are rated on a 5-point Likert scale (1 = "strongly disagree"; 5 = "strongly agree") by the youth about their clinician, with higher values reflecting higher perceptions of cultural humility. Reliability is strong.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Implementation Climate Scale (ICS)
Implementation climate will also be measured for each site in order to understand organizational-level climate to support STAY implementation. The Implementation Climate Scale (ICS) will be used to understand clinicians' perceptions of the policies, practices, procedures and behaviors that are rewarded, supported and expected to facilitate effective STAY implementation. This 38-item measure (0 = "Not at all"; 4 = Very great extent") assesses strategic climate for evidence-based practice implementation, with higher values indicating a more positive and supportive implementation climate. The investigators will adjust measures to assess specifically for STAY. Internal consistency is strong (α = 0.81-0.91) with high factor loadings on 6 factors among a sample of 630 individuals working in 128 teams.
Time frame: Baseline, 6 Months Post-Training
Treatment Outcome Expectations Scale (TOES)
Treatment outcome expectations will be measured by the caregiver and youth-reported Treatment Outcome Expectations Scale (TOES). TOES contains eight items that measure youth and caregiver expectations of possible therapy outcomes. Items are scored on a three-point Likert scale (0="I do not expect this"; 3= "I do expect this"). The mean of each informant's scores is calculated to yield a TOES Total Score, with a higher total score indicating higher expectations. Sound psychometric properties have been established, with internal consistency ranging from (α = 0.83-0.91) for youth reported TOES, and adequate internal consistency (α = 0.86) for caregiver reported TOES. Confirmatory factor analysis revealed that a one-factor model fit data for youth responses on TOES best, although fit indices for a one-factor model were less ideal for the caregiver reported TOES.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Treatment Process Expectations Index (TPEI)
Expectations about youth and caregivers' role in treatment and the treatment process will be measured by the Treatment Process Expectations Index (TPEI). The TPEI contains nine items, which are scored on a three-point Likert scale (0= "I do not expect this"; 3= "I do expect this"). Psychometric findings are not available, as responses on the TPEI are meant to be used on an item level, offering clinical utility and relevance that can supplement data from TOES.
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Therapist Cultural Comfort Scale (TCCS)
The perceived cultural comfort of therapists will be measured by the youth reported Therapist Cultural Comfort Scale (TCCS). The TCCS contains 13 items, which are scored on a five-point Likert scale (1= "Strongly Disagree"; 5= "Strongly Agree"). An exploratory and confirmatory factor analysis supported two subscales, a Negative subscale, which had items that were negative markers of cultural comfort, and Positive subscale, which had items indicative of greater cultural comfort. Sound psychometric properties have been found including reliability, convergent validity, and incremental validity. Internal consistencies have been found adequate (α = 0.77-0.94). Adequate test-retest reliability has also been demonstrated (interclass correlation coefficient= .82).
Time frame: Baseline, 2-Month Follow-Up, 4-Month Follow-Up
Intervention Usability Scale (IUS)
Usability will be evaluated by a clinician self-report on the Intervention Usability Scale (IUS). This 10-item measure (0= "Strongly Disagree"; 4= "Strongly Agree") was adapted from the System Usability Scale (SUS) to assess usability of evidence-based psychosocial interventions; a two-factor structure has been found, with adequate internal consistency (α = 0.83). Higher values indicate greater perceived usability.
Time frame: 6 Months Post-Training