When the Torch Is Passed, Does the Flame Still Burn? Testing a “Train the Supervisor” Model for the Child STEPs Treatment Program

Objective: We assessed sustainability of an empirically supported, transdiagnostic youth psychotherapy program when therapist supervision was shifted from external experts to internal clinic staff. Method: One hundred sixty-eight youths, aged 6–15 years, 59.5% male, 85.1% Caucasian, were treated for anxiety, depression, traumatic stress, or conduct problems by clinicians employed in community mental health clinics. In Phase 1 (2.7 years), 1 group of clinicians, the Sustain group, received training in Child STEPs (a modular transdiagnostic treatment + weekly feedback on youth response) and treated clinic-referred youths, guided by weekly supervision from external STEPs experts. In Phase 2 (2.9 years), Sustain clinicians treated additional youths but with supervision by clinic staff who had been trained to supervise STEPs. Also in Phase 2, a new group, External Supervision clinicians, received training and supervision from external STEPs experts and treated referred youths. Phase 2 youths were randomized to Sustain or External Supervision clinicians. Groups were compared on 3 therapist fidelity measures and 14 clinical outcome measures. Results: Sustain clinicians maintained their previous levels of fidelity and youth outcomes after switching from external to internal supervision; and in Phase 2, the Sustain and External Supervision groups also did not differ on fidelity or youth outcomes. Whereas all 34 group comparisons were nonsignificant, trends with the largest effect sizes showed better clinical outcomes for internal than external supervision. Conclusions: Implementation of empirically supported transdiagnostic treatment may be sustained when supervision is transferred from external experts to trained clinic staff, potentially enhancing cost-effectiveness and staying power in clinical practice.

[1]  Aaron R. Lyon,et al.  Behavioral Rehearsal for Analogue Fidelity: Feasibility in a State-Funded Children’s Mental Health Initiative , 2017, Administration and Policy in Mental Health and Mental Health Services Research.

[2]  S. Bearman,et al.  Building an Evidence Base for Effective Supervision Practices: An Analogue Experiment of Supervision to Increase EBT Fidelity , 2017, Administration and Policy in Mental Health and Mental Health Services Research.

[3]  J. Weisz,et al.  Initial Test of a Principle-Guided Approach to Transdiagnostic Psychotherapy With Children and Adolescents , 2017, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[4]  Terje Ogden,et al.  Is There a Scale-up Penalty? Testing Behavioral Change in the Scaling up of Parent Management Training in Norway , 2015, Administration and Policy in Mental Health and Mental Health Services Research.

[5]  Danielle R Adams,et al.  The Perfect Storm: Collision of the Business of Mental Health and the Implementation of Evidence-Based Practices. , 2015, Psychiatric services.

[6]  C. Becker,et al.  Reducing eating disorder risk factors: A pilot effectiveness trial of a train-the-trainer approach to dissemination and implementation. , 2015, The International journal of eating disorders.

[7]  Mark H. C. Lai,et al.  Examining the Rule of Thumb of Not Using Multilevel Modeling: The “Design Effect Smaller Than Two” Rule , 2015 .

[8]  Nathaniel J. Williams,et al.  Assessing and changing organizational social contexts for effective mental health services. , 2015, Annual review of public health.

[9]  John R. Weisz,et al.  Odd Couple? Reenvisioning the Relation Between Science and Practice in the Dissemination-Implementation Era , 2014 .

[10]  Brad J. Nakamura,et al.  Long-term outcomes for the Child STEPs randomized effectiveness trial: a comparison of modular and standard treatment designs with usual care. , 2013, Journal of consulting and clinical psychology.

[11]  J. Weisz,et al.  Performance of evidence-based youth psychotherapies compared with usual clinical care: a multilevel meta-analysis. , 2013, JAMA psychiatry.

[12]  P. Kendall,et al.  An Examination of Behavioral Rehearsal During Consultation as a Predictor of Training Outcomes , 2013, Administration and Policy in Mental Health and Mental Health Services Research.

[13]  Patricia Chamberlain,et al.  Three Collaborative Models for Scaling Up Evidence-Based Practices , 2012, Administration and Policy in Mental Health and Mental Health Services Research.

[14]  Kimberly Hoagwood,et al.  The organizational social context of mental health services and clinician attitudes toward evidence-based practice: a United States national study , 2012, Implementation Science.

[15]  S. Murphy,et al.  A "SMART" design for building individualized treatment sequences. , 2012, Annual review of clinical psychology.

[16]  D. DeGarmo,et al.  Sustaining Fidelity Following the Nationwide PMTO™ Implementation in Norway , 2011, Prevention Science.

[17]  J. Weisz,et al.  Youth Top Problems: using idiographic, consumer-guided assessment to identify treatment needs and to track change during psychotherapy. , 2011, Journal of consulting and clinical psychology.

[18]  J. Weisz,et al.  The Youth Self Report: Applicability and Validity Across Younger and Older Youths , 2011, Journal of clinical child and adolescent psychology : the official journal for the Society of Clinical Child and Adolescent Psychology, American Psychological Association, Division 53.

[19]  A. Kazdin,et al.  Rebooting Psychotherapy Research and Practice to Reduce the Burden of Mental Illness , 2011, Perspectives on psychological science : a journal of the Association for Psychological Science.

[20]  G. Aarons,et al.  Advancing a Conceptual Model of Evidence-Based Practice Implementation in Public Service Sectors , 2010, Administration and Policy in Mental Health and Mental Health Services Research.

[21]  S. Reise,et al.  Evaluation of the Brief Problem Checklist: child and caregiver interviews to measure clinical progress. , 2010, Journal of consulting and clinical psychology.

[22]  Robert E Drake,et al.  The role of staff turnover in the implementation of evidence-based practices in mental health care. , 2008, Psychiatric services.

[23]  K. Kelleher,et al.  Building Bridges to Evidence-based Practice: The MacArthur Foundation Child System and Treatment Enhancement Projects (Child STEPs) , 2008, Administration and Policy in Mental Health and Mental Health Services Research.

[24]  K. Kelleher,et al.  A Survey of the Infrastructure for Children’s Mental Health Services: Implications for the Implementation of Empirically Supported Treatments (ESTs) , 2008, Administration and Policy in Mental Health and Mental Health Services Research.

[25]  Patrick M Flynn,et al.  Moving innovations into treatment: A stage-based approach to program change. , 2007, Journal of substance abuse treatment.

[26]  Thomas M Vogt,et al.  Evaluating the impact of health promotion programs: using the RE-AIM framework to form summary measures for decision making involving complex issues. , 2006, Health education research.

[27]  Bruce F Chorpita,et al.  Identifying and Selecting the Common Elements of Evidence Based Interventions: A Distillation and Matching Model , 2005, Mental health services research.

[28]  M. Nock,et al.  From science to practice: The flexible use of evidence-based treatments in clinical settings. , 2004, Journal of the American Academy of Child and Adolescent Psychiatry.

[29]  Yuk Fai Cheong,et al.  HLM 6: Hierarchical Linear and Nonlinear Modeling , 2000 .

[30]  Katherine H. Tsai,et al.  Child STEPs in California: A Cluster Randomized Effectiveness Trial Comparing Modular Treatment With Community Implemented Treatment for Youth With Anxiety, Depression, Conduct Problems, or Traumatic Stress , 2017, Journal of consulting and clinical psychology.

[31]  A. Mack Testing Standard and Modular Designs for Psychotherapy Treating Depression, Anxiety, and Conduct Problems in Youth: A Randomized Effectiveness Trial , 2013 .

[32]  J. Weisz,et al.  Modular Approach to Therapy for Children with Anxiety, Depression, Trauma, or Conduct Problems (MATCH-ADTC) , 2009 .

[33]  P. Kendall,et al.  Smoothing the trail for dissemination of evidence-based practices for youth: Flexibility within fidelity. , 2007 .

[34]  J. Weisz,et al.  Why are we here at the clinic? Parent-child (dis)agreement on referral problems at outpatient treatment entry. , 2001, Journal of consulting and clinical psychology.