Evaluating therapist adherence in motivational interviewing by comparing performance with standardized and real patients.

OBJECTIVE The goal of measuring therapist adherence is to determine whether a therapist can perform a given treatment. Yet, the evaluation of therapist behaviors in most clinical trials is limited. Typically, randomized trials have few therapists and minimize therapist variability through training and supervision. Furthermore, therapist adherence is confounded with uncontrolled differences in patients across therapists. Consequently, the extent to which adherence measures capture differences in actual therapist adherence versus other sources of variance is unclear. METHOD We estimated intra-class correlations (ICCs) for therapist adherence in sessions with real and standardized patients (RPs and SPs), using ratings from a motivational interviewing (MI) dissemination trial (Baer et al., 2009) in which 189 therapists recorded 826 sessions with both patient types. We also examined the correlations of therapist adherence between SP and RP sessions, and the reliability of therapist level adherence scores with generalizability coefficients (GCs). RESULTS ICCs for therapist adherence were generally large (average ICC for SPs = .44; average ICC for RPs = .40), meaning that a given therapist's adherence scores were quite similar across sessions. Both ICCs and GCs were larger for SP sessions compared to RPs on global measures of MI adherence, such as Empathy and MI Spirit. Correlations between therapist adherence with real and standardized patients were moderate to large on 3 of 5 adherence measures. CONCLUSION Differences in therapist-level adherence ratings were substantial, and standardized patients have promise as tools to evaluate therapist behavior.

[1]  R. Gallop,et al.  How many treatment sessions and patients are needed to create a stable score of adherence and competence in the treatment of cocaine dependence? , 2012, Psychotherapy research : journal of the Society for Psychotherapy Research.

[2]  Edward H. Haertel,et al.  4 Reliability Coefficients and Generalizability Theory , 2006 .

[3]  Sandra Johnson,et al.  Applying Generalizability Theory using EduG , 2009 .

[4]  K. Carroll,et al.  Correspondence of motivational interviewing adherence and competence ratings in real and role-played client sessions. , 2013, Psychological assessment.

[5]  The influence of therapist variance on the dependability of therapists' alliance scores: a brief comment on "The dependability of alliance assessments: the alliance-outcome correlation is larger than you think" (Crits-Christoph et al., 2011). , 2012, Journal of consulting and clinical psychology.

[6]  D. Kashy,et al.  Studying psychotherapy using the one-with-many design: The therapeutic alliance as an exemplar. , 2009 .

[7]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[8]  D. A. Kenny,et al.  Dyadic Data Analysis , 2006 .

[9]  Bryan Hartzler,et al.  Agency context and tailored training in technology transfer: a pilot evaluation of motivational interviewing training for community counselors. , 2009, Journal of substance abuse treatment.

[10]  B. Wampold,et al.  Uniformity of evidence-based treatments in practice? Therapist effects in the delivery of cognitive processing therapy for PTSD. , 2013, Journal of counseling psychology.

[11]  Zac E. Imel,et al.  Mutual influence in therapist competence and adherence to motivational enhancement therapy. , 2011, Drug and alcohol dependence.

[12]  Bryan Hartzler,et al.  An evaluation of workshop training in motivational interviewing for addiction and mental health clinicians. , 2004, Drug and alcohol dependence.

[13]  W. Miller,et al.  A SMALL STUDY OF TRAINING IN MOTIVATIONAL INTERVIEWING: DOES ONE WORKSHOP CHANGE CLINICIAN AND CLIENT BEHAVIOR? , 2001 .

[14]  Andrew Gelman,et al.  Data Analysis Using Regression and Multilevel/Hierarchical Models , 2006 .

[15]  James F. Boswell,et al.  Patient characteristics and variability in adherence and competence in cognitive-behavioral therapy for panic disorder. , 2013, Journal of consulting and clinical psychology.

[16]  W. Miller,et al.  A randomized trial of methods to help clinicians learn motivational interviewing. , 2004, Journal of consulting and clinical psychology.

[17]  K. Hood,et al.  When smokers are resistant to change: experimental analysis of the effect of patient resistance on practitioner behaviour. , 2005, Addiction.

[18]  Lawrence G. Smith,et al.  The use of standardised patients to assess clinical competence: does practice make perfect? , 2006, Medical education.

[19]  B. Wampold,et al.  Estimating variability in outcomes attributable to therapists: a naturalistic study of outcomes in managed care. , 2005, Journal of consulting and clinical psychology.

[20]  Jarrod Had.  MCMC Methods for Multi-Response Generalized Linear Mixed Models: The MCMCglmm R Package , 2010 .

[21]  Stacey M. L. Hendrickson,et al.  Assessing competence in the use of motivational interviewing. , 2005, Journal of substance abuse treatment.

[22]  R. Gallop,et al.  The dependability of alliance assessments: the alliance-outcome correlation is larger than you might think. , 2011, Journal of consulting and clinical psychology.

[23]  K. Jackson,et al.  Effect of a significant other on client change talk in motivational interviewing. , 2013, Journal of consulting and clinical psychology.

[24]  R. DeRubeis,et al.  Therapist adherence/competence and treatment outcome: A meta-analytic review. , 2010, Journal of consulting and clinical psychology.