Importance of program integrity: outcome evaluation of a gender-responsive, cognitive-behavioral program for female offenders

Research Summary We used a quasi-experimental design to evaluate the effectiveness of Moving On, a gender-responsive, cognitive-behavioral program designed for female offenders. Between 2001 and 2013, there were two distinct periods in which Moving On was administered with, and without, fidelity among female Minnesota prisoners. To determine whether program integrity matters, we examined the performance of Moving On across these two periods. By using multiple comparison groups, we found that Moving On significantly reduced two of the four measures of recidivism when it was implemented with fidelity. The program did not have a significant impact on any of the four recidivism measures, however, when it operated without fidelity. Policy Implication The growth of the "what works" literature and the emphasis on evidence-based practices have helped foster the notion that correctional systems can improve public safety by reducing recidivism. Given that Moving On's success hinged on whether it was delivered with integrity, our results show that correctional practitioners can take an effective intervention and make it ineffective. Providing offenders with evidence-based interventions that lack therapeutic integrity not only promotes a false sense of effectiveness, but also it squanders the limited supply of programming resources available to correctional agencies. The findings suggest that ensuring program integrity is critical to the efficient use of successful interventions that deliver on the promise of reduced recidivism. Language: en

[1]  Charles E McCulloch,et al.  Relaxing the rule of ten events per variable in logistic and Cox regression. , 2007, American journal of epidemiology.

[2]  P. Gendreau,et al.  Tertiary prevention: What the meta-analyses of the offender treatment literature tell us about "what works." , 1990 .

[3]  F. T. Cullen,et al.  THE TWELVE PEOPLE WHO SAVED REHABILITATION: HOW THE SCIENCE OF CRIMINOLOGY MADE A DIFFERENCE , 2005 .

[4]  T. Palmer Programmatic and Nonprogrammatic Aspects of Successful Intervention: New Directions for Research , 1995 .

[5]  E. Ann Carson,et al.  Prisoners in 2014 , 2015 .

[6]  David B. Wilson,et al.  A Quantitative Review of Structured, Group-Oriented, Cognitive-Behavioral Programs for Offenders , 2005 .

[7]  Edward J. Latessa,et al.  Examining the Interaction between Level of Risk and Dosage of Treatment , 2013 .

[8]  D. Rubin,et al.  Constructing a Control Group Using Multivariate Matched Sampling Methods That Incorporate the Propensity Score , 1985 .

[9]  A. Stevens,et al.  Twisting Arms Or a Helping Hand?Assessing the Impact of ‘Coerced’ and Comparable ‘Voluntary’ Drug Treatment Options , 2006 .

[10]  Don A. Andrews,et al.  What Works for Female Offenders: A Meta-Analytic Review , 1999 .

[11]  Valerie A. Clark,et al.  The Effects of Prison-Based Educational Programming on Recidivism and Employment , 2014 .

[12]  Mark W. Lipsey,et al.  The Effectiveness of Correctional Rehabilitation: A Review of Systematic Reviews , 2007 .

[13]  Michael S. Trevisan,et al.  Evaluability assessment: A primer , 2002 .

[14]  Valerie A. Clark,et al.  Blessed Be the Social Tie That Binds , 2013 .

[15]  W. Shadish,et al.  Experimental and Quasi-Experimental Designs for Generalized Causal Inference , 2001 .

[16]  Emily J. Salisbury,et al.  Gender-Responsive Lessons Learned and Policy Implications for Women in Prison , 2012 .

[17]  Frank S. Pearson,et al.  The Effects of Behavioral/Cognitive-Behavioral Programs on Recidivism , 2002 .

[18]  D. Mackenzie,et al.  The Effectiveness of Cognitive Behavioral Treatment for Adult Offenders: A Methodological, Quality-Based Review , 2001 .

[19]  Francis T. Cullen,et al.  Can 14,737 women be wrong? A meta‐analysis of the LSI‐R and recidivism for female offenders* , 2009 .

[20]  D. A. Andrews,et al.  Classification for Effective Rehabilitation , 1990 .

[21]  J. Durlak,et al.  Implementation Matters: A Review of Research on the Influence of Implementation on Program Outcomes and the Factors Affecting Implementation , 2008, American journal of community psychology.

[22]  Matthew D. Makarios,et al.  Examining the Predictors of Recidivism Among Men and Women Released From Prison in Ohio , 2010 .

[23]  J. Scroggins,et al.  Reentry and the (Unmet) Needs of Women , 2010 .

[24]  Jason W. Osbourne,et al.  Four Assumptions of Multiple Regression That Researchers Should Always Test. , 2002 .

[25]  Emily J. Salisbury,et al.  Women’s Risk Factors and Their Contributions to Existing Risk/Needs Assessment , 2010 .

[26]  G. Duwe The Development, Validity, and Reliability of the Minnesota Screening Tool Assessing Recidivism Risk (MnSTARR) , 2014 .

[27]  P. Schmidt,et al.  Evaluating Correctional Programs , 1980 .

[28]  David B. Wilson,et al.  Does incarceration-based drug treatment reduce recidivism? A meta-analytic synthesis of the research , 2007 .

[29]  M. Brecht,et al.  PRETREATMENT CHARACTERISTICS AND TREATMENT PERFORMANCE OF LEGALLY COERCED VERSUS VOLUNTARY METHADONE MAINTENANCE ADMISSIONS , 1989 .

[30]  J. S. Wormith,et al.  Offender Coercion in Treatment , 2008 .

[31]  James Bonta,et al.  Rehabilitating criminal justice policy and practice. , 2010 .

[32]  Markus Breitenbach,et al.  Women’s Pathways to Serious and Habitual Crime , 2012 .

[33]  Kristy Holtfreter,et al.  The Transition from Prison to Community Initiative , 2014 .

[34]  G. Duwe Prison-based chemical dependency treatment in Minnesota: An outcome evaluation , 2010 .

[35]  Mark W. Lipsey,et al.  Cognitive-Behavioral Programs for Offenders , 2001 .

[36]  P. Austin Double propensity-score adjustment: A solution to design bias or bias due to incomplete matching , 2016, Statistical methods in medical research.

[37]  M. Hiller,et al.  Legal Pressure, Treatment Readiness, and Engagement in Long-Term Residential Programs , 2000 .

[38]  J. Concato,et al.  A simulation study of the number of events per variable in logistic regression analysis. , 1996, Journal of clinical epidemiology.

[39]  P. Gendreau,et al.  The Forgotten Issue in Effective Correctional Treatment: Program Implementation , 1999 .

[40]  Aleksandra Nesovic,et al.  Psychometric evaluation of the Correctional Program Assessment Inventory (CPAI) , 2003 .

[41]  Christopher T. Lowenkamp,et al.  DOES CORRECTIONAL PROGRAM QUALITY REALLY MATTER? THE IMPACT OF ADHERING TO THE PRINCIPLES OF EFFECTIVE INTERVENTION* , 2006 .

[42]  Donald Rubin,et al.  Estimating Causal Effects from Large Data Sets Using Propensity Scores , 1997, Annals of Internal Medicine.

[43]  Edward J. Latessa,et al.  Making the Next Step: Using Evaluability Assessment to Improve Correctional Programming , 2001 .

[44]  Melissa D. Grady,et al.  Does Volunteering for Sex Offender Treatment Matter? Using Propensity Score Analysis to Understand the Effects of Volunteerism and Treatment on Recidivism , 2013, Sexual abuse : a journal of research and treatment.

[45]  Shelley L. Brown,et al.  Using Dynamic Factors to Predict Recidivism Among Women , 2015 .

[46]  C. Dowden,et al.  Managing correctional treatment for reduced recidivism: A meta‐analytic review of programme integrity , 2005 .

[47]  G. Duwe,et al.  Removing a Nail From the Boot Camp Coffin , 2008 .