Meta-Evaluation: Discovering What Works Best in Welfare Provision

Numerous programmes designed to encourage welfare recipients to move into work and off benefit have been evaluated in the United States. Many have randomly assigned potential participants into ‘experimental’ and ‘control’ groups to generate unbiased estimates of the effectiveness of the programmes. The results of the evaluations have been selectively influential in shaping policy developments on both sides of the Atlantic, but a thorough understanding of the diversity of experience has been lacking. Applying meta-analysis techniques to a specially constructed database of evaluations in over 50 US sites, this article reports on the first programme-level, systematic meta-evaluation of welfare-to-work programmes. The results confirm the superiority of approaches that prioritize immediate work over human-capital investment but reveal that caseload characteristics and local environment can be equally important as or even more important than programme design. The article concludes with a discussion of the potential and limitations of meta-evaluation.

[1]  Ray Pawson,et al.  Evidence-based Policy: In Search of a Method , 2002 .

[2]  David H. Greenberg,et al.  When Welfare-to-Work Programs Seem to Work Well: Explaining Why Riverside and Portland Shine So Brightly , 2005 .

[3]  William R. Shadish,et al.  Combining estimates of effect size. , 1994 .

[4]  R. Rohh.  ALTERNATIVE METHODS FOR EVALUATING THE IMPACT OF INTERVENTIONS An Overview , 2001 .

[5]  J. Peck,et al.  Exporting workfare/importing welfare-to-work: exploring the politics of Third Way policy transfer , 2001 .

[6]  G. Barrie Wetherill,et al.  Random Effects Models , 1981 .

[7]  Howard S. Bloom,et al.  Linking program implementation and effectiveness: Lessons from a pooled sample of welfare‐to‐work experiments , 2003 .

[8]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[9]  David H. Greenberg,et al.  A Meta-Analysis of Government-Sponsored Training Programs , 2003 .

[10]  David H. Greenberg,et al.  Digest of Social Experiments , 2004 .

[11]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .

[12]  R. Hanka.  The Handbook of Research Synthesis , 1994 .

[13]  Ray Pawson,et al.  Evidence-based Policy: The Promise of `Realist Synthesis' , 2002 .

[14]  J. Gill.  Hierarchical Linear Models , 2005 .

[15]  David H. Greenberg,et al.  Do welfare‐to‐work programmes work for long? , 2004 .

[16]  M. Wickham‐Jones,et al.  From Clinton to Blair: The Democratic (Party) Origins of Welfare to Work , 1999 .

[17]  M. Patton.  Qualitative research and evaluation methods , 1980 .

[18]  Robert Walker,et al.  Conducting Meta‐Analyses of Evaluations of Government‐Funded Training Programs , 2005 .

[19]  Charles F. Manski,et al.  Designing Programs for Heterogeneous Populations: The Value of Covariate Information , 2001 .