Conducting Meta‐Analyses of Evaluations of Government‐Funded Training Programs

Government-funded training programs in the United States have often been subject to rigorous evaluation. Indeed, many of these programs have been evaluated with random assignment, although sophisticated quasi-experimental methods have also been used. Until very recently, however, there has been little systematic attempt to use the cumulative information vested in these evaluations to attempt determine which kinds of programs work best in which setting and with respect to which types of client. Meta-analysis-a set of statistical procedures for systematically synthesizing findings from separate studies-can, in theory at least, address these and other topics that evaluation of individual programs cannot. This article discusses the steps in conducting such a synthesis, summarizes the results of three recently conducted meta-analyses of training and welfare-to-work programs, identifies limitations to the meta-analytic approach, and considers ways in which some of these limitations can be overcome. Copyright 2005 by The Policy Studies Organization.

[1]  David H. Greenberg,et al.  Do welfare‐to‐work programmes work for long? , 2004 .

[2]  David H. Greenberg,et al.  The Dissemination and Utilization of Welfare‐to‐Work Experiments in State Policymaking , 2000 .

[3]  Mark W. Lipsey,et al.  Practical Meta-Analysis , 2000 .

[4]  David H. Greenberg,et al.  A Meta-Analysis of Government-Sponsored Training Programs , 2003 .

[5]  Charles F. Manski Designing Programs for Heterogeneous Populations: The Value of Covariate Information , 2001 .

[6]  G. Glass Primary, Secondary, and Meta-Analysis of Research1 , 1976 .

[7]  Michael Lynch,et al.  An Offer You Can't Refuse , 2000 .

[8]  Edward Pauly,et al.  From Welfare to Work. , 1993 .

[9]  Howard S. Bloom,et al.  Linking program implementation and effectiveness: Lessons from a pooled sample of welfare‐to‐work experiments , 2003 .

[10]  H. Beecher,et al.  The powerful placebo. , 1955, Journal of the American Medical Association.

[11]  J. Gill Hierarchical Linear Models , 2005 .

[12]  Jan de Leeuw,et al.  Introducing Multilevel Modeling , 1998 .

[13]  David H. Greenberg,et al.  Meta-Evaluation: Discovering What Works Best in Welfare Provision , 2004 .

[14]  David H. Greenberg,et al.  When Welfare-to-Work Programs Seem to Work Well: Explaining Why Riverside and Portland Shine So Brightly , 2005 .

[15]  David H. Greenberg,et al.  What Happens To The Effects Of Government-Funded Training Programs Over Time? , 2004, The Journal of Human Resources.

[16]  Stephen W. Raudenbush,et al.  Random effects models. , 1994 .

[17]  David H. Greenberg,et al.  Evaluating Government Training Programs for the Economically Disadvantaged , 1997 .

[18]  L. Hedges,et al.  The Handbook of Research Synthesis , 1995 .