NBER WORKING PAPER SERIES EVALUATING THE DIFFERENTIAL EFFECTS OF ALTERNATIVE WELFARE-TO-WORK TRAINING COMPONENTS: A RE-ANALYSIS OF THE CALIFORNIA GAIN PROGRAM

We show how data from an evaluation in which subjects are randomly assigned to some treatment versus a control group can be combined with nonexperimental methods to estimate the differential effects of alternative treatments. We propose tests for the validity of these methods. We use these methods and tests to analyze the differential effects of labor force attachment (LFA) versus human capital development (HCD) training components with data from California’s Greater Avenues to Independence (GAIN) program. While LFA is more effective than HCD training in the short term, we find that HCD is relatively more effective in the longer term.

[1]  James J. Heckman,et al.  Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs: the Case of Manpower Training , 1989 .

[2]  D. Rubin.  Matched Sampling for Causal Effects: Matching to Remove Bias in Observational Studies , 1973 .

[3]  J. Riccio.  GAIN: Benefits, Costs, and Three-Year Impacts of a Welfare-to-Work Program. California's Greater Avenues for Independence Program. , 1994 .

[4]  J. Riccio.  GAIN: Early Implementation Experiences and Lessons. California's Greater Avenues for Independence Program. , 1989 .

[5]  James J. Heckman,et al.  Alternative methods for evaluating the impact of interventions: An overview , 1985 .

[6]  M. Lechner.  Identification and Estimation of Causal Effects of Multiple Treatments Under the Conditional Independence Assumption , 1999, SSRN Electronic Journal.

[7]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator , 1998 .

[8]  Kenneth A. Couch,et al.  New Evidence on the Long-Term Effects of Employment Training Programs , 1992, Journal of Labor Economics.

[9]  Oscar A. Mitnik.  How Do Training Programs Assign Participants to Training? Characterizing the Assignment Rules of Government Agencies for Welfare-to-Work Programs in California , 2009, SSRN Electronic Journal.

[10]  P. Rosenbaum.  The Role of a Second Control Group in an Observational Study , 1987 .

[11]  Jeffrey A. Smith,et al.  Does Matching Overcome Lalonde's Critique of Nonexperimental Estimators? , 2000 .

[12]  Petra E. Todd,et al.  Matching As An Econometric Evaluation Estimator: Evidence from Evaluating a Job Training Programme , 1997 .

[13]  R. Lalonde.  Evaluating the Econometric Evaluations of Training Programs with Experimental Data , 1984 .

[14]  Stephen Freedman,et al.  The Los Angeles Jobs-First GAIN Evaluation First-Year Findings on Participation Patterns and Impacts , 1999 .

[15]  Philip K. Robins,et al.  Evaluating Program Evaluations: New Evidence on Commonly Used Nonexperimental Methods , 1995 .

[16]  G. Hamilton,et al.  The Role of Education and Training in Welfare Reform. Welfare Reform and Beyond. Policy Brief. , 2002 .

[17]  Judith M. Gueron,et al.  The Role of Education and Training in Welfare Reform , 2002 .

[18]  J. Riccio,et al.  GAIN: Program Strategies, Participation Patterns, and First-Year Impacts in Six Counties. California's Greater Avenues for Independence Program. , 1992 .

[19]  Robert J. LaLonde,et al.  The Promise of Public Sector-Sponsored Training Programs , 1995 .

[20]  H. Hoynes.  Local Labor Markets and Welfare Spells: Do Demand Conditions Matter? , 1996, Review of Economics and Statistics.

[21]  D. Rubin,et al.  Using Multivariate Matched Sampling and Regression Adjustment to Control Bias in Observational Studies , 1978 .

[22]  J. Angrist,et al.  Choosing Among Alternative Nonexperimental Methods for Estimating the Impact of Social Programs : The Case of Manpower Training , 2007 .

[23]  Robert J. LaLonde,et al.  THE EFFECT OF SAMPLE SELECTION AND INITIAL CONDITIONS IN DURATION MODELS: EVIDENCE FROM EXPERIMENTAL DATA ON TRAINING , 1996 .

[24]  David H. Greenberg,et al.  Evaluating Government Training Programs for the Economically Disadvantaged , 1997 .

[25]  M. Lechner.  Program Heterogeneity and Propensity Score Matching: An Application to the Evaluation of Active Labor Market Policies , 2002, Review of Economics and Statistics.

[26]  V. J. Hotz,et al.  Predicting the efficacy of future training programs using past experiences at other locations , 2005 .

[27]  L. Gennetian,et al.  The Los Angeles Jobs-First GAIN Evaluation: Final Report on a Work First Program in a Major Urban Center , 2000 .

[28]  Lance Lochner,et al.  General Equilibrium Treatment Effects: A Study of Tuition Policy , 1998 .

[29]  J. Heckman,et al.  The Economics and Econometrics of Active Labor Market Programs , 1999 .

[30]  Lynn A. Karoly,et al.  Welfare Reform: Effects of a Decade of Change , 2005 .

[31]  H. Bloom.  Learning more from social experiments: evolving analytic approaches , 2006 .

[32]  G. Imbens,et al.  Efficient Estimation of Average Treatment Effects Using the Estimated Propensity Score , 2000 .

[33]  Wen-Ling Lin,et al.  The GAIN Evaluation: Five-Year Impacts on Employment, Earnings, and AFDC Receipt , 1996 .

[34]  D. Rubin.  ASSIGNMENT TO TREATMENT GROUP ON THE BASIS OF A COVARIATE , 1976 .

[35]  D. Rubin,et al.  Assessing Sensitivity to an Unobserved Binary Covariate in an Observational Study with Binary Outcome , 1983 .

[36]  G. Imbens,et al.  Large Sample Properties of Matching Estimators for Average Treatment Effects , 2004 .

[37]  J. Heckman,et al.  Longitudinal Analysis of Labor Market Data: Alternative methods for evaluating the impact of interventions , 1985 .

[38]  G. Imbens.  The Role of the Propensity Score in Estimating Dose-Response Functions , 1999 .

[39]  R. Moffitt,et al.  Five Years-After: The Long-Term Effects of Welfare-to-Work Programs. , 1998 .

[40]  Rajeev Dehejia,et al.  Was There a Riverside Miracle? A Hierarchical Framework for Evaluating Programs With Grouped Data , 2003 .

[41]  G. Burtless,et al.  Five Years After: The Long-Term Effects of Welfare-to-Work Programs , 1996 .

[42]  James J. Heckman,et al.  Characterizing Selection Bias Using Experimental Data , 1998 .