Assessing Rothstein's critique of teacher value‐added models

Value-added models of teacher effectiveness yield consistent estimates of teacher quality under the assumption that students are randomly assigned to classrooms conditional on ability. Rothstein (2010) tested and rejected this underlying sorting assumption, casting doubt on the usefulness of the value-added framework. In this paper, I illustrate that the falsification tests employed by Rothstein perform quite poorly in small samples and I propose an alternative testing strategy that is less sensitive to sample size. I also show that none of the proposed falsification tests works well when the achievement production function is misspecified. Finally, I return to the same North Carolina sample employed by Rothstein and retest the assumption of conditional random assignment. Once I account for the “smallness” of the data and allowteacher inputs to persist at reasonable rates, I fail to reject conditional random assignment. Keywords. Teacher value-added, model testing. JEL classification. I20, C10.

[1]  E. Hanushek,et al.  Teachers, Schools, and Academic Achievement , 1998 .

[2]  J. MacKinnon,et al.  Estimation and inference in econometrics , 1994 .

[3]  R. Creecy,et al.  Computing Person and Firm Effects Using Linked Longitudinal Employer-Employee Data , 2002 .

[4]  A. Amrein-Beardsley.  Methodological Concerns About the Education Value-Added Assessment System , 2008 .

[5]  F. Kramarz,et al.  What Makes a Test Score? The Respective Contributions of Pupils, Schools, and Peers in Achievement in English Primary Education , 2008, SSRN Electronic Journal.

[6]  James J. Heckman,et al.  Estimating the Technology of Cognitive and Noncognitive Skill Formation , 2010, Econometrica : journal of the Econometric Society.

[7]  Lars Lefgren,et al.  WORKING PAPER SERIES THE PERSISTENCE OF TEACHER-INDUCED LEARNING GAINS , 2008 .

[8]  Helen F. Ladd,et al.  Teacher-Student Matching and the Assessment of Teacher Effectiveness , 2006, The Journal of Human Resources.

[9]  J. Wooldridge,et al.  Solutions manual and supplementary material for econometric analysis of cross section and panel data, second edition , 2007 .

[10]  P. Portugal,et al.  A Simple Feasible Procedure to fit Models with High-dimensional Fixed Effects , 2010 .

[11]  Tim R. Sass,et al.  Value-Added Models and the Measurement of Teacher Productivity. CALDER Working Paper No. 54. , 2010 .

[12]  Jesse Rothstein,et al.  Student Sorting and Bias in Value-Added Estimation: Selection on Observables and Unobservables , 2009, Education Finance and Policy.

[13]  Lars Lefgren,et al.  The Persistence of Teacher-Induced Learning , 2010, The Journal of Human Resources.

[14]  C. Jackson.  Student Demographics, Teacher Sorting, and Teacher Quality: Evidence from the End of School Desegregation , 2009, Journal of Labor Economics.

[15]  Jeffrey M. Wooldridge,et al.  Solutions Manual and Supplementary Materials for Econometric Analysis of Cross Section and Panel Data , 2003 .

[16]  James J. Heckman,et al.  Handbook of Econometrics , 1985 .

[17]  Jonah E. Rockoff.  The Impact of Individual Teachers on Student Achievement: Evidence from Panel Data , 2004 .

[18]  Joel L. Horowitz,et al.  Bootstrap Critical Values for Tests Based on Generalized-Method-of-Moments Estimators , 1996 .

[19]  Gordon K. Smyth,et al.  Partitioned algorithms for maximum likelihood and other non-linear estimation , 1996, Stat. Comput..

[20]  Joel L. Horowitz,et al.  Bootstrap Methods For Covariance Structures , 1996 .

[21]  G. Kézdi.  Robust Standard Error Estimation in Fixed-Effects Panel Models , 2003 .

[22]  Richard J. Murnane,et al.  Using Panel Data to Improve Estimates of the Determinants of Educational Achievement. , 1979 .

[23]  Daniel F. McCaffrey,et al.  A Review of Stata Commands for Fixed-Effects Estimation in Normal Linear Models , 2012 .

[24]  Jonah E. Rockoff,et al.  Non-Cognitive Ability, Test Scores, and Teacher Quality: Evidence from 9th Grade Teachers in North Carolina , 2012 .

[25]  Dan Goldhaber,et al.  Is a good elementary teacher always good? Assessing teacher performance estimates across subjects , 2013 .

[26]  Peter Arcidiacono,et al.  Estimating spillovers using panel data, with an application to the classroom , 2012 .

[27]  Tim R. Sass,et al.  Value-Added Models and the Measurement of Teacher Quality , 2006 .

[28]  Cory Koedel,et al.  Does Student Sorting Invalidate Value-Added Models of Teacher Effectiveness? An Extended Analysis of the Rothstein Critique , 2009, Education Finance and Policy.

[29]  Tim R. Sass,et al.  What Makes for a Good Teacher and Who Can Tell? Working Paper 30. , 2009 .

[30]  Brian A. Jacob,et al.  Can Principals Identify Effective Teachers? Evidence on Subjective Performance Evaluation in Education , 2008, Journal of Labor Economics.

[31]  H. White,et al.  Some heteroskedasticity-consistent covariance matrix estimators with improved finite sample properties☆ , 1985 .

[32]  H. White.  A Heteroskedasticity-Consistent Covariance Matrix Estimator and a Direct Test for Heteroskedasticity , 1980 .

[33]  Thomas J. Kane,et al.  The Promise and Pitfalls of Using Imprecise School Accountability Measures , 2002 .

[34]  Daniel F. McCaffrey,et al.  A Review of Stata Routines for Fixed Effects Estimation in Normal Linear Models , 2010 .

[35]  Joseph G. Altonji,et al.  Small Sample Bias in GMM Estimation of Covariance Structures , 1994 .

[36]  Joseph A. Martineau.  Distorting Value Added: The Use of Longitudinal, Vertically Scaled Student Achievement Data for Growth-Based, Value-Added Accountability , 2006 .

[37]  Jesse Rothstein,et al.  Teacher Quality in Educational Production: Tracking, Decay, and Student Achievement , 2008 .

[38]  Josh Kinsler.  Beyond Levels and Growth: Estimating Teacher Value-Added and its Persistence , 2012 .

[39]  D. Koretz.  Limitations in the Use of Achievement Tests as Measures of Educators' Productivity , 2002 .

[40]  J. Betts,et al.  Value Added to What? How a Ceiling in the Testing Instrument Influences Value-Added Estimation , 2009, Education Finance and Policy.

[41]  M. Eichenbaum,et al.  Small-Sample Properties of GMM-Based Wald Tests , 1996 .