The Causes and Consequences of Test Score Manipulation: Evidence from the New York Regents Examinations

We show that the design and decentralized scoring of New York’s high school exit exams—the Regents Examinations—led to systematic manipulation of test scores just below important proficiency cutoffs. Exploiting a series of reforms that eliminated score manipulation, we find heterogeneous effects of test score manipulation on academic outcomes. While inflating a score increases the probability of a student graduating from high school by about 17 percentage points, the probability of taking advanced coursework declines by roughly 10 percentage points. We argue that these results are consistent with test score manipulation helping less advanced students on the margin of dropping out but hurting more advanced students that are not pushed to gain a solid foundation in the introductory material. (JEL H75, I21, I28)

[1]  D. Ou.  To leave or not to leave? A regression discontinuity analysis of the impact of failing the high school exit exam , 2010 .

[2]  Rebecca Diamond,et al.  The Long-Term Consequences of Teacher Discretion in Grading of High-Stakes Tests , 2016 .

[3]  Petra E. Todd,et al.  On the Specification and Estimation of the Production Function for Cognitive Achievement , 2003 .

[4]  J. Angrist,et al.  In a Small Moment: Class Size and Moral Hazard in the Mezzogiorno , 2014, SSRN Electronic Journal.

[5]  Derek A. Neal,et al.  NBER WORKING PAPER SERIES THE CONSEQUENCES OF USING ONE ASSESSMENT SYSTEM TO PURSUE TWO OBJECTIVES , 2013 .

[6]  Henrik Jacobsen Kleven,et al.  Using Notches to Uncover Optimization Frictions and Structural Elasticities: Theory and Evidence from Pakistan* , 2013 .

[7]  Mikael Lindahl,et al.  Fighting Corruption in Education: What Works and Who Benefits? , 2017, SSRN Electronic Journal.

[8]  Jonah E. Rockoff,et al.  Short Run Impacts of Accountability on School Quality , 2008 .

[9]  Roland G. Fryer.  Teacher Incentives and Student Achievement: Evidence from New York City Public Schools , 2011, Journal of Labor Economics.

[10]  David A. Jaeger,et al.  Degrees matter: new evidence on sheepskin effects in the returns to education. , 1996 .

[11]  Magnus Johannesson,et al.  Are boys discriminated in Swedish high schools , 2011 .

[12]  J. Heckman,et al.  Investing in Our Young People , 2010 .

[13]  Jonah E. Rockoff,et al.  Rules and Discretion in the Evaluation of Students and Schools: The Case of the New York Regents Examinations , 2011 .

[14]  G. Stobart.  Incentives and test-based accountability in education , 2014 .

[15]  Richard J. Murnane,et al.  Extending the regression-discontinuity approach to multiple assignment variables , 2011 .

[16]  J. Cullen,et al.  Tinkering Toward Accolades: School Gaming Under a Performance Accountability System , 2006 .

[17]  Simon Burgess,et al.  Test Scores, Subjective Assessment, and Stereotyping of Ethnic Minorities , 2013, Journal of Labor Economics.

[18]  Victor Lavy,et al.  Do gender stereotypes reduce girls' or boys' human capital outcomes? Evidence from a natural experiment , 2008 .

[19]  Emmanuel Saez,et al.  Do Taxpayers Bunch at Kink Points? , 1999 .

[20]  Raj Chetty,et al.  Adjustment Costs, Firm Responses, and Micro vs. Macro Labor Supply Elasticities: Evidence from Danish Tax Records. , 2011, The quarterly journal of economics.

[21]  R. Murnane,et al.  The Consequences of High School Exit Examinations for Low-Performing Urban Students: Evidence From Massachusetts , 2010 .

[22]  Brian A. Jacob,et al.  The Measurement of Student Ability in Modern Assessment Systems , 2016 .

[23]  Leigh L. Linden,et al.  Discrimination in Grading , 2012 .

[24]  Roland G. Fryer,et al.  The Impact of Attending a School with High-Achieving Peers: Evidence from New York City Exam Schools , 2014 .

[25]  James J. Heckman,et al.  Estimating the Technology of Cognitive and Noncognitive Skill Formation , 2010, Econometrica : journal of the Econometric Society.

[26]  N. Kober,et al.  State High School Exit Exams: A Baseline Report. , 2002 .

[27]  Patrick A. Puhani,et al.  The Long�?Term Effects of Early Track Choice , 2017 .

[28]  Raj Chetty,et al.  Measuring the Impacts of Teachers I: Evaluating Bias in Teacher Value-Added Estimates , 2014 .

[29]  Paco Martorell,et al.  The Signaling Value of a High School Diploma , 2014, Journal of Political Economy.

[30]  Philip Oreopoulos.  Do dropouts drop out too soon? Wealth, health and happiness from compulsory schooling , 2007 .

[31]  Jrn-Steffen Pischke,et al.  Zero Returns to Compulsory Schooling in Germany: Evidence and Interpretation , 2005, The Review of Economics and Statistics.

[32]  G. Weber,et al.  Changes in Compulsory Schooling, Education and the Distribution of Wages in Europe , 2007 .

[33]  Victor Lavy.  Performance Pay and Teachers' Effort, Productivity and Grading Ethics , 2004 .

[34]  Christopher R. Taber,et al.  Selection on Observed and Unobserved Variables: Assessing the Effectiveness of Catholic Schools , 2000, Journal of Political Economy.

[35]  Victor Lavy,et al.  On the Origins of Gender Human Capital Gaps: Short and Long Term Consequences of Teachers’ Stereotypical Biases , 2015 .

[36]  Camille Terrier.  Boys Lag Behind: How Teachers' Gender Biases Affect Student Achievement , 2020, Economics of Education Review.

[37]  J. Angrist,et al.  Does Compulsory School Attendance Affect Schooling and Earnings? , 1990 .

[38]  A. Ebenstein,et al.  The Long-Run Economic Consequences of High-Stakes Examinations: Evidence from Transitory Variation in Pollution , 2016 .

[39]  Serena Canaan,et al.  Returns to Education Quality for Low-Skilled Students: Evidence from a Discontinuity , 2016, Journal of Labor Economics.