Do National and State Assessments Converge for Educational Accountability? A Meta-Analytic Synthesis of Multiple Measures in Maine and Kentucky

Given the policy imperative of using multiple measures for state education accountability under the No Child Left Behind Act (NCLB), this study examines similarities and discrepancies between the National Assessment of Educational Progress (NAEP) and the states' own math assessment results in Kentucky and Maine, with a focus on 3 major academic performance indicators: proficiency level, achievement gap, and achievement gain. Using meta-analytic techniques, the study synthesizes multiple measures from the two states over the periods of 1992–1996 and 2000–2003. It pinpoints the areas and degrees of the discrepancies and explores contributing factors. It also reports emerging convergence of the NAEP and state assessments under the NCLB.

[1]  Jaekyung Lee,et al.  Tracking Achievement Gaps and Assessing the Impact of NCLB on the Gaps , 2006 .

[2]  R. Linn.  Assessments and Accountability , 2000 .

[3]  John A. Dossey.  Can Students Do Mathematical Problem Solving? Results from Constructed-Response Questions in NAEP's 1992 Mathematics Assessment. , 1993 .

[4]  Damian W. Betebenner,et al.  Accountability Systems: Implications of Requirements of the No Child Left Behind Act of 2001 , 2002 .

[5]  James W. Pellegrino,et al.  Grading the Nation's Report Card: Evaluating NAEP and Transforming the Assessment of Educational Progress. , 1999 .

[6]  Clyde M. Reese.  NAEP 1996 Mathematics Report Card for the Nation and the States. Findings from the National Assessment of Educational Progress. , 1997 .

[7]  R. Linn.  The Design and Evaluation of Educational Assessment and Accountability Systems. CSE Technical Report. , 2001 .

[8]  Monty Neill.  Implementing Performance Assessments: A Guide to Classroom, School and System Reform. , 1995 .

[9]  G. Cizek.  Reactions to National Academy of Education Report, "Setting Performance Standards for Student Achievement.". , 1993 .

[10]  Samuel A. Livingston,et al.  Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. , 1982 .

[11]  B. Fuller,et al.  Is the No Child Left Behind Act Working? The Reliability of How States Track Achievement. Working Paper 06-1. , 2006 .

[12]  Jaekyung Lee,et al.  Using National and State Assessments To Evaluate the Performance of State Education Systems: Learning from the Cases of Kentucky and Maine. Research Report. Statewide Systemic Initiatives (SSI) Study. , 2002 .

[13]  Larry V. Hedges,et al.  Fixed-Effects Models , 2022, The SAGE Encyclopedia of Research Design.

[14]  Daniel Koretz,et al.  The Validity of Gains in Scores on the Kentucky Instructional Results Information System (KIRIS). , 1998 .

[15]  Daniel F. McCaffrey,et al.  What Do Test Scores in Texas Tell Us , 2000 .

[16]  Lorrie Shepard.  Setting Performance Standards for Student Achievement. A Report of the National Academy of Education Panel on the Evaluation of the NAEP Trial State Assessment: An Evaluation of the 1992 Achievement Levels. , 1993 .

[17]  Alija Kulenović,et al.  Standards for Educational and Psychological Testing , 1999 .

[18]  Nancy L. Allen,et al.  Technical Report of the NAEP 1996 State Assessment Program in Mathematics. , 1997 .

[19]  Stephen W. Raudenbush,et al.  Random effects models. , 1994 .

[20]  Gregory J. Cizek,et al.  Setting performance standards : concepts, methods, and perspectives , 2001 .

[21]  John A. Centra,et al.  The Student as Godfather? The Impact of Student Ratings on Academia1 , 1973 .

[22]  Martha L. Stocking,et al.  Developing a Common Metric in Item Response Theory , 1982 .

[23]  More problems with gap closing philosophy and research. , 2005, The American psychologist.

[24]  Ronald K. Hambleton,et al.  A Response to "Setting Reasonable and Useful Performance Standards" in the National Academy of Science's Grading the Nations Report Card , 2005 .

[25]  William R. Shadish,et al.  Combining estimates of effect size. , 1994 .

[26]  Educational Evaluation.  Standards for Educational and Psychological Testing , 1999 .

[27]  Wendy M. Yen,et al.  Multiple Measures: Alternative Design and Analysis Models , 2005 .