Refining Methods for Estimating Critical Values for an Alignment Index

Abstract The alignment among standards, assessments, and teachers’ instruction is an essential element of standards-based educational reforms. The Surveys of Enacted Curriculum (SEC) is the only common tool that can be used to measure the alignment among all three of these sources (Martone & Sireci, 2009). Prior SEC alignment work has been limited by not allowing for significance tests. A recent article (Fulmer, 2011) provided a first attempt to address this shortcoming of the SEC, but that work was limited in several ways. We extend Fulmer's simulation approach by accounting for important elements of the SEC procedures, including the proper framework size, number of standards and assessment points, number of raters, rater cell-splitting rates, and rater agreement results. The results indicate that inferences about relative alignment may be heavily influenced by features of the alignment procedures. Thus, our method should be broadly applied to future SEC alignment investigations.

[1]  Stephen G. Sireci,et al.  Evaluating Alignment Between Curriculum, Assessment, and Instruction , 2009 .

[2]  Andrew C. Porter,et al.  Common Core Standards , 2011 .

[3]  Xiufeng Liu,et al.  Alignment Between the Science Curriculum and Assessment in Selected NY State Regents Exams , 2008 .

[4]  Morgan S. Polikoff.  The Association of State Policy Attributes With Teachers’ Instructional Alignment , 2012 .

[5]  A. W. F. Edwards,et al.  A History of Likelihood , 1974 .

[6]  Andrew C. Porter,et al.  Reform up Close: An Analysis of High School Mathematics and Science Classrooms. Final Report. , 1993 .

[7]  Andrew C. Porter,et al.  Measuring the Content of Instruction: Uses in Research and Practice , 2002 .

[8]  Norman L. Webb,et al.  Alignment of Science and Mathematics Standards and Assessments in Four States. Research Monograph No. 18. , 1999 .

[9]  Morgan S. Polikoff,et al.  The Quality of Content Analyses of State Student Achievement Tests and Content Standards , 2008 .

[10]  Andrew C. Porter,et al.  Alignment as a Teacher Variable , 2007 .

[11]  Andrew C. Porter,et al.  New Tools for Analyzing Teaching, Curriculum and Standards in Mathematics & Science. Results from Survey of Enacted Curriculum Project. Final Report. , 2001 .

[12]  Morgan S. Polikoff.  Instructional Alignment under No Child Left Behind , 2012, American Journal of Education.

[13]  The Redundancy of Mathematics Instruction in U.S. Elementary and Middle Schools , 2012, The Elementary School Journal.

[14]  Gavin W. Fulmer.  Estimating Critical Values for Strength of Alignment Among Curriculum, Assessments, and Instruction , 2011 .

[15]  James M. Robins,et al.  Conditioning, Likelihood, and Coherence: A Review of Some Foundational Concepts , 2000 .

[16]  Andrew C. Porter,et al.  Is There a de Facto National Intended Curriculum? Evidence From State Content Standards , 2009 .

[17]  John Schwille,et al.  Content Decision Making and The Politics of Education , 1979 .

[18]  Andrew C. Porter,et al.  A Curriculum out of Balance , 1989 .

[19]  Andrew C. Porter,et al.  How Well Aligned Are State Assessments of Student Achievement With State Content Standards? , 2011 .

[20]  Beaumie Kim,et al.  Alignment between the physics content standard and the standardized test: A comparison among the United States‐New York State, Singapore, and China‐Jiangsu , 2009 .

[21]  John R. Schwille,et al.  Teachers as Policy Brokers in the Content of Elementary School Mathematics. Research Series No. 113. , 1982 .