Comparability of Computer and Paper-and-Pencil Versions of Algebra and Biology Assessments

This study examined comparability of student scores obtained from computerized and paper-and-pencil formats of the large-scale statewide end-of-course (EOC) examinations in the two subject areas of Algebra and Biology. Evidence in support of comparability of computerized and paper-based tests was sought by examining scale scores, item parameter estimates, test characteristic curves, test information functions, Rasch ability estimates at the content domain level, and the equivalence of the construct. Overall, the results support the comparability of computerized and paper-based tests at the item-level, subtest-level and whole test-level in both subject areas. For both subject areas, no evidence was found to suggest that the administration mode changed the construct being measured.

[1]  Gordon W. Cheung,et al.  Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance , 2002 .

[2]  Randy Elliot Bennett,et al.  Online Assessment in Mathematics and Writing: Reports from the NAEP Technology-Based Assessment Project, Research and Development Series. NCES 2005-457. , 2005 .

[3]  P. Lachenbruch.  Statistical Power Analysis for the Behavioral Sciences (2nd ed.) , 1989 .

[4]  Michael Russell.  Testing On Computers , 1999 .

[5]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[6]  Fritz Drasgow.  The work ahead: A psychometric infrastructure for computerized adaptive tests , 2005 .

[7]  Tom Plati,et al.  Effects of Computer Versus Paper Administrations of a State-Mandated Writing Assessment , 2022 .

[8]  H. Huynh,et al.  Computer-Based and Paper-and-Pencil Administration Mode Effects on a Statewide End-of-Course English Test , 2008 .

[9]  Michael Russell,et al.  Testing On Computers , 1999 .

[10]  Steven J. Fitzpatrick,et al.  Score Comparability of Online and Paper Administrations of the Texas Assessment of Knowledge and Skills , 2006 .

[11]  Katie Larsen McClarty,et al.  Item-Level Comparative Analysis of Online and Paper Administrations of the Texas Assessment of Knowledge and Skills , 2008 .

[12]  P. Bentler,et al.  Cutoff criteria for fit indexes in covariance structure analysis : Conventional criteria versus new alternatives , 1999 .

[13]  B. Wright,et al.  Best test design , 1979 .

[14]  K. Goulden,et al.  Effect Sizes for Research: A Broad Practical Approach , 2006 .

[15]  Mary Pommerich,et al.  Developing Computerized Versions of Paper-and-Pencil Tests: Mode Effects for Passage-Based Tests , 2004 .

[16]  Andrew J. Poggio,et al.  A Comparative Evaluation of Score Results from Computerized and Paper & Pencil Mathematics Testing in a Large Scale State Assessment Program , 2005 .

[17]  Michael Russell,et al.  Testing Writing on Computers: An Experiment Comparing Student Performance on Tests Conducted via Computer and via Paper-and-Pencil , 1997 .