Item Selection for the Development of Parallel Forms From an IRT-Based Seed Test Using a Sampling and Classification Approach

Two sampling-and-classification–based procedures were developed for automated test assembly: the Cell Only and the Cell and Cube methods. A simulation study based on a 540-item bank was conducted to compare the performance of the procedures with the performance of a mixed-integer programming (MIP) method for assembling multiple parallel test forms. The study investigated the statistical equivalence of the forms generated by the three test assembly methods (Cell Only, Cell and Cube, and MIP) in terms of test information functions, test characteristic curves, mean square deviations, and practical constraints, such as content balancing and nonoverlap among forms. The results indicated that the 13-point MIP method outperformed the other two methods in terms of the “closeness” test information functions between the reference form and the assembled parallel tests. Regarding test characteristic curves, the Cell Only and Cell and Cube methods yielded more similar test characteristic curves than the MIP method. Constraining test information functions apparently does not guarantee that the assembled forms will yield similar test characteristic curves. Overall, the Cell Only and Cell and Cube methods have the potential to provide results similar to the optimization approach.

[1]  Richard M. Leucht Computer-Assisted Test Assembly Using Optimization Heuristics. , 1998 .

[2]  Dmitry I. Belov,et al.  Monte Carlo Test Assembly for Item Pool Analysis and Extension , 2005 .

[3]  Pei-hua Chen,et al.  IRT-based automated test assembly: a sampling and stratification perspective , 2005 .

[4]  F. Samejima Weakly parallel tests in latent trait theory with some criticisms of classical test theory , 1977 .

[5]  R. Luecht,et al.  Item Selection Using an Average Growth Approximation of Target Information Functions , 1992 .

[6]  Ellen Boekkooi-Timminga The Construction of Parallel Tests From IRT-Based Item Banks , 1989 .

[7]  Matthew Finkelman,et al.  Automated Test Assembly for Cognitive Diagnosis Models Using a Genetic Algorithm , 2009 .

[8]  Mark J. Gierl,et al.  Computer Software Review: Conducting Automated Test Assembly Using the Premium Solver Platform Version 7.0 With Microsoft Excel and the Large-Scale LP/QP Solver Engine Add-In , 2008 .

[9]  Willem J. van der Linden,et al.  Linear Models for Optimal Test Design , 2005 .

[10]  Shameem Nyla NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION , 2004 .

[11]  Donovan R. Hare,et al.  Automated Simultaneous Assembly of Multistage Testlets for a High-Stakes Licensing Examination , 2007 .

[12]  Bernard P. Veldkamp,et al.  Multidimensional Constrained Test Assembly , 2002 .

[13]  H. Gulliksen Theory of mental tests , 1952 .

[14]  Ronald D. Armstrong,et al.  An automated test development of parallel tests from a seed test , 1992 .

[15]  Alfred J. Verschoor,et al.  Parallel Test Construction Using Classical Item Parameters , 1998 .

[16]  Fred W. Glover,et al.  A user's guide to tabu search , 1993, Ann. Oper. Res..

[17]  M. Stocking,et al.  A Model and Heuristic For Solving Very Large Item Selection Problems , 1993 .

[18]  Frederick S. Hillier,et al.  Introduction of Operations Research , 1967 .

[19]  F. S. Hillier,et al.  Introduction to Operations Research, 10th ed. , 1986 .

[20]  Mark J. Gierl,et al.  Three Applications of Automated Test Assembly within a User-Friendly Modeling Environment , 2009 .

[21]  Michael R. Bussieck,et al.  General Algebraic Modeling System (GAMS) , 2004 .

[22]  T. Davey Linear Models for Optimal Test Design by van der Linden, W.J. , 2009 .

[23]  Linda F. Wightman Practical Issues in Computerized Test Assembly , 1998 .

[24]  Dmitry I. Belov Uniform Test Assembly , 2008 .