A Conditional Exposure Control Method for Multidimensional Adaptive Testing

In computerized adaptive testing (CAT), ensuring the security of test items is a crucial practical consideration. A common approach to reducing item theft is to define maximum item exposure rates, i.e., to limit the proportion of examinees to whom a given item can be administered. Numerous methods for controlling exposure rates have been proposed for tests employing the unidimensional 3-PL model. The present article explores the issues associated with controlling exposure rates when a multidimensional item response theory (MIRT) model is utilized and exposure rates must be controlled conditional upon ability. This situation is complicated by the exponentially increasing number of possible ability values in multiple dimensions. The article introduces a new procedure, called the generalized Stocking-Lewis method, that controls the exposure rate for students of comparable ability as well as with respect to the overall population. A realistic simulation set compares the new method with three other approaches: Kullback-Leibler information with no exposure control, Kullback-Leibler information with unconditional Sympson-Hetter exposure control, and random item selection.

[1]  Daniel M. Bolt,et al.  Estimation of Compensatory and Noncompensatory Multidimensional Item Response Models Using Markov Chain Monte Carlo , 2003 .

[2]  Wim J van der Linden,et al.  Constraining Item Exposure in Computerized Adaptive Testing With Shadow Tests , 2002 .

[3]  Martha L. Stocking,et al.  Controlling Item Exposure Conditional on Ability in Computerized Adaptive Testing , 1998 .

[4]  Hua-Hua Chang,et al.  A Global Information Approach to Computerized Adaptive Testing , 1996 .

[5]  Mark D. Reckase,et al.  The Difficulty of Test Items That Measure More Than One Ability , 1985 .

[6]  J. Mcbride,et al.  Reliability and Validity of Adaptive Ability Tests in a Military Setting , 1983 .

[7]  Richard M. Luecht,et al.  Multidimensional Computerized Adaptive Testing in a Certification or Licensure Context , 1996 .

[8]  Terry Ackerman,et al.  Graphical Representation of Multidimensional Item Response Theory Analyses , 1996 .

[9]  D. Segall.  Principles of Multidimensional Adaptive Testing , 2000 .

[10]  Willem J. van der Linden,et al.  Multidimensional Adaptive Testing with a Minimum Error-Variance Criterion , 1999 .

[11]  M. Stocking,et al.  A Model and Heuristic For Solving Very Large Item Selection Problems , 1993 .

[12]  C. Parsons,et al.  Application of Unidimensional Item Response Theory Models to Multidimensional Data , 1983 .

[13]  Z. Ying,et al.  a-Stratified Multistage Computerized Adaptive Testing with b Blocking , 2001 .

[14]  Daniel O. Segall,et al.  Multidimensional adaptive testing , 1996 .

[15]  Cheng-Der Fuh,et al.  A Strategy for Controlling Item Exposure in Multidimensional Computerized Adaptive Testing , 2008 .

[16]  William D. Schafer,et al.  Trait Parameter Recovery Using Multidimensional Computerized Adaptive Testing in Reading and Mathematics , 2005 .

[17]  Martha L. Stocking,et al.  A Method for Severely Constrained Item Selection in Adaptive Testing , 1992 .

[18]  Z. Ying,et al.  a-Stratified Multistage Computerized Adaptive Testing , 1999 .

[19]  Bernard P. Veldkamp,et al.  Multidimensional adaptive testing with constraints on test content , 2002 .

[20]  David A. Harrison,et al.  Robustness of Irt Parameter Estimation to Violations of The Unidimensionality Assumption , 1986 .