Constraining Item Exposure in Computerized Adaptive Testing With Shadow Tests

Item-exposure control in computerized adaptive testing is implemented by imposing item-ineligibility constraints on the assembly process of the shadow tests. The method resembles Sympson and Hetter’s (1985) method of item-exposure control in that the decisions to impose the constraints are probabilistic. The method does not, however, require time-consuming simulation studies to set values for control parameters before the operational use of the test. Instead, it can set the probabilities of item ineligibility adaptively during the test using the actual item-exposure rates. An empirical study using an item pool from the Law School Admission Test showed that application of the method yielded perfect control of the item-exposure rates and had negligible impact on the bias and mean-squared error functions of the ability estimator.

[1]  Vicente Ponsoda,et al.  A Comparison of Item Exposure Control Methods in Computerized Adaptive Testing , 1998 .

[2]  Bernard P. Veldkamp,et al.  An Integer Programming Approach to Item Bank Design , 2000 .

[3]  Bernard P. Veldkamp,et al.  Designing item pools for computerized adaptive testing , 1999 .

[4]  Cornelis A.W. Glas,et al.  Computerized adaptive testing : theory and practice , 2000 .

[5]  Cynthia G. Parshall,et al.  New Algorithms for Item Selection and Exposure Control with Computerized Adaptive Testing. , 1995 .

[6]  Anthony R. Zara,et al.  Procedures for Selecting Items for Computerized Adaptive Tests. , 1989 .

[7]  van der Linden,et al.  Constrained adaptive testing with shadow tests , 2000 .

[8]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[9]  Wim J. van der Linden,et al.  A Model for Optimal Constrained Adaptive Testing , 1998 .

[10]  J. Mcbride,et al.  Reliability and Validity of Adaptive Ability Tests in a Military Setting , 1983 .

[11]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[12]  Martha L. Stocking,et al.  Optimal Design of Item Banks for Computerized Adaptive Tests , 1996 .

[13]  Richard M. Leucht.  Computer-Assisted Test Assembly Using Optimization Heuristics. , 1998 .

[14]  Howard Wainer.  Rescuing Computerized Testing by Breaking Zipf's Law , 2000 .

[15]  Hua-Hua Chang,et al.  Implementing Content Constraints in Alpha-Stratified Adaptive Testing Using a Shadow Test Approach , 2001 .

[16]  Martha L. Stocking,et al.  Controlling Item Exposure Conditional on Ability in Computerized Adaptive Testing , 1995 .

[17]  Martha L. Stocking,et al.  Methods of Controlling the Exposure of Items in CAT , 2000 .

[18]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[19]  Martha L. Stocking,et al.  A Method for Severely Constrained Item Selection in Adaptive Testing , 1992 .

[20]  Z. Ying,et al.  a-Stratified Multistage Computerized Adaptive Testing , 1999 .

[21]  Deborah J. Harris,et al.  Redeveloping the exposure control parameters of CAT items when a pool is modified , 2002 .

[22]  Martha L. Stocking,et al.  A Method for Severely Constrained Item Selection in Adaptive Testing , 1992 .

[23]  Manfred Steffen,et al.  The GRE Computer Adaptive Test: Operational Issues , 2000 .

[24]  Willem J. van der Linden,et al.  Optimal Assembly of Psychological and Educational Tests , 1998 .