Item Selection Criteria With Practical Constraints in Cognitive Diagnostic Computerized Adaptive Testing

For item selection in cognitive diagnostic computerized adaptive testing (CD-CAT), ideally, a single item selection index should be created to simultaneously regulate precision, exposure status, and attribute balancing. For this purpose, in this study, we first proposed an attribute-balanced item selection criterion, namely, the standardized weighted deviation global discrimination index (SWDGDI), and subsequently formulated the constrained progressive index (CP_SWDGDI) by casting the SWDGDI in a progressive algorithm. A simulation study revealed that the SWDGDI method was effective in balancing attribute coverage and the CP_SWDGDI method was able to simultaneously balance attribute coverage and item pool usage while maintaining acceptable estimation precision. This research also demonstrates the advantage of a relatively low number of attributes in CD-CAT applications.

[1]  Theodorus Johannes Hendrikus Maria Eggen,et al.  Item Selection in Adaptive Testing with the Sequential Probability Ratio Test , 1999 .

[2]  Hua-Hua Chang,et al.  Restrictive Stochastic Item Selection Methods in Cognitive Diagnostic Computerized Adaptive Testing. , 2011 .

[3]  Z. Ying,et al.  a-Stratified Multistage Computerized Adaptive Testing with b Blocking , 2001 .

[4]  M. Stocking,et al.  A Model and Heuristic For Solving Very Large Item Selection Problems , 1993 .

[5]  Ying Cheng.  Improving Cognitive Diagnostic Computerized Adaptive Testing by Balancing Attribute Coverage: The Modified Maximum Global Discrimination Index Method , 2010 .

[6]  Vicente Ponsoda,et al.  A Comparison of Item Exposure Control Methods in Computerized Adaptive Testing , 1998 .

[7]  Z. Ying,et al.  a-Stratified Multistage Computerized Adaptive Testing , 1999 .

[8]  Edward H. Haertel.  Using restricted latent class models to map the skill structure of achievement items , 1989 .

[9]  Shu-ying Chen,et al.  Effects of Practical Constraints on Item Selection Rules at the Early Stages of Computerized Adaptive Testing , 2004 .

[10]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[11]  Angela J. Verschoor,et al.  A Binary Programming Approach to Automated Test Assembly for Cognitive Diagnosis Models , 2010 .

[12]  David J. Weiss,et al.  Improving Measurement Quality and Efficiency with Adaptive Testing , 1982 .

[13]  Hua-Hua Chang,et al.  A Global Information Approach to Computerized Adaptive Testing , 1996 .

[14]  K. Tatsuoka.  RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .

[15]  Jeffrey A Douglas,et al.  Test Construction for Cognitive Diagnosis , 2005 .

[16]  Hua-Hua Chang,et al.  Two-Phase Item Selection Procedure for Flexible Content Balancing in CAT , 2007 .

[17]  Pui-Wa Lei,et al.  Controlling item exposure and test overlap on the fly in computerized adaptive testing. , 2008, The British journal of mathematical and statistical psychology.

[18]  Chuan Ju Lin.  Item Selection Criteria With Practical Constraints for Computerized Classification Testing , 2011 .

[19]  Louis A. Roussos,et al.  The fusion model skills diagnosis system , 2007 .

[20]  Ying Cheng.  When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT , 2009 .

[21]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[22]  Hua-Hua Chang,et al.  A Comparison of Item Selection Rules at the Early Stages of Computerized Adaptive Testing , 2000 .