A Practitioner's Guide for Variable-length Computerized Classification Testing

Variable-length computerized classification tests, CCTs, (Lin & Spray, 2000; Thompson, 2006) are a powerful and efficient approach to testing for the purpose of classifying examinees into groups. CCTs are designed by the specification of at least five technical components: psychometric model, calibrated item bank, starting point, item selection algorithm, and termination criterion. Several options exist for each of these CCT components, creating a myriad of possible designs. Confusion among designs is exacerbated by the lack of a standardized nomenclature. This article outlines the components of a CCT, common options for each component, and the interaction of options for different components, so that practitioners may more efficiently design CCTs. It also offers a suggestion of nomenclature.

[1]  Lawrence M. Rudner.  An Examination of Decision-Theory Adaptive Testing Procedures , 2009 .

[2]  Judith A. Spray,et al.  The Effect of Model Misspecification on Classification Decisions Made Using a Computerized Test , 1999 .

[3]  Theodorus Johannes Hendrikus Maria Eggen,et al.  Computerized Adaptive Testing for Classifying Examinees into three Categories , 2000 .

[4]  David J. Weiss,et al.  A Comparison of IRT-Based Adaptive Mastery Testing and a Sequential Mastery Testing Procedure , 1983 .

[5]  Chuan-Ju Lin,et al.  Effects of Item-Selection Criteria on Classification Testing with the Sequential Probability Ratio Test. ACT Research Report Series. , 2000 .

[6]  Chi-Yu Huang,et al.  Unidimensional Approximations for a Computerized Classification Test When the Item Pool and Latent Space Are Multidimensional. ACT Research Report Series 97-5. , 1997 .

[7]  Beiling Xiao.  Strategies for Computerized Adaptive Grading Testing , 1999 .

[8]  Tianyou Wang,et al.  Computerized Classification Testing under Practical Constraints with a Polytomous Model , 1999 .

[9]  Barbara G. Dodd,et al.  Computerized Adaptive Testing With Polytomous Items , 1995 .

[10]  Cornelis A.W. Glas,et al.  Testlet-based adaptive mastery testing , 1999 .

[11]  David J. Weiss,et al.  APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS , 1984 .

[12]  Hans J. Vos,et al.  A Bayesian Procedure in the Context of Sequential Mastery Testing , 2000 .

[13]  Mark D. Reckase,et al.  Comparison of SPRT and Sequential Bayes Procedures for Classifying Examinees Into Two Categories Using a Computerized Test , 1996 .

[14]  R. Linn,et al.  Sequential Testing for Dichotomous Decisions , 1970 .

[15]  Mark D. Reckase,et al.  A Procedure for Decision Making Using Tailored Testing , 1983 .

[16]  Kathleen M. Sheehan,et al.  Using Bayesian Decision Theory to Design a Computerized Mastery Test , 1990 .

[17]  R. Weitzman.  Sequential Testing for Selection , 1982 .

[18]  Howard Wainer,et al.  Computerized Adaptive Testing: A Primer , 2000 .

[19]  R. Owen,et al.  A Bayesian Sequential Procedure for Quantal Response in the Context of Adaptive Mental Testing , 1975 .

[20]  Mark D. Reckase,et al.  The Selection of Test Items for Decision Making with a Computer Adaptive Test. , 1994 .

[21]  Alexander Weissman.  A Feedback Control Strategy for Enhancing Item Selection Efficiency in Computerized Adaptive Testing , 2006 .

[22]  E. Muraki.  A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[23]  A. Wald,et al.  A Sequential Decision Procedure for Choosing One of Three Hypotheses Concerning the Unknown Mean of a Normal Distribution , 1949 .

[24]  Chi-Yu Huang,et al.  Unidimensional Approximations for a Computerized Test When the Item Pool and Latent Space are Multidimensional , 1997 .

[25]  Judith A. Spray,et al.  Effect of Item Selection on Item Exposure Rates within a Computerized Classification Test , 1998 .

[26]  Tianyou Wang,et al.  Comparing and Combining Dichotomous and Polytomous Items with SPRT Procedure in Computerized Classification Testing , 1998 .

[27]  Z. Govindarajulu.  The sequential statistical analysis of hypothesis testing, point and interval estimation, and decision theory , 1987 .

[28]  Y. Chang,et al.  Application of Sequential Interval Estimation to Adaptive Mastery Testing , 2005 .

[29]  Kathleen M. Sheehan,et al.  Computerized Mastery Testing With Nonequivalent Testlets , 1990 .

[30]  S. Embretson,et al.  Item response theory for psychologists , 2000 .

[31]  Theodorus Johannes Hendrikus Maria Eggen,et al.  Item Selection in Adaptive Testing with the Sequential Probability Ratio Test , 1999 .

[32]  Theodore W. Frick.  Computerized Adaptive Mastery Tests as Expert Systems , 1992 .

[33]  Douglas R. Glasnapp,et al.  Effects of Estimation Bias on Multiple-Category Classification With an IRT-Based Adaptive Classification Procedure , 2006 .

[34]  Cynthia G. Parshall,et al.  Practical Considerations in Computer-Based Testing , 2002 .

[35]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[36]  P. Armitage.  Sequential Analysis with More than Two Alternative Hypotheses, and its Relation to Discriminant Function Analysis , 1950 .

[37]  Chuan-Ju Lin,et al.  Estimating Item Parameters from Classical Indices for Item Pool Development with a Computerized Classification Test. ACT Research Report Series. , 2000 .

[38]  Judith A. Spray.  Multiple-Category Classification Using a Sequential Probability Ratio Test. , 1993 .

[39]  Hong Jiao.  The Effects of Model Misfit in Computerized Classification Test , 2003 .

[40]  Mark D. Reckase,et al.  The Effect of Item Parameter Estimation Error on Decisions Made Using the Sequential Probability Ratio Test , 1987 .

[41]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.