An Alternative Method for Scoring Adaptive Tests

Modern applications of computerized adaptive testing are typically grounded in item response theory (IRT; Lord, 1980). While the IRT foundations of adaptive testing provide a number of approaches to adaptive test scoring that may seem natural and efficient to psychometricians, these approaches may be more demanding for test takers, test score users, and interested regulatory institutions to comprehend. An alternative method, based on more familiar equated number-correct scores and identical to that used to score and equate many conventional tests, is explored and compared with one that relies more directly on IRT. It is concluded that scoring adaptive tests using the familiar number-correct score, accompanied by the necessary equating to adjust for the intentional differences in adaptive test difficulty, is a statistically viable, although slightly less efficient, method of adaptive test scoring. To enhance the prospects for enlightened public debate about adaptive testing, it may be preferable to use this more familiar approach. Public attention would then likely be focused on issues more central to adaptive testing, namely, the adaptive nature of the test.

[1]  M. Stocking,et al.  A Model and Heuristic For Solving Very Large Item Selection Problems , 1993 .

[2]  R. Owen,et al.  A Bayesian Sequential Procedure for Quantal Response in the Context of Adaptive Mental Testing , 1975 .

[3]  R. Mislevy.  Estimating latent distributions , 1984 .

[4]  Linda L. Cook,et al.  Irt Versus Conventional Equating Methods: A Comparative Study of Scale Stability , 1983 .

[5]  F. Lord,et al.  An Investigation of Methods for Reducing Sampling Error in Certain IRT Procedures , 1983 .

[6]  Linda L. Cook,et al.  Problems Related to the Use of Conventional and Item Response Theory Equating Methods in Less Than Optimal Circumstances , 1987 .

[7]  Mark D. Reckase,et al.  TECHNICAL GUIDELINES FOR ASSESSING COMPUTERIZED ADAPTIVE TESTS , 1984 .

[8]  I. Bejar,et al.  A Study of Pre-Equating Based on Item Response Theory , 1982 .

[9]  Martha L. Stocking.  Controlling Item Exposure Rates in a Realistic Adaptive Testing Paradigm. , 1993 .

[10]  Martha L. Stocking,et al.  Two Simulated Feasibility Studies in Computerised Adaptive Testing , 1987 .

[11]  David J. Weiss,et al.  Strategies of Adaptive Ability Measurement. , 1974 .

[12]  J. Mcbride,et al.  Reliability and Validity of Adaptive Ability Tests in a Military Setting , 1983 .

[13]  Robert J. Mislevy,et al.  Bayes modal estimation in item response models , 1986 .

[14]  F. Lord.  Applications of Item Response Theory To Practical Testing Problems , 1980 .

[15]  Ronald K. Hambleton,et al.  Applications of Item Response Theory , 1983 .

[16]  M C Killcross.  A Review of Research in Tailored Testing , 1976 .

[17]  William C. Ward.  The College Board computerized placement tests: an application of computerized adaptive testing , 1987 .

[18]  Gail A. Herndon.  The chronicle of higher education , 1977 .

[19]  Martha L. Stocking,et al.  A Method for Severely Constrained Item Selection in Adaptive Testing , 1992 .

[20]  Effect on Equating Results of Matching Samples on an Anchor Test. , 1990 .

[21]  Martha L. Stocking,et al.  TWO SIMULATED FEASIBILITY STUDIES IN COMPUTERIZED ADAPTIVE TESTING (CAT)1,2,3 , 1984 .

[22]  Frederic M. Lord,et al.  Unbiased estimators of ability parameters, of their variance, and of their parallel-forms reliability , 1983 .

[23]  Martha L. Stocking,et al.  CASE STUDIES IN COMPUTER ADAPTIVE TEST DESIGN THROUGH SIMULATION1,2 , 1993 .

[24]  Linda L. Cook,et al.  Simulation Results of Effects on Linear and Curvilinear Observed-and True-Score Equating Procedures of Matching on a Fallible Criterion , 1990 .

[25]  Frederic M. Lord.  A BROAD-RANGE TAILORED TEST OF VERBAL ABILITY , 1975 .

[26]  David J. Weiss,et al.  Improving Measurement Quality and Efficiency with Adaptive Testing , 1982 .