An Adaptive Testing Strategy for Mastery Decisions.

Abstract : In an attempt to increase the efficiency of mastery testing while maintaining a high level of confidence for each mastery decision, the theory and technology of item characteristics curve (ICC) response theory and adaptive testing were applied to the problem of judging individuals' competencies against a prespecified mastery level to determine whether each individual is a 'master' or a 'nonmaster' of a specified content domain. Items from two conventionally administered classroom mastery tests administered in a military training environment were calibrated using the unidimensional three-parameter logistic ICC model. Then, using response data originally obtained from the conventional administration of the tests, a computerized adaptive mastery testing (AMT) strategy was applied in a real-data simulation. Results obtained from the AMT procedure were compared to results obtained from the traditional mastery testing paradigm in terms of the reduction in mean test length, information characteristics, and the correspondence between decisions made by the two procedures for three different mastery levels and for each of the two tests. The AMT procedure reduced the average test length 30% to 81% over all circumstances examined (with modal test length reductions of up to 92%), while reaching the same decision as the conventional procedure for 96% of the trainees. Additional advantages and possible applications of AMT procedures in certain classroom situations are noted and discussed.

[1]  M. Reckase.  Unifactor Latent Trait Models Applied to Multifactor Tests: Results and Implications , 1979 .

[2]  D. Weiss,et al.  Effect of Point-in-Time in Instruction on the Measurement of Achievement. , 1979 .

[3]  David J. Weiss,et al.  Relationships Among Achievement Level Estimates from Three Item Characteristic Curve Scoring Methods. , 1979 .

[4]  S. M. Pine,et al.  Effects of Computerized Adaptive Testing on Black and White Students. , 1979 .

[5]  David J. Weiss,et al.  Computer Programs for Scoring Test Data with Item Characteristic Curve Models. , 1979 .

[6]  David J. Weiss,et al.  An Item Bias Investigation of a Standardized Aptitude Test. , 1978 .

[7]  David J. Weiss,et al.  A Construct Validation of Adaptive Achievement Testing. , 1978 .

[8]  David J. Weiss,et al.  A Comparison of the Fairness of Adaptive and Conventional Testing Strategies. , 1978 .

[9]  David J. Weiss,et al.  An Adaptive Testing Strategy for Achievement Test Batteries. , 1977 .

[10]  David J. Weiss,et al.  An Information Comparison of Conventional and Adaptive Tests in the Measurement of Classroom Achievement. , 1977 .

[11]  Isaac I. Bejar.  Calibration of an Item Pool for the Adaptive Measurement of Achievement. , 1977 .

[12]  David J. Weiss,et al.  A Rapid Item-Search Procedure for Bayesian Adaptive Testing. , 1977 .

[13]  David J. Weiss,et al.  A Comparison of Information Functions of Multiple-Choice and Free-Response Vocabulary Items. Research Report 77-2. , 1977 .

[14]  James R. McBride.  Some Properties of a Bayesian Adaptive Ability Testing Strategy , 1977 .

[15]  David J. Weiss,et al.  Effects of Item Characteristics on Test Fairness. , 1976 .

[16]  Nancy E. Betz,et al.  Effects of Immediate Knowledge of Results and Adaptive Testing on Ability Test Performance , 1976 .

[17]  David J. Weiss,et al.  Psychological Effects of Immediate Knowledge of Results and Adaptive Ability Testing. Research Report 76-4. , 1976 .

[18]  David J. Weiss,et al.  A Simulation Study of Stradaptive Ability Testing. , 1975 .

[19]  David J. Weiss,et al.  Empirical and Simulation Studies of Flexilevel Ability Testing. Research Report No. 75-3. , 1975 .

[20]  David J. Weiss,et al.  Strategies of Adaptive Ability Measurement. , 1974 .

[21]  David J. Weiss,et al.  Simulation Studies of Two-Stage Ability Testing. , 1974 .

[22]  David J. Weiss,et al.  An Empirical Investigation of Computer-Administered Pyramidal Ability Testing. Research Report No. 74-3. , 1974 .

[23]  David J. Weiss,et al.  A Computer Software System for Adaptive Ability Measurement. , 1974 .

[24]  Ronald K. Hambleton,et al.  TOWARD AN INTEGRATION OF THEORY AND METHOD FOR CRITERION-REFERENCED TESTS1,2 , 1973 .

[25]  David J. Weiss,et al.  Ability Measurement: Conventional or Adaptive. , 1973 .

[26]  Roger J. Owen.  A BAYESIAN APPROACH TO TAILORED TESTING , 1969 .

[27]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[28]  F. Samejima.  Estimation of latent ability using a response pattern of graded scores , 1968 .

[29]  Melvin R. Novick,et al.  Some latent train models and their use in inferring an examinee's ability , 1966 .

[30]  J. Horn.  A rationale and test for the number of factors in factor analysis , 1965, Psychometrika.

[31]  F. Yates.  Contributions to Mathematical Statistics , 1951, Nature.