A Bayesian Procedure in the Context of Sequential Mastery Testing

The purpose of this paper is to derive optimal rules for sequential mastery tests. In a sequential mastery test, the decision is to classify a subject as a master, a nonmaster, or continuing testing and administering another random item. The framework of Bayesian sequential decision theory is used; that is, optimal rules are obtained by minimizing the posterior expected losses associated with all possible decision rules at each stage of testing. The main advantage of this approach is that costs of testing can be taken explicitly into account. The binomial model is assumed for the probability of a correct response given the true level of functioning, whereas threshold loss is adopted for the loss function involved. The paper concludes with a simulation study, in which the Bayesian sequential strategy is compared with other procedures that exist for similar classification decision problems in the literature.

[1]  Judith A. Spray.  Multiple-Category Classification Using a Sequential Probability Ratio Test. , 1993 .

[2]  Willem J. van der Linden,et al.  A compensatory approach to optimal selection with mastery scores , 1994 .

[3]  H J Vos,et al.  A Simultaneous Approach to Optimizing Treatment Assignments with Mastery Scores. , 1997, Multivariate behavioral research.

[4]  David J. Weiss,et al.  A Comparison of IRT-Based Adaptive Mastery Testing and a Sequential Mastery Testing Procedure , 1983 .

[5]  P. Armitage.  Sequential Analysis with More than Two Alternative Hypotheses, and its Relation to Discriminant Function Analysis , 1950 .

[6]  Kathleen M. Sheehan,et al.  Using Bayesian Decision Theory to Design a Computerized Mastery Test , 1990 .

[7]  David J. Weiss,et al.  APPLICATION OF COMPUTERIZED ADAPTIVE TESTING TO EDUCATIONAL PROBLEMS , 1984 .

[8]  J. Andel.  Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[9]  Rand R. Wilcox,et al.  A Review of the Beta-Binomial Model and its Extensions , 1981 .

[10]  L. Nedelsky.  Absolute Grading Standards for Objective Tests , 1954 .

[11]  R. Owen,et al.  A Bayesian Sequential Procedure for Quantal Response in the Context of Adaptive Mental Testing , 1975 .

[12]  Mark D. Reckase,et al.  Comparison of SPRT and Sequential Bayes Procedures for Classifying Examinees Into Two Categories Using a Computerized Test , 1996 .

[13]  Willem J. van der Linden,et al.  Decision models for use with criterion-referenced tests , 1980 .

[14]  Kathleen M. Sheehan,et al.  Computerized Mastery Testing With Nonequivalent Testlets , 1990 .

[15]  Cornelis A.W. Glas,et al.  Testlet-based adaptive mastery testing , 1999 .

[16]  Mark D. Reckase,et al.  A Procedure for Decision Making Using Tailored Testing , 1983 .

[17]  R. A. FISHER.  Tables for Statisticians and Biometricians , 1933, Nature.

[18]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[19]  W. R. Buckland,et al.  Distributions in Statistics: Continuous Multivariate Distributions , 1973 .

[20]  Wim J. van der Linden,et al.  Optimal Cutting Scores Using A Linear Loss Function , 1977 .

[21]  Hendrik J. Vos.  Simultaneous optimization of quota‐restricted selection decisions with mastery scores , 1997 .

[22]  M. Degroot.  Optimal Statistical Decisions , 1970 .

[23]  Wim J. van der Linden,et al.  Applications of decision theory to test-based decision making , 1987 .