Investigating the saltus model as a tool for setting standards

The saltus model (Wilson, 1989; Draney, Wilson, Gluck, & Spiel, 2008) is a latent variable mixture model originally designed for detecting developmental discontinuities. One difference between this model and many other latent class models is that a clear ordering of the latent classes is hypothesized, from lowest to highest proficiency. At higher classes, the relative difficulty of performance on associated sets of items decreases compared to lower sets of items and lower groups of students. Such an ordering could potentially prove useful in the process of setting standards. Although this model has previously been used primarily for the exploration of developmental differences, this paper will investigate the possibility of using the saltus model as a tool to assist in standard setting. Illustrative analysis will be conducted with data that were used in a recent study of the Bookmark method for standard setting, and results will be compared.

[1]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[2]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .

[3]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[4]  Mark Wilson.  Saltus: A psychometric model of discontinuity in cognitive development. , 1989 .

[5]  Jürgen Rost,et al.  Rasch Models in Latent Classes: An Integration of Two Approaches to Item Analysis , 1990 .

[6]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[7]  Max-Planck-Institut für Bildungsforschung. Projektgruppe Bildungsbericht.  Das Bildungswesen in der Bundesrepublik Deutschland : Strukturen und Entwicklungen im Überblick , 1994 .

[8]  D. M. Lewis,et al.  Standard setting : A bookmark approach , 1996 .

[9]  R. Mislevy,et al.  Marginal maximum likelihood estimation for a psychometric model of discontinuous development , 1996 .

[10]  Sylvia Richardson,et al.  Markov Chain Monte Carlo in Practice , 1997 .

[11]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[12]  Gregory J. Cizek,et al.  Setting performance standards : concepts, methods, and perspectives , 2001 .

[13]  Allan S. Cohen,et al.  A Mixture Item Response Model for Multiple-Choice Data , 2001 .

[14]  Mark Wilson,et al.  Application of the Saltus Model to Stagelike Data: Some Applications and Current Developments , 2007 .

[15]  Mark R. Wilson.  Chapter 8 Mixture Models in a developMental Context , 2007 .

[16]  Olaf Köller,et al.  Calibrating standards-based assessment tasks for English as a first foreign language: standard-setting procedures in Germany. Volume 2. , 2010 .

[17]  Setting standards in line with the common European framework of reference , 2010 .

[18]  K. Draney,et al.  Selecting cut scores with a composite of item types: the construct mapping procedure. , 2011, Journal of applied measurement.