Exploring levels of performance using the mixture Rasch model for standard setting 1

This study proposes a method using the mixture Rasch model to set performance standards. The mixture Rasch model classifies examinees into qualitatively distinct latent groups based on the information in the response patterns and quantifies individual differences within-group utilizing a continuous latent trait.The goals of this study are twofold. One is to demonstrate an application of the mixture Rasch model to statistically validate the performance proficiency levels set by policy makers and reviewed by content experts. The other is to demonstrate how performance cut scores can be obtained based on the results from data analysis using the mixture Rasch model. In general, this study presents a mixture Rasch model-based approach for setting performance standards which is expected to facilitate the standard setting process by providing the data driven information related to the policy specified performance levels and the performance cut scores set by the panelists.

[1]  K. T. Taube.  The Incorporation of Empirical Item Difficulty Data into the Angoff Standard-Setting Procedure , 1997, Evaluation & the health professions.

[2]  H. Everson,et al.  MODELING THE MIXTURE OF IRT AND PATTERN RESPONSES BY A MODIFIED HYBRID MODEL1 , 1995 .

[3]  Placy Rj,et al.  Setting performance standards. , 1976 .

[4]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[5]  F. Robin,et al.  Using cluster analysis to facilitate standard setting. , 1999 .

[6]  Identifiers California,et al.  Annual Meeting of the National Council on Measurement in Education , 1998 .

[7]  Kentaro Yamamoto,et al.  A HYBRID Model for Test Speededness , 2007 .

[8]  Stuart R. Kahl.  Using Actual Student Work To Determine Cut Scores for Proficiency Levels: New Methods for New Tests. , 1994 .

[9]  D. M. Lewis,et al.  Standard setting : A bookmark approach , 1996 .

[10]  Robert J. Mislevy,et al.  Modeling item responses when different subjects employ different solution strategies , 1990 .

[11]  Allan S. Cohen,et al.  Model Selection Methods for Mixture Dichotomous IRT Models , 2009 .

[12]  Barbara S. Plake,et al.  Standard Setting: An Alternative Approach , 1997 .

[13]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[14]  Ronald K. Hambleton,et al.  Using an Extended Angoff Procedure To Set Standards on Complex Performance Assessments. , 1995 .

[15]  R. Hambleton.  Latent Ability Scales: Interpretations and Uses. , 1980 .

[16]  Allan S. Cohen,et al.  Item Parameter Estimation Under Conditions of Test Speededness: Application of a Mixture Rasch Model With Ordinal Constraints , 2002 .

[17]  Kentaro Yamamoto,et al.  HYBRID MODEL OF IRT AND LATENT CLASS MODELS , 1982 .

[18]  S. Sireci.  Using Cluster Analysis To Solve the Problem of Standard Setting. , 1995 .

[19]  Ronald K. Hambleton,et al.  Competency Test Development, Validation, and Standard-Setting. , 1978 .

[20]  H. Akaike.  Factor analysis and AIC , 1987 .

[21]  Kentaro Yamamoto,et al.  ESTIMATING THE EFFECTS OF TEST LENGTH AND TEST TIME ON PARAMETER ESTIMATION USING THE HYBRID MODEL , 1995 .

[22]  G. Cizek,et al.  Setting performance standards : foundations, methods, and innovations , 2012 .

[23]  Samuel A. Livingston,et al.  Passing Scores: A Manual for Setting Standards of Performance on Educational and Occupational Tests. , 1982 .

[24]  R. L. Ebel,et al.  Essentials of educational measurement , 1972 .

[25]  Jürgen Rost,et al.  Rasch Models in Latent Classes: An Integration of Two Approaches to Item Analysis , 1990 .

[26]  Allan S. Cohen,et al.  A Mixture Item Response Model for Multiple-Choice Data , 2001 .

[27]  George B. Macready,et al.  The Use of Loglinear Models for Assessing Differential Item Functioning Across Manifest and Latent Examinee Groups , 1990 .

[28]  Stuart R. Kahl.  Setting Standards for Performance Levels Using the Student-Based Constructed-Response Method. , 1995 .

[29]  Richard S. Brown.  Using Latent Class Analysis to Set Academic Performance Standards , 2007 .

[30]  Samuel A. Livingston,et al.  Basic Skills Assessment. Manual for Setting Standards on the Basic Skills Assessment Tests. , 1977 .