Model Selection Methods for Mixture Dichotomous IRT Models

This study examines model selection indices for use with dichotomous mixture item response theory (IRT) models. Five indices are considered: Akaike's information coefficient (AIC), Bayesian information coefficient (BIC), deviance information coefficient (DIC), pseudo-Bayes factor (PsBF), and posterior predictive model checks (PPMC). The five indices provide somewhat different recommendations for a set of real data. Results from a simulation study indicate that BIC selects the correct (i.e., the generating) model well under most conditions simulated and for all three of the dichotomous mixture IRT models considered. PsBF is almost as effective. AIC and PPMC tend to select the more complex model under some conditions. DIC is least effective for this use.

[1]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[2]  Luc BAUWENS,et al.  Bayesian Methods , 2011 .

[3]  Sun-Joo Cho.  A multilevel mixture IRT model for DIF analysis , 2007 .

[4]  Allan S. Cohen,et al.  IRT Model Selection Methods for Dichotomous Items , 2007 .

[5]  Hal S. Stern,et al.  Posterior Predictive Assessment of Item Response Theory Models , 2006 .

[6]  Daniel M. Bolt,et al.  A Comparison of Alternative Models for Testlets , 2006 .

[7]  A. Cohen,et al.  The Role of Extended Time and Item Content on a High–Stakes Mathematics Test , 2005 .

[8]  Allan S. Cohen,et al.  A Mixture Model Analysis of Differential Item Functioning , 2005 .

[9]  Frank B. Baker,et al.  Item Response Theory : Parameter Estimation Techniques, Second Edition , 2004 .

[10]  M. Forster.  Chapter 3: Simplicity and Unification in Model Selection , 2004 .

[11]  Allan S. Cohen,et al.  A Method for Maintaining Scale Stability in the Presence of Test Speededness , 2003 .

[12]  Peter Congdon,et al.  Applied Bayesian Modelling , 2003 .

[13]  Allan S. Cohen,et al.  Item Parameter Estimation Under Conditions of Test Speededness: Application of a Mixture Rasch Model With Ordinal Constraints , 2002 .

[14]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[15]  Jun Yu,et al.  Deviance Information Criterion for Comparing Stochastic Volatility Models , 2002 .

[16]  S. Sahu.  Bayesian Estimation and Model Choice in Item Response Models , 2002 .

[17]  Jeff Gill,et al.  What are Bayesian Methods , 2008 .

[18]  Allan S. Cohen,et al.  A Mixture Item Response Model for Multiple-Choice Data , 2001 .

[19]  Bruce Western,et al.  Bayesian Analysis for Sociologists , 1999 .

[20]  P. Boeck,et al.  Confirmatory Analyses of Componential Test Structure Using Multidimensional Item Response Theory. , 1999, Multivariate behavioral research.

[21]  Identifiers California,et al.  Annual Meeting of the National Council on Measurement in Education , 1998 .

[22]  C. Mitchell Dayton,et al.  Model Selection Information Criteria for Non-Nested Latent Class Models , 1997 .

[23]  L. Wasserman,et al.  Practical Bayesian Density Estimation Using Mixtures of Normals , 1997 .

[24]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[25]  A. Gelfand,et al.  Bayesian Model Choice: Asymptotics and Exact Calculations , 1994 .

[26]  F. Baker,et al.  Item response theory : parameter estimation techniques , 1993 .

[27]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[28]  B. Leroux.  Consistent estimation of a mixing distribution , 1992 .

[29]  R. Hambleton,et al.  Fundamentals of Item Response Theory , 1991 .

[30]  Jürgen Rost,et al.  Rasch Models in Latent Classes: An Integration of Two Approaches to Item Analysis , 1990 .

[31]  Robert J. Mislevy,et al.  Modeling item responses when different subjects employ different solution strategies , 1990 .

[32]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[33]  Brenda H. Loyd,et al.  VERTICAL EQUATING USING THE RASCH MODEL , 1980 .

[34]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[35]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[36]  Gary L. Marco,et al.  Item characteristic curve solutions to three intractable testing problems. , 1977 .

[37]  RELATIVE EFFICIENCY OF NUMBER‐RIGHT AND FORMULA SCORES , 1975 .

[38]  H. Akaike.  A new look at the statistical model identification , 1974 .

[39]  E. B. Andersen,et al.  A goodness of fit test for the rasch model , 1973 .