The Impact of Markov Chain Convergence on Estimation of Mixture IRT Model Parameters

A nonconverged Markov chain can potentially lead to invalid inferences about model parameters. The purpose of this study was to assess the effect of a nonconverged Markov chain on the estimation of parameters for mixture item response theory models using a Markov chain Monte Carlo algorithm. A simulation study was conducted to investigate the accuracy of model parameters estimated with different degree of convergence. Results indicated the accuracy of the estimated model parameters for the mixture item response theory models decreased as the number of iterations of the Markov chain decreased. In particular, increasing the number of burn-in iterations resulted in more accurate estimation of mixture IRT model parameters. In addition, the different methods for monitoring convergence of a Markov chain resulted in different degrees of convergence despite almost identical accuracy of estimation.

[1]  Fritz Drasgow,et al.  Recovery of Two- and Three-Parameter Logistic Item Characteristic Curves: A Monte Carlo Study , 1982 .

[2]  K. Sijtsma,et al.  The effect of differential motivation on IRT linking , 2015 .

[3]  Andrew B. Lawson,et al.  Bayesian Biostatistics: Lesaffre/Bayesian Biostatistics , 2012 .

[4]  Suhartono,et al.  On The Markov Chain Monte Carlo Convergence Diagnostic of Bayesian Finite Mixture Model for Income Distribution , 2018, Journal of Physics: Conference Series.

[5]  Allan S. Cohen,et al.  A Mixture Model Analysis of Differential Item Functioning , 2005 .

[6]  Philip Heidelberger,et al.  Simulation Run Length Control in the Presence of an Initial Transient , 1983, Oper. Res..

[7]  Clement A. Stone,et al.  Recovery of Marginal Maximum Likelihood Estimates in the Two-Parameter Logistic Response Model: An Evaluation of MULTILOG , 1992 .

[8]  F. Bartolucci,et al.  Evaluation of Student Performance through a Multidimensional Finite Mixture IRT Model , 2017, Multivariate behavioral research.

[9]  Allan S. Cohen,et al.  Item Parameter Estimation Under Conditions of Test Speededness: Application of a Mixture Rasch Model With Ordinal Constraints , 2002 .

[10]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[11]  Seock-Ho Kim,et al.  An Evaluation of a Markov Chain Monte Carlo Method for the Rasch Model , 2001 .

[12]  Wen-Chung Wang,et al.  Mixture Item Response Models for Inattentive Responding Behavior , 2018 .

[13]  Seock-Ho Kim,et al.  The Impact of Non-Normality on Extraction of Spurious Latent Classes in Mixture IRT Models , 2016, Applied psychological measurement.

[14]  Sandip Sinharay,et al.  Experiences With Markov Chain Monte Carlo Convergence Assessment in Two Psychometric Examples , 2004 .

[15]  G. Macready,et al.  A Mixture Rasch Model–Based Computerized Adaptive Test for Latent Class Identification , 2012 .

[16]  Bradley P. Carlin,et al.  Markov Chain Monte Carlo conver-gence diagnostics: a comparative review , 1996 .

[17]  Allan S. Cohen,et al.  Model Selection Methods for Mixture Dichotomous IRT Models , 2009 .

[18]  Daniel M. Bolt,et al.  Estimating Item Response Theory Models Using Markov Chain Monte Carlo Methods , 2007 .

[19]  O. Lüdtke,et al.  Performance decline in low-stakes educational assessments: different mixture modeling approaches , 2017 .

[20]  Gareth O. Roberts,et al.  Convergence assessment techniques for Markov chain Monte Carlo , 1998, Stat. Comput..

[21]  Jimmy de la Torre,et al.  Parameter Estimation With Small Sample Size A Higher-Order IRT Model Approach , 2010 .

[22]  Brian J. Smith,et al.  boa: An R Package for MCMC Output Convergence Assessment and Posterior Inference , 2007 .

[23]  Kuan Yu Jin,et al.  Item Response Theory Models for Performance Decline during Testing. , 2014 .

[24]  Jürgen Rost,et al.  Rasch Models in Latent Classes: An Integration of Two Approaches to Item Analysis , 1990 .

[25]  Charles J. Geyer,et al.  Practical Markov Chain Monte Carlo , 1992 .

[26]  Hung-Yu Huang,et al.  Mixture IRT Model With a Higher-Order Structure for Latent Traits , 2017, Educational and psychological measurement.

[27]  Hung-Yu Huang.  Mixture Random-Effect IRT Models for Controlling Extreme Response Style on Rating Scales , 2016, Front. Psychol..

[28]  Adrian E. Raftery,et al.  [Practical Markov Chain Monte Carlo]: Comment: One Long Run with Diagnostics: Implementation Strategies for Markov Chain Monte Carlo , 1992 .

[29]  Prathiba Natesan,et al.  Recovery of Graded Response Model Parameters , 2012 .

[30]  W. Holmes Finch,et al.  Parameter Estimation with Mixture Item Response Theory Models: A Monte Carlo Comparison of Maximum Likelihood and Bayesian Methods , 2012 .