A Bayesian generalized multiple group IRT model with model-fit assessment tools

The multiple group IRT model (MGM) proposed by Bock and Zimowski (1997) provides a useful framework for analyzing item response data from clustered respondents. In the MGM, the selected groups of respondents are of specific interest such that group-specific population distributions need to be defined. The main goal is to explore the potentials of an MCMC estimation procedure and Bayesian model-fit tools for the MGM. We develop a full Gibbs sampling algorithm (FGSA) for estimation as well as a Metropolis-Hastings within Gibss sampling algorithm (MHWGS) in order to use non-conjugate priors. The FGSA is compared with Bilog-MG, which uses marginal maximum likelihood (MML) and marginal maximum a posteriori (MMAP) methods. That is; Bilog-MG provides maximum likelihood (ML) and expected a posteriori (EAP) estimates for both item and population parameters, and maximum a posteriori (MAP) estimates for the latent traits. We conclude that, in general, the results from our approach are slightly better than Bilog-MG. Besides a simultaneous MCMC estimation procedure, model-fit assessment tools are developed. Furthermore, the prior sensitivity is investigated with respect to the parameters of the latent population distributions. It will be shown that the FGSA provides a wide set of model-fit tools. The proposed model assessment tools can evaluate important model assumptions of (1) the item response function (IRF) and (2) the latent trait distributions. The utility of the proposed estimation and model-fit assessment methods will be shown using data from a longitudinal data study concerning first to fourth graders of sampled Brazilian public schools.

[1]  H. Bolfarine,et al.  A skew item response model , 2006 .

[2]  John Geweke,et al.  Efficient Simulation from the Multivariate Normal and Student-t Distributions Subject to Linear Constraints and the Evaluation of Constraint Probabilities , 1991 .

[3]  Bengt Muthén,et al.  Multiple Group IRT Modeling: Applications to Item Bias Analysis , 1985 .

[4]  F. B. Gonçalves,et al.  An Integrated Bayesian Model for DIF Analysis , 2009 .

[5]  J. Fox,et al.  Bayesian estimation of a multilevel IRT model using gibbs sampling , 2001 .

[6]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[7]  J. Albert.  Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling , 1992 .

[8]  H. Stern,et al.  Bayesian Model Checking and Model Diagnostics , 2005 .

[9]  Allan S. Cohen,et al.  Detection of Differential Item Functioning in Multiple Groups. , 1995 .

[10]  Hal S. Stern,et al.  Posterior Predictive Assessment of Item Response Theory Models , 2006 .

[11]  A. Béguin,et al.  MCMC estimation and some model-fit analysis of multidimensional IRT models , 2001 .

[12]  Item Parameter Recovery for the Nominal Response Model , 1999 .

[13]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[14]  D. Andrade,et al.  An estimation method for latent traits and population parameters in Nominal Response Model , 2010 .

[15]  J. Fox.  Bayesian Item Response Modeling: Theory and Applications , 2010 .

[16]  Dani Gamerman,et al.  Markov Chain Monte Carlo: Stochastic Simulation for Bayesian Inference , 1997 .

[17]  Sandip Sinharay.  Bayesian item fit analysis for unidimensional item response theory models. , 2006, The British journal of mathematical and statistical psychology.

[18]  J. Fox,et al.  Bayesian modification indices for IRT models , 2005 .

[19]  Randall D. Penfield.  Assessing Differential Item Functioning Among Multiple Groups: A Comparison of Three Mantel-Haenszel Procedures , 2001 .

[20]  R. Mislevy.  Estimating latent distributions , 1984 .

[21]  Heleno Bolfarine,et al.  Bayesian inference for a skew-normal IRT model under the centred parameterization , 2011, Comput. Stat. Data Anal..

[22]  Robert J. Mislevy.  Bayes modal estimation in item response models , 1986 .

[23]  Cees A. W. Glas,et al.  DETECTION OF DIFFERENTIAL ITEM FUNCTIONING USING LAGRANGE MULTIPLIER TESTS , 1996 .

[24]  C. Q. da-Silva,et al.  Bayesian inference for an item response model for modeling test anxiety , 2011, Comput. Stat. Data Anal..