A general Bayesian multilevel multidimensional IRT model for locally dependent data

Many item response theory (IRT) models take a multidimensional perspective to deal with sources that induce local item dependence (LID), with these models often making an orthogonal assumption about the dimensional structure of the data. One reason for this assumption is because of the indeterminacy issue in estimating the correlations among the dimensions in structures often specified to deal with sources of LID (e.g., bifactor and two-tier structures), and the assumption usually goes untested. Unfortunately, the mere fact that assessing these correlations is a challenge for some estimation methods does not mean that data seen in practice support such orthogonal structure. In this paper, a Bayesian multilevel multidimensional IRT model for locally dependent data is presented. This model can test whether item response data violate the orthogonal assumption that many IRT models make about the dimensional structure of the data when addressing sources of LID, and this test is carried out at the dimensional level while accounting for sampling clusters. Simulations show that the model presented is effective at carrying out this task. The utility of the model is also illustrated on an empirical data set.

[1]  K. Holzinger,et al.  The Bi-factor method , 1937 .

[2]  Jean-Paul Fox,et al.  Bayesian Item Response Modeling , 2010 .

[3]  Akihito Kamata,et al.  Modeling local item dependence with the hierarchical generalized linear model. , 2005, Journal of applied measurement.

[4]  A. Pettitt,et al.  Introduction to MCMC , 2012 .

[5]  Michael C Neale,et al.  People are variables too: multilevel structural equations modeling. , 2005, Psychological methods.

[6]  Yanyan Sheng,et al.  BAYESIAN IRT MODELS INCORPORATING GENERAL AND SPECIFIC ABILITIES , 2009 .

[7]  Ken A. Fujimoto,et al.  Acculturative family distancing: psychometric analysis with the extended two-tier item response theory. , 2014, Psychological assessment.

[8]  Donald Hedeker,et al.  Full-Information Item Bifactor Analysis of Graded Response Data , 2007 .

[9]  Eric T. Bradlow,et al.  A Bayesian random effects model for testlets , 1999 .

[10]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[11]  Bengt Muthén,et al.  Multilevel Factor Analysis of Class and Student Achievement Components , 1991 .

[12]  S. West,et al.  A Comparison of Bifactor and Second-Order Models of Quality of Life , 2006, Multivariate behavioral research.

[13]  M. Reckase Multidimensional Item Response Theory , 2009 .

[14]  D. Thissen,et al.  Local Dependence Indexes for Item Pairs Using Item Response Theory , 1997 .

[15]  Akihito Kamata,et al.  A Bifactor Multidimensional Item Response Theory Model for Differential Item Functioning Analysis on Testlet-Based Items , 2011 .

[16]  Raymond J. Adams,et al.  Rasch models for item bundles , 1995 .

[17]  Terry E. Duncan,et al.  Multilevel Covariance Structure Analysis of Sibling Antisocial Behavior. , 1998 .

[18]  Akihito Kamata,et al.  Item Analysis by the Hierarchical Generalized Linear Model. , 2001 .

[19]  Peter Congdon Bayesian statistical modelling , 2002 .

[20]  Sun-Joo Cho,et al.  Multilevel multidimensional item response model with a multilevel latent covariate. , 2015, The British journal of mathematical and statistical psychology.

[21]  P. Hewson Bayesian Data Analysis 3rd edn A. Gelman, J. B. Carlin, H. S. Stern, D. B. Dunson, A. Vehtari and D. B. Rubin, 2013 Boca Raton, Chapman and Hall–CRC 676 pp., £44.99 ISBN 1‐439‐84095‐4 , 2015 .

[22]  J. Fox,et al.  Bayesian estimation of a multilevel IRT model using gibbs sampling , 2001 .

[23]  E. Muraki A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[24]  Jean-Paul Fox,et al.  Using Item Response Theory to Measure Extreme Response Style in Marketing Research: A Global Investigation , 2008 .

[25]  Frank Rijmen,et al.  Efficient Full Information Maximum Likelihood Estimation for Multidimensional IRT Models. Research Report. ETS RR-09-03. , 2009 .

[26]  Alan E. Gelfand,et al.  Model Determination using sampling-based methods , 1996 .

[27]  L. Pettit The Conditional Predictive Ordinate for the Normal Distribution , 1990 .

[28]  S. Geisser,et al.  A Predictive Approach to Model Selection , 1979 .

[29]  J. Fox Bayesian Item Response Modeling: Theory and Applications , 2010 .

[30]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[31]  R. Rumberger Hierarchical linear models: Applications and data analysis methods: and. Newbury Park, CA: Sage, 1992. (ISBN 0-8039-4627-9), pp. xvi + 265. Price: U.S. $45.00 (cloth) , 1997 .

[32]  S. Chib,et al.  Analysis of multivariate probit models , 1998 .

[33]  Gareth O. Roberts,et al.  Examples of Adaptive MCMC , 2009 .

[34]  R. H. Klein Entink,et al.  A Multivariate Multilevel Approach to the Modeling of Accuracy and Speed of Test Takers , 2008, Psychometrika.

[35]  M. Reckase,et al.  Development and Application of a Multivariate Logistic Latent Trait Model , 1972 .

[36]  J-P Fox,et al.  Multilevel IRT using dichotomous and polytomous response data. , 2005, The British journal of mathematical and statistical psychology.

[37]  Y. Zhang,et al.  Polytomous multilevel testlet models for testlet-based assessments with complex sampling designs. , 2015, The British journal of mathematical and statistical psychology.

[38]  Li Cai,et al.  A Two-Tier Full-Information Item Factor Analysis Model with Applications , 2010 .

[39]  Adrian E. Raftery,et al.  Bayes factors and model uncertainty , 1995 .

[40]  Eric T. Bradlow,et al.  Testlet Response Theory and Its Applications , 2007 .

[41]  Kimberly S. Maier A Rasch Hierarchical Measurement Model , 2001 .

[42]  J. Wood,et al.  Acculturative family distancing (AFD) and depression in Chinese American families. , 2010, Journal of consulting and clinical psychology.

[43]  Wen-Chung Wang,et al.  The Rasch Testlet Model , 2005 .

[44]  Akihito Kamata,et al.  A Multilevel Testlet Model for Dual Local Dependence , 2012 .