The Bayesian Multilevel Trifactor Item Response Theory Model

Advancements in item response theory (IRT) have led to models for dual dependence, which control for cluster and method effects during a psychometric analysis. Currently, however, this class of models does not include one that controls for when the method effects stem from two method sources in which one source functions differently across the aspects of another source (i.e., a nested method–source interaction). For this study, then, a Bayesian IRT model is proposed, one that accounts for such interaction among method sources while controlling for the clustering of individuals within the sample. The proposed model accomplishes these tasks by specifying a multilevel trifactor structure for the latent trait space. Details of simulations are also reported. These simulations demonstrate that this model can identify when item response data represent a multilevel trifactor structure, and it does so in data from samples as small as 250 cases nested within 50 clusters. Additionally, the simulations show that misleading estimates for the item discriminations could arise when the trifactor structure reflected in the data is not correctly accounted for. The utility of the model is also illustrated through the analysis of empirical data.

[1]  E. Muraki.  A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[2]  Robert J. Sampson,et al.  6. A Multivariate, Multilevel Rasch Model with Application to Self-Reported Criminal Behavior , 2003 .

[3]  A. Gelman.  Prior distributions for variance parameters in hierarchical models (comment on article by Browne and Draper) , 2004 .

[4]  Frank Rijmen,et al.  Hierarchical factor item response theory models for PIRLS: capturing clustering effects at multiple levels , 2011 .

[5]  Andrew Gelman,et al.  Handbook of Markov Chain Monte Carlo , 2011 .

[6]  Sun-Joo Cho,et al.  Detecting Differential Item Discrimination (DID) and the Consequences of Ignoring DID in Multilevel Item Response Models. , 2017 .

[7]  D. Eignor.  The standards for educational and psychological testing. , 2013 .

[8]  D. Thissen,et al.  Local Dependence Indexes for Item Pairs Using Item Response Theory , 1997 .

[9]  Jean-Paul Fox,et al.  Relaxing Measurement Invariance in Cross-National Consumer Research Using a Hierarchical IRT Model , 2007 .

[10]  Y. Zhang,et al.  Polytomous multilevel testlet models for testlet-based assessments with complex sampling designs. , 2015, The British journal of mathematical and statistical psychology.

[11]  Akihito Kamata,et al.  A Multilevel Testlet Model for Dual Local Dependence , 2012 .

[12]  Raymond J. Adams,et al.  Rasch models for item bundles , 1995 .

[13]  Wen-Chung Wang,et al.  Item Response Theory Models for Wording Effects in Mixed-Format Scales , 2015, Educational and psychological measurement.

[14]  A. Pettitt,et al.  Introduction to MCMC , 2012 .

[15]  Kimberly S. Maier.  A Rasch Hierarchical Measurement Model , 2001 .

[16]  J. Wood,et al.  Acculturative family distancing (AFD) and depression in Chinese American families. , 2010, Journal of consulting and clinical psychology.

[17]  W. Lijun,et al.  The Multidimensional Testlet-Effect Rasch Model , 2014 .

[18]  Anthony S. Bryk,et al.  Hierarchical Linear Models: Applications and Data Analysis Methods , 1992 .

[19]  Ken A. Fujimoto,et al.  A general Bayesian multilevel multidimensional IRT model for locally dependent data , 2018, The British journal of mathematical and statistical psychology.

[20]  B. Muthén,et al.  Multilevel Covariance Structure Analysis , 1994 .

[21]  M. Zuckerman,et al.  Attribution of success and failure revisited, or: The motivational bias is alive and well in attribution theory , 1979 .

[22]  Daniel O. Segall,et al.  General ability measurement: An application of multidimensional item response theory , 2001 .

[23]  Adrian E. Raftery,et al.  Bayes factors and model uncertainty , 1995 .

[24]  S. West,et al.  A Comparison of Bifactor and Second-Order Models of Quality of Life , 2006, Multivariate behavioral research.

[25]  R. Dedrick,et al.  A Multilevel Bifactor Approach to Construct Validation of Mixed-Format Scales , 2018, Educational and psychological measurement.

[26]  J-P Fox,et al.  Multilevel IRT using dichotomous and polytomous response data. , 2005, The British journal of mathematical and statistical psychology.

[27]  Simon Jackman,et al.  Bayesian Analysis for the Social Sciences , 2009 .

[28]  Alan E. Gelfand,et al.  Model Determination using sampling-based methods , 1996 .

[29]  Ruth E Baldasaro,et al.  A trifactor model for integrating ratings across multiple informants. , 2013, Psychological methods.

[30]  Michael C Neale,et al.  People are variables too: multilevel structural equations modeling. , 2005, Psychological methods.

[31]  Bengt Muthén,et al.  Multilevel Factor Analysis of Class and Student Achievement Components , 1991 .

[32]  Li Cai,et al.  A Two-Tier Full-Information Item Factor Analysis Model with Applications , 2010 .

[33]  P. Gustafson,et al.  Conservative prior distributions for variance parameters in hierarchical models , 2006 .

[34]  J. Fox,et al.  Bayesian estimation of a multilevel IRT model using gibbs sampling , 2001 .

[35]  Akihito Kamata,et al.  Modeling local item dependence with the hierarchical generalized linear model. , 2005, Journal of applied measurement.

[36]  Akihito Kamata,et al.  Item Analysis by the Hierarchical Generalized Linear Model. , 2001 .

[37]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .