An Aggregate IRT Procedure for Exploratory Factor Analysis

An aggregation strategy is proposed to potentially address practical limitation related to computing resources for two-level multidimensional item response theory (MIRT) models with large data sets. The aggregate model is derived by integration of the normal ogive model, and an adaptation of the stochastic approximation expectation maximization algorithm is used for estimation. This methodology is used to conduct an exploratory factor analysis of the 2007 mathematics data from Trends in International Mathematics and Science Study (TIMSS) fourth grade to illustrate potential uses. A comparison to flexMIRT and two brief simulations indicate the aggregate model provides accurate estimates of Level 2 parameters despite loss of information ensuing from key assumption.

[1]  R. Hambleton,et al.  Item Response Theory , 1984, The History of Educational Measurement.

[2]  J. Heckman,et al.  Bayesian Exploratory Factor Analysis , 2014, Journal of econometrics.

[3]  Leslie Rutkowski,et al.  Handbook of International Large-Scale Assessment : Background, Technical Issues, and Methods of Data Analysis , 2013 .

[4]  S. Rabe-Hesketh,et al.  Generalized multilevel structural equation modeling , 2004 .

[5]  J-P Fox.  Stochastic EM for estimating the parameters of a multilevel IRT model. , 2003, The British journal of mathematical and statistical psychology.

[6]  I. Mullis,et al.  TIMSS 2011 International Results in Mathematics. , 2012 .

[7]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[8]  Clement A. Stone,et al.  Providing Subscale Scores for Diagnostic Information: A Case Study When the Test is Essentially Unidimensional , 2009 .

[9]  Bengt Muthén,et al.  Multilevel Factor Analysis of Class and Student Achievement Components , 1991 .

[10]  Li Cai,et al.  Generalized full-information item bifactor analysis. , 2011, Psychological methods.

[11]  M. Reckase.  Multidimensional Item Response Theory , 2009 .

[12]  J. D. L. Torre,et al.  Summarizing Item Difficulty Variation with Parcel Scores. , 2008 .

[13]  R. D. Bock,et al.  The Next Stage in Educational Assessment , 1982 .

[14]  Matthew S. Johnson,et al.  A BAYESIAN HIERARCHICAL MODEL FOR LARGE-SCALE EDUCATIONAL SURVEYS: AN APPLICATION TO THE NATIONAL ASSESSMENT OF EDUCATIONAL PROGRESS , 2004 .

[15]  Trevor Williams,et al.  TIMSS 2007 U.S. Technical Report and User Guide , 2009 .

[16]  B. Muthén,et al.  Exploratory Structural Equation Modeling , 2009 .

[17]  Robert J. Mislevy,et al.  Item Response Models for Grouped Data , 1983 .

[18]  Robert C MacCallum,et al.  Recovery of Weak Common Factors by Maximum Likelihood and Ordinary Least Squares Estimation , 2003, Multivariate behavioral research.

[19]  F. Kong,et al.  A stochastic approximation algorithm with Markov chain Monte-carlo method for incomplete data estimation problems. , 1998, Proceedings of the National Academy of Sciences of the United States of America.

[20]  Guangjian Zhang,et al.  Estimating Standard Errors in Exploratory Factor Analysis , 2014, Multivariate behavioral research.

[21]  Carmen Ximénez,et al.  Recovery of weak factor loadings in confirmatory factor analysis under conditions of model misspecification , 2009, Behavior research methods.

[22]  R. D. Bock,et al.  An Item Response Curve Model for Matrix-Sampling Data: The California Grade-Three Assessment. , 1981 .

[23]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[24]  Frank Rijmen,et al.  Formal Relations and an Empirical Comparison among the Bi‐Factor, the Testlet, and a Second‐Order Multidimensional IRT Model , 2010 .

[25]  Karen B Givvin,et al.  Mathematics Teaching in the United States Today (and Tomorrow): Results from the TIMSS 1999 Video Study , 2006 .

[26]  Mark D. Reckase,et al.  Multidimensional Item Response Theory Models , 2009 .

[27]  É. Moulines,et al.  Convergence of a stochastic approximation version of the EM algorithm , 1999 .

[28]  France Mentré,et al.  Implementation and Evaluation of the SAEM Algorithm for Longitudinal Ordered Categorical Data with an Illustration in Pharmacokinetics–Pharmacodynamics , 2010, The AAPS Journal.

[29]  Tihomir Asparouhov,et al.  Item Response Modeling in Mplus: A Multi-Dimensional, Multi-Level, and Multi-Timepoint Example , 2013 .

[30]  S. Rabe-Hesketh,et al.  Maximum likelihood estimation of limited and discrete dependent variable models with nested random effects , 2005 .

[31]  A. Béguin,et al.  MCMC estimation and some model-fit analysis of multidimensional IRT models , 2001 .

[32]  Minjeong Jeon,et al.  Fitting an item response theory model with random item effects across groups by a variational approximation method , 2013, Ann. Oper. Res..

[33]  Li Cai,et al.  HIGH-DIMENSIONAL EXPLORATORY ITEM FACTOR ANALYSIS BY A METROPOLIS–HASTINGS ROBBINS–MONRO ALGORITHM , 2010 .

[34]  Xiao-Li Meng,et al.  Fitting Full-Information Item Factor Models and an Empirical Investigation of Bridge Sampling , 1996 .

[35]  Albert Maydeu-Olivares,et al.  Multidimensional Item Response Theory Modeling of Binary Data: Large Sample Properties of NOHARM Estimates , 2001 .

[36]  G. Celeux,et al.  Stochastic versions of the em algorithm: an experimental study in the mixture case , 1996 .

[37]  Daniel M. Bolt,et al.  Hierarchical Item Response Theory Modeling , 2014 .

[38]  J. Albert.  Bayesian Estimation of Normal Ogive Item Response Curves Using Gibbs Sampling , 1992 .

[39]  Identifiers California,et al.  Annual Meeting of the National Council on Measurement in Education , 1998 .

[40]  Dorothy T. Thayer,et al.  EM algorithms for ML factor analysis , 1982 .

[41]  K. Wisnicki,et al.  MULTILEVEL FACTOR ANALYSIS OF CLASS AND STUDENT ACHIEVEMENT COMPONENTS CSE Technical Report , 2007 .

[42]  Robert I. Jennrich,et al.  Gradient Projection Algorithms and Software for Arbitrary Rotation Criteria in Factor Analysis , 2005 .

[43]  Karen B. Givvin,et al.  Mathematics Teaching in the United States Today (and Tomorrow): Results From the TIMSS 1999 Video Study , 2005 .

[44]  Jean-Paul Fox,et al.  Bayesian modeling of measurement error in predictor variables using item response theory , 2003 .

[45]  G. Celeux,et al.  A stochastic approximation type EM algorithm for the mixture problem , 1992 .

[46]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters , 1982 .

[47]  H. Robbins.  A Stochastic Approximation Method , 1951 .

[48]  R. D. Bock,et al.  Marginal maximum likelihood estimation of item parameters: Application of an EM algorithm , 1981 .

[49]  Robert J. Mislevy,et al.  A HIERARCHICAL ITEM RESPONSE MODEL FOR EDUCATIONAL TESTING , 1989 .