On Longitudinal Item Response Theory Models: A Didactic

Recent work on measuring growth with categorical outcome variables has combined the item response theory (IRT) measurement model with the latent growth curve model and extended the assessment of growth to multidimensional IRT models and higher order IRT models. However, there is a lack of synthetic studies that clearly evaluate the strength and limitations of different multilevel IRT models for measuring growth. This study aims to introduce the various longitudinal IRT models, including the longitudinal unidimensional IRT model, longitudinal multidimensional IRT model, and longitudinal higher order IRT model, which cover a broad range of applications in education and social science. Following a comparison of the parameterizations, identification constraints, strengths, and weaknesses of the different models, a real data example is provided to illustrate the application of different longitudinal IRT models to model students’ growth trajectories on multiple latent abilities.

[1]  B. Muthén,et al.  Growth mixture modeling with non‐normal distributions , 2015, Statistics in medicine.

[2]  Jan de Leeuw,et al.  On the relationship between item response theory and factor analysis of discretized variables , 1987 .

[3]  Chun Wang,et al.  Improving Measurement Precision of Hierarchical Latent Traits Using Adaptive Testing , 2014 .

[4]  Shameem Nyla.  NATIONAL COUNCIL ON MEASUREMENT IN EDUCATION , 2004 .

[5]  Tomokazu Haebara,et al.  EQUATING LOGISTIC ABILITY SCALES BY A WEIGHTED LEAST SQUARES METHOD , 1980 .

[6]  Kimberly S. Maier,et al.  Using a Multivariate Multilevel Polytomous Item Response Theory Model to Study Parallel Processes of Change: The Dynamic Association Between Adolescents' Social Isolation and Engagement With Delinquent Peers in the National Youth Survey , 2010, Multivariate behavioral research.

[7]  Yasuyo Sawaki,et al.  Factor structure of the TOEFL Internet-based test , 2009 .

[8]  Andrew Thomas,et al.  WinBUGS - A Bayesian modelling framework: Concepts, structure, and extensibility , 2000, Stat. Comput..

[9]  D. Rindskopf,et al.  Some Theory and Applications of Confirmatory Second-Order Factor Analysis. , 1988, Multivariate behavioral research.

[10]  Steven W. Nydick,et al.  Comparing Two Algorithms for Calibrating the Restricted Non-Compensatory Multidimensional IRT Model , 2015, Applied psychological measurement.

[11]  J. Mcardle,et al.  Latent difference score structural models for linear dynamic analyses with incomplete longitudinal data. , 2001 .

[12]  Kenneth A. Bollen,et al.  Latent curve models: A structural equation perspective , 2005 .

[13]  Chun Wang,et al.  Fitting a linear-linear piecewise growth mixture model with unknown knots: A comparison of two common approaches to inference. , 2015, Psychological methods.

[14]  A Multilevel Higher Order Item Response Theory Model for Measuring Latent Growth in Longitudinal Data , 2015, Applied psychological measurement.

[15]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[16]  Jimmy de la Torre,et al.  Simultaneous Estimation of Overall and Domain Abilities: A Higher-Order IRT Model Approach , 2009 .

[17]  David B. Dunson,et al.  Bayesian Structural Equation Modeling , 2007 .

[18]  Kevin J. Grimm,et al.  Modeling life-span growth curves of cognition using longitudinal data with multiple samples and changing scales of measurement. , 2009, Psychological methods.

[19]  T. Lecerf,et al.  Orthogonal higher order structure and confirmatory factor analysis of the French Wechsler Adult Intelligence Scale (WAIS-III). , 2011, Psychological assessment.

[20]  B. Muthén.  Bayesian Analysis In Mplus : A Brief Introduction , 2010 .

[21]  Akihito Kamata,et al.  A Note on the Relation Between Factor Analytic and Item Response Theory Models , 2008 .

[22]  Yves Rosseel,et al.  lavaan: An R Package for Structural Equation Modeling , 2012 .

[23]  S. West,et al.  Teacher's Corner: Testing Measurement Invariance of Second-Order Factor Models , 2005 .

[24]  N. Laird.  Empirical Bayes methods for two-way contingency tables , 1978 .

[25]  J. Mcardle.  Dynamic but Structural Equation Modeling of Repeated Measures Data , 1988 .

[26]  D. Andrade,et al.  Item response theory for longitudinal data: population parameter estimation , 2005 .

[27]  H. Goldstein.  Nonlinear multilevel models, with an application to discrete response data , 1991 .

[28]  L. Cronbach,et al.  How we should measure "change": Or should we? , 1970 .

[29]  Chun Wang,et al.  A mixture hierarchical model for response times and response accuracy. , 2015, The British journal of mathematical and statistical psychology.

[30]  G. Verbeke,et al.  Statistical inference in generalized linear mixed models: a review. , 2006, The British journal of mathematical and statistical psychology.

[31]  M. C. Edwards,et al.  Measurement and the Study of Change , 2009 .

[32]  Steven W. Nydick,et al.  The Sequential Probability Ratio Test and Binary Item Response Models , 2014 .

[33]  Gyenam Kim-Kang,et al.  Adaptive Measurement of Individual Change , 2008 .

[34]  D. Bates,et al.  Linear Mixed-Effects Models using 'Eigen' and S4 , 2015 .

[35]  Erling B. Andersen,et al.  Estimating latent correlations between repeated testings , 1985 .

[36]  S. McKay Curtis,et al.  BUGS Code for Item Response Theory , 2010 .

[37]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[38]  K. Grimm,et al.  Measurement Models, Estimation, and the Study of Change , 2013 .

[39]  Z. Li,et al.  Specifying Ability Growth Models Using a Multidimensional Item Response Model for Repeated Measures Categorical Ordinal Item Response Data , 2016, Multivariate behavioral research.

[40]  Stephen G West,et al.  Testing Measurement Invariance in Longitudinal Data With Ordered-Categorical Measures , 2017, Psychological methods.

[41]  Lihua Yao.  Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores: Theory and Applications , 2012, Psychometrika.

[42]  C. DeYoung.  Higher-order factors of the Big Five in a multi-informant sample. , 2006, Journal of personality and social psychology.

[43]  Lihua Yao.  Multidimensional CAT Item Selection Methods for Domain Scores and Composite Scores With Item Exposure Control and Content Constraints , 2014 .

[44]  Hung-Yu Huang,et al.  Multilevel Higher-Order Item Response Theory Models , 2014 .

[45]  S. Reise,et al.  Exploring the measurement invariance of psychological instruments: Applications in the substance use domain. , 1997 .

[46]  Aja Louise Murray,et al.  The limitations of model fit in comparing the bi-factor versus higher-order models of human cognitive ability structure , 2013 .

[47]  Li Cai,et al.  Generalized full-information item bifactor analysis. , 2011, Psychological methods.

[48]  M. Reckase.  Multidimensional Item Response Theory , 2009 .

[49]  Paul De Boeck,et al.  Latent Class Models for Diary Method Data: Parameter Estimation by Local Computations , 2007, Psychometrika.

[50]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[51]  Christopher K. Wikle,et al.  Bayesian Multidimensional IRT Models With a Hierarchical Structure , 2008 .

[52]  Noreen Goldman,et al.  An assessment of estimation procedures for multilevel models with binary responses , 1995 .

[53]  Bradley P. Carlin,et al.  Bayesian measures of model complexity and fit , 2002 .

[54]  Li Cai,et al.  A Two-Tier Full-Information Item Factor Analysis Model with Applications , 2010 .

[55]  J. Gustafsson,et al.  General and Specific Abilities as Predictors of School Achievement. , 1993, Multivariate behavioral research.

[56]  Xiao-Li Meng,et al.  The EM Algorithm—an Old Folk‐song Sung to a Fast New Tune , 1997 .

[57]  Harvey Goldstein,et al.  Improved Approximations for Multilevel Models with Binary Responses , 1996 .

[58]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[59]  Eric R. Ziegel,et al.  Generalized Linear Models , 2002, Technometrics.

[60]  E. Ferrer,et al.  Factorial Invariance within Longitudinal Structural Equation Models: Measuring the Same Construct across Time. , 2010, Child development perspectives.

[61]  N. Kohli,et al.  A Second-Order Longitudinal Model for Binary Outcomes: Item Response Theory Versus Structural Equation Modeling , 2016 .

[62]  Mark Wilson,et al.  Formulating latent growth using an explanatory item response model approach. , 2012, Journal of applied measurement.

[63]  Alberto Maydeu-Olivares,et al.  Estimation of IRT graded response models: limited versus full information methods. , 2009, Psychological methods.

[64]  Susan E. Embretson,et al.  A multidimensional latent trait model for measuring learning and change , 1991 .

[65]  Martha L. Stocking,et al.  Developing a Common Metric in Item Response Theory , 1982 .

[66]  Matthias von Davier,et al.  Measuring Growth in a Longitudinal Large-Scale Assessment with a General Latent Variable Model , 2011 .

[67]  Thomas R. Boucher,et al.  Test Equating, Scaling, and Linking: Methods and Practices , 2007 .

[68]  William Meredith,et al.  Latent curve analysis , 1990 .

[69]  N. Breslow,et al.  Approximate inference in generalized linear mixed models , 1993 .

[70]  Jimmy de la Torre,et al.  Parameter Estimation With Small Sample Size A Higher-Order IRT Model Approach , 2010 .

[71]  R Core Team,et al.  R: A language and environment for statistical computing. , 2014 .

[72]  Emilio Ferrer,et al.  Curve of Factors Model: A Latent Growth Modeling Approach for Educational Research , 2018, Educational and psychological measurement.

[73]  Xue Zhang,et al.  Bayesian Model Selection Methods for Multilevel IRT Models: A Comparison of Five DIC‐Based Indices , 2019, Journal of Educational Measurement.