Bayesian analysis of dynamic item response models in educational testing

Item response theory (IRT) models have been widely used in educational measurement testing. When there are repeated observations available for individuals through time, a dynamic structure for the latent trait of ability needs to be incorporated into the model, to accommodate changes in ability. Other complications that often arise in such settings include a violation of the common assumption that test results are conditionally independent, given ability and item difficulty, and that test item difficulties may be partially specified, but subject to uncertainty. Focusing on time series dichotomous response data, a new class of state space models, called Dynamic Item Response (DIR) models, is proposed. The models can be applied either retrospectively to the full data or on-line, in cases where real-time prediction is needed. The models are studied through simulated examples and applied to a large collection of reading test data obtained from MetaMetrics, Inc.

[1]  Robert J. Jannarone,et al.  Conjunctive item response theory kernels , 1986 .

[2]  L. Fahrmeir.  Posterior Mode Estimation by Extended Kalman Filtering for Multivariate Dynamic Generalized Linear Models , 1992 .

[3]  Paul De Boeck,et al.  Random Item IRT Models , 2008 .

[4]  C. N. Morris,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[5]  Ronald J. M. M. Does,et al.  A stochastic growth model applied to repeated tests of academic knowledge , 1989 .

[6]  Francesco Bartolucci,et al.  Assessment of School Performance Through a Multilevel Latent Markov Rasch Model , 2009, 0909.4961.

[7]  Eric T. Bradlow,et al.  A Bayesian random effects model for testlets , 1999 .

[8]  William Stout,et al.  A nonparametric approach for assessing latent trait unidimensionality , 1987 .

[9]  Susan E. Embretson,et al.  A multidimensional latent trait model for measuring learning and change , 1991 .

[10]  Ronald J. M. M. Does,et al.  Approximations of Normal IRT Models for Change , 1999 .

[11]  E. B. Andersen,et al.  Asymptotic Properties of Conditional Maximum‐Likelihood Estimators , 1970 .

[12]  D. F. Andrews,et al.  Scale Mixtures of Normal Distributions , 1974 .

[13]  Jong Hee Park,et al.  Modeling Preference Changes via a Hidden Markov Item Response Theory Model , 2011 .

[14]  Donald Hedeker,et al.  Full-information item bi-factor analysis , 1992 .

[15]  David M. Williamson,et al.  Calibrating Item Families and Summarizing the Results Using Family Expected Response Functions , 2003 .

[16]  Sonderforschungsbereich.  State Space Mixed Models for Longitudinal Observations with Binary and Binomial Responses , 2007 .

[17]  Cees A. W. Glas,et al.  Application of Multidimensional Item Response Theory Models to Longitudinal Data , 2006 .

[18]  Andrew D. Martin,et al.  Dynamic Ideal Point Estimation via Markov Chain Monte Carlo for the U.S. Supreme Court, 1953–1999 , 2002, Political Analysis.

[19]  William Stout,et al.  A New Item Response Theory Modeling Approach with Applications to Unidimensionality Assessment and Ability Estimation , 1990 .

[20]  G. Rasch.  On General Laws and the Meaning of Measurement in Psychology , 1961 .

[21]  J. Berger.  The case for objective Bayesian analysis , 2006 .

[22]  D. Andrich,et al.  Quantifying Response Dependence Between Two Dichotomous Items Using the Rasch Model , 2010 .

[23]  Xiaojing Wang,et al.  Bayesian Modeling Using Latent Structures , 2012 .

[24]  Frederic M. Lord.  THE RELATION OF TEST SCORE TO THE TRAIT UNDERLYING THE TEST , 1952 .

[25]  R. Darrell Bock,et al.  Fitting a response model forn dichotomously scored items , 1970 .