USING THE GENERAL DIAGNOSTIC MODEL TO MEASURE LEARNING AND CHANGE IN A LONGITUDINAL LARGE-SCALE ASSESSMENT

A general diagnostic model was used to specify and compare two multidimensional item-response-theory (MIRT) models for longitudinal data: (a) a model that handles repeated measurements as multiple, correlated variables over time (Andersen, 1985) and (b) a model that assumes one common variable over time and additional orthogonal variables that quantify the change (Embretson, 1991). Using MIRT-model ability distributions that we allowed to vary across subpopulations defined by type of school, we also compared (a) a model with a single two-dimensional ability distribution to (b) extensions of the Andersen and Embretson approaches, assuming multiple populations. In addition, we specified a hierarchical-mixture distribution variant of the (Andersen and Embretson) MIRT models and compared it to all four of the above alternatives. These four types of models are growth-mixture models that allow for variation of the mixing proportions across clusters in a hierarchically organized sample. To illustrate the models presented in this paper, we applied the models to the PISA-I-Plus data for assessing learning and change across multiple subpopulations. The results indicate that (a) the Embretson-type model with multiple-group assumptions fits the data better than the other models investigated, and (b) the higher performing group shows larger improvement at Time Point 2 than the lower performing group.

[1]  E. Muraki.  A GENERALIZED PARTIAL CREDIT MODEL: APPLICATION OF AN EM ALGORITHM , 1992 .

[2]  Robert J. Mislevy,et al.  Randomization-based inference about latent variables from complex samples , 1991 .

[3]  Mark Wilson,et al.  Application of the Saltus Model to Stagelike Data: Some Applications and Current Developments , 2007 .

[4]  H. Akaike.  A new look at the statistical model identification , 1974 .

[5]  G. H. Fischer,et al.  The linear logistic test model as an instrument in educational research , 1973 .

[6]  G. H. Fischer,et al.  Gain Scores Revisited Under an IRT Perspective , 2001 .

[7]  Matthias von Davier,et al.  HIERARCHICAL GENERAL DIAGNOSTIC MODELS , 2007 .

[8]  D. Andrade,et al.  Item response theory for longitudinal data: population parameter estimation , 2005 .

[9]  Karen Draney,et al.  Objective measurement : theory into practice , 1992 .

[10]  Erling B. Andersen,et al.  Estimating latent correlations between repeated testings , 1985 .

[11]  Anne Boomsma,et al.  Essays on Item Response Theory. Lecture Notes in Statistics, 157 , 2001 .

[12]  Jeroen K. Vermunt,et al.  7. Multilevel Latent Class Models , 2003 .

[13]  Matthias von Davier,et al.  19 Mixture Distribution Item Response Models , 2006 .

[14]  R. Hambleton,et al.  Handbook of Modern Item Response Theory , 1997 .

[15]  Matthias von Davier,et al.  Polytomous Mixed Rasch Models , 1995 .

[16]  Gerhard H. Fischer,et al.  Some neglected problems in IRT , 1995 .

[17]  Matthias von Davier,et al.  A general diagnostic model applied to language testing data. , 2008, The British journal of mathematical and statistical psychology.

[18]  I. W. Molenaar,et al.  Rasch models: foundations, recent developments and applications , 1995 .

[19]  R. Darrell Bock,et al.  Multiple Group IRT , 1997 .

[20]  Jürgen Rost,et al.  Rasch Models in Latent Classes: An Integration of Two Approaches to Item Analysis , 1990 .

[21]  Matthias von Davier,et al.  COGNITIVE DIAGNOSIS FOR NAEP PROFICIENCY DATA , 2006 .

[22]  Matthias von Davier,et al.  Mixture-Distribution and HYBRID Rasch Models , 2007 .

[23]  Matthias von Davier,et al.  Mixture Distribution Rasch Models , 1995 .

[24]  H.L.J. van der Maas,et al.  An IRT Model with a Parameter-Driven Process for Change , 2005 .

[25]  Matthias von Davier,et al.  Cognitive Diagnosis for NAEP Proficiency Data. Research Report. ETS RR-06-08. , 2006 .

[26]  Matthias von Davier,et al.  A GENERAL DIAGNOSTIC MODEL APPLIED TO LANGUAGE TESTING DATA , 2005 .

[27]  G. Masters.  A rasch model for partial credit scoring , 1982 .

[28]  Susan E. Embretson,et al.  A multidimensional latent trait model for measuring learning and change , 1991 .

[29]  Matthias von Davier,et al.  A General Diagnostic Model Applied to Language Testing Data. Research Report. ETS RR-05-16. , 2005 .

[30]  Mark Wilson.  Saltus: A psychometric model of discontinuity in cognitive development. , 1989 .

[31]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[32]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[33]  G. Rudinger,et al.  Analyzing Homogeneity and Heterogeneity of Change Using Rasch and Latent Class Models: A Comparative and Integrative Approach , 1995 .