FITTING THE STRUCTURED GENERAL DIAGNOSTIC MODEL TO NAEP DATA

Xu and von Davier (2006) demonstrated the feasibility of using the general diagnostic model (GDM) to analyze National Assessment of Educational Progress (NAEP) proficiency data. Their work showed that the GDM analysis not only led to conclusions for gender and race groups similar to those published in the NAEP Report Card, but also allowed flexibility in estimating multidimensional skills simultaneously. However, Xu and von Davier noticed that estimating the latent skill distributions will be much more challenging with this model when there is a large number of subgroups to estimate. To make the GDM more applicable to NAEP data analysis, which requires a fairly large subgroups analysis, this study developed a log-linear model to reduce the number of parameters in the latent skill distribution without sacrificing the accuracy of inferences. This paper describes such a model and applies the model in the analysis of NAEP reading assessments for 2003 and 2005. The comparisons between using this model and the unstructured model were made through the use of various results, such as the differences between item parameter estimates and the differences between estimated latent class distributions. The results in general show that using the log-linear model is efficient.

[1]  Joseph B. Lang,et al.  On the comparison of multinomial and Poisson log-linear models , 1996 .

[2]  Paul F. Lazarsfeld,et al.  Latent Structure Analysis. , 1969 .

[3]  Jeffrey A Douglas,et al.  Higher-order latent trait models for cognitive diagnosis , 2004 .

[4]  Dean Follmann,et al.  Consistent estimation in the rasch model based on nonparametric margins , 1988 .

[5]  Matthias von Davier,et al.  PARTIALLY OBSERVED MIXTURES OF IRT MODELS: AN EXTENSION OF THE GENERALIZED PARTIAL CREDIT MODEL , 2003 .

[6]  A. Agresti.  Categorical data analysis , 1993 .

[7]  Shelby J. Haberman,et al.  Latent-Class Item Response Models. Research Report. ETS RR-05-28. , 2005 .

[8]  Matthias von Davier,et al.  A general diagnostic model applied to language testing data. , 2008, The British journal of mathematical and statistical psychology.

[9]  E. Maris.  Psychometric latent response models , 1995 .

[10]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[11]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions , and Connections with Nonparametric IRT , 2001 .

[12]  A. Formann.  Linear Logistic Latent Class Analysis for Polytomous Data , 1992 .

[13]  T. Heinen,et al.  Latent Class and Discrete Latent Trait Models: Similarities and Differences , 1996 .

[14]  Shelby J. Haberman,et al.  LATENT-CLASS ITEM RESPONSE MODELS , 2005 .

[15]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[16]  H. Akaike.  A new look at the statistical model identification , 1974 .

[17]  M. W. Birch.  Maximum Likelihood in Three-Way Contingency Tables , 1963 .

[18]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[19]  S. Haberman.  Analysis of qualitative data , 1978 .

[20]  Sarah M. Hartz,et al.  A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. , 2002 .

[21]  Matthias von Davier,et al.  A General Diagnostic Model Applied to Language Testing Data. Research Report. ETS RR-05-16. , 2005 .

[22]  Alan Agresti,et al.  Categorical Data Analysis , 1991, International Encyclopedia of Statistical Science.

[23]  N. Laird.  Nonparametric Maximum Likelihood Estimation of a Mixing Distribution , 1978 .

[24]  Matthias von Davier,et al.  A GENERAL DIAGNOSTIC MODEL APPLIED TO LANGUAGE TESTING DATA , 2005 .

[25]  Juni Palmgren,et al.  The Fisher information matrix for log linear models arguing conditionally on observed explanatory variable , 1981 .