Robustness of Hierarchical Modeling of Skill Association in Cognitive Diagnosis Models

Several types of parameterizations of attribute correlations in cognitive diagnosis models use the reduced reparameterized unified model. The general approach presumes an unconstrained correlation matrix with K(K−1)/2 parameters, whereas the higher order approach postulates K parameters, imposing a unidimensional structure on the correlation matrix between the latent skills. This article investigates the differences in performance between the correlational structure parameterizations (a general structure, a higher order single-factor structure, and a baseline uniform distributional approach constraining the attributes to be independent) across a wide variety of simulated multidimensional attribute spaces. Results suggest that the correlational approaches perform equally well with respect to classification and item parameter estimation accuracy, regardless of the violations of the assumptions of the higher order model. Findings suggest the general robustness of the higher order model and the associated estimation procedure. The three approaches are also used to analyze a real-world test; results suggest that such tests can be analyzed effectively by the higher order algorithm.

[1]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[2]  Jeffrey A Douglas,et al.  Higher-order latent trait models for cognitive diagnosis , 2004 .

[3]  Edward H. Haertel.  Using restricted latent class models to map the skill structure of achievement items , 1989 .

[4]  Richard J. Patz,et al.  A Straightforward Approach to Markov Chain Monte Carlo Methods for Item Response Models , 1999 .

[5]  C. Mitchell Dayton,et al.  The Use of Probabilistic Models in the Assessment of Mastery , 1977 .

[6]  D. Rubin,et al.  Inference from Iterative Simulation Using Multiple Sequences , 1992 .

[7]  Ding Shuliang,et al.  Test Construction for Cognitive Diagnosis , 2010, 2010 Asia-Pacific Conference on Wearable Computing Systems.

[8]  Jonathan L. Templin.  Generalized Linear Mixed Proficiency Models for Cognitive Diagnosis , 2004 .

[9]  Sarah M. Hartz,et al.  A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. , 2002 .

[10]  K. Tatsuoka.  Toward an Integration of Item-Response Theory and Cognitive Error Diagnosis. , 1987 .

[11]  Anders Christoffersson,et al.  Factor analysis of dichotomized variables , 1975 .

[12]  J. Templin,et al.  Measurement of psychological disorders using cognitive diagnosis models. , 2006, Psychological methods.

[13]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[14]  Xiao-Li Meng,et al.  Modeling covariance matrices in terms of standard deviations and correlations, with application to shrinkage , 2000 .