Identifiability of the Latent Attribute Space and Conditions of Q-Matrix Completeness for Attribute Hierarchy Models

Educational researchers have argued that a realistic view of the role of attributes in cognitively diagnostic modeling should account for the possibility that attributes are not isolated entities, but interdependent in their effect on test performance. Different approaches have been discussed in the literature; among them the proposition to impose a hierarchical structure so that mastery of one or more attributes is a prerequisite of mastering one or more other attributes. A hierarchical organization of attributes constrains the latent attribute space such that several proficiency classes, as they exist if attributes are not hierarchically organized, are no longer defined, because the corresponding attribute combinations cannot occur with the given attribute hierarchy. Hence, the identification of the latent attribute space is often difficult—especially, if the number of attributes is large. As an additional complication, constructing a complete Q-matrix may not at all be straightforward if the attributes underlying the test items are supposed to have a hierarchical structure. In this article, the conditions of identifiability of the latent space if attributes are hierarchically organized and the conditions of completeness of the Q-matrix are studied.

[1]  Laine Bradshaw,et al.  Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies , 2014, Psychometrika.

[2]  Mark J. Gierl,et al.  Cognitive diagnostic assessment for education: Theory and applications. , 2007 .

[3]  Chia-Yi Chiu,et al.  Cluster Analysis for Cognitive Diagnosis: Theory and Applications , 2009 .

[4]  Kikumi K. Tatsuoka,et al.  A Probabilistic Model for Diagnosing Misconceptions By The Pattern Classification Approach , 1985 .

[5]  C. Mitchell Dayton,et al.  The Use of Probabilistic Models in the Assessment of Mastery , 1977 .

[6]  Jeffrey A Douglas,et al.  Higher-order latent trait models for cognitive diagnosis , 2004 .

[7]  John T. Willse,et al.  Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables , 2009 .

[8]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[9]  K. Tatsuoka Cognitive Assessment: An Introduction to the Rule Space Method , 2009 .

[10]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[11]  Jonathan Templin,et al.  Diagnostic Measurement: Theory, Methods, and Applications , 2010 .

[12]  Mark J. Gierl,et al.  The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka's Rule-Space Approach , 2004 .

[13]  Matthias von Davier,et al.  A General Diagnostic Model Applied to Language Testing Data. Research Report. ETS RR-05-16. , 2005 .

[14]  Chia-Yi Chiu,et al.  A Procedure for Assessing the Completeness of the Q-Matrices of Cognitively Diagnostic Tests , 2017, Psychometrika.

[15]  Edward H. Haertel Using restricted latent class models to map the skill structure of achievement items , 1989 .

[16]  Louis V. DiBello,et al.  31A Review of Cognitively Diagnostic Assessment and a Summary of Psychometric Models , 2006 .

[17]  J. D. L. Torre,et al.  The Generalized DINA Model Framework. , 2011 .