Hierarchical Diagnostic Classification Models: A Family of Models for Estimating and Testing Attribute Hierarchies

Although latent attributes that follow a hierarchical structure are anticipated in many areas of educational and psychological assessment, current psychometric models are limited in their capacity to objectively evaluate the presence of such attribute hierarchies. This paper introduces the Hierarchical Diagnostic Classification Model (HDCM), which adapts the Log-linear Cognitive Diagnosis Model to cases where attribute hierarchies are present. The utility of the HDCM is demonstrated through simulation and by an empirical example. Simulation study results show the HDCM is efficiently estimated and can accurately test for the presence of an attribute hierarchy statistically, a feature not possible when using more commonly used DCMs. Empirically, the HDCM is used to test for the presence of a suspected attribute hierarchy in a test of English grammar, confirming the data is more adequately represented by hierarchical attribute structure when compared to a crossed, or nonhierarchical structure.

[1]  Louis A. Roussos,et al.  The fusion model skills diagnosis system , 2007 .

[2]  A. Shapiro.  Asymptotic distribution of test statistics in the analysis of moment structures under inequality constraints , 1985 .

[3]  S. Chipman,et al.  Cognitively diagnostic assessment , 1995 .

[4]  Conor Dolan,et al.  On the likelihood ratio test in structural equation modeling when parameters are subject to boundary constraints. , 2006, Psychological methods.

[5]  J. Templin,et al.  Measurement of psychological disorders using cognitive diagnosis models. , 2006, Psychological methods.

[6]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[7]  Lesa Hoffman,et al.  Obtaining Diagnostic Classification Model Estimates Using Mplus , 2013 .

[8]  K. Tatsuoka.  Toward an Integration of Item-Response Theory and Cognitive Error Diagnosis. , 1987 .

[9]  Matthias von Davier,et al.  A GENERAL DIAGNOSTIC MODEL APPLIED TO LANGUAGE TESTING DATA , 2005 .

[10]  Mark J. Gierl,et al.  Cognitive Diagnostic Assessment for Education: Using the Attribute Hierarchy Method to Make Diagnostic Inferences About Examinees' Cognitive Skills , 2007 .

[11]  Paul F. Lazarsfeld,et al.  Latent Structure Analysis. , 1969 .

[12]  J. D. L. Torre,et al.  The Generalized DINA Model Framework. , 2011 .

[13]  John T. Willse,et al.  Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables , 2009 .

[14]  R. Adams,et al.  Construction Versus Choice in Cognitive Measurement , 1995 .

[15]  G. Molenberghs,et al.  Linear Mixed Models for Longitudinal Data , 2001 .

[16]  K. Tatsuoka,et al.  Application of the rule-space procedure to language testing: examining attributes of a free response listening test , 1998 .

[17]  Jimmy de la Torre.  Erratum to: The Generalized DINA Model Framework , 2011 .

[18]  Albert Maydeu-Olivares,et al.  Limited- and Full-Information Estimation and Goodness-of-Fit Testing in 2n Contingency Tables , 2005 .

[19]  Laine Bradshaw,et al.  Combining Item Response Theory and Diagnostic Classification Models: A Psychometric Model for Scaling Ability and Diagnosing Misconceptions , 2014, Psychometrika.

[20]  B. Lindsay,et al.  Semiparametric Estimation in the Rasch Model and Related Exponential Response Models, Including a Simple Latent Class Model for Item Analysis , 1991 .

[21]  Mark J. Gierl,et al.  The Attribute Hierarchy Method for Cognitive Assessment: A Variation on Tatsuoka's Rule-Space Approach , 2004 .

[22]  Ying Cui,et al.  The Hierarchy Consistency Index: Evaluating Person Fit for Cognitive Diagnostic Assessment , 2009 .

[23]  Sarah M. Hartz,et al.  A Bayesian framework for the unified model for assessing cognitive abilities: Blending theory with practicality. , 2002 .

[24]  Ying Cui,et al.  Testing Expert-Based and Student-Based Cognitive Models: An Application of the Attribute Hierarchy Method and Hierarchy Consistency Index , 2009 .

[25]  Laine Bradshaw,et al.  Measuring the Reliability of Diagnostic Classification Model Examinee Estimates , 2013, J. Classif..

[26]  Jonathan Templin,et al.  Diagnostic Measurement: Theory, Methods, and Applications , 2010 .

[27]  Edward H. Haertel.  Using restricted latent class models to map the skill structure of achievement items , 1989 .

[28]  Hye-Jeong Choi,et al.  A model that combines diagnostic classification assessment with mixture item response theory models , 2010 .

[29]  Mark J. Gierl,et al.  Cognitive diagnostic assessment for education: Theory and applications. , 2007 .

[30]  Mark Gierl,et al.  Reliability and Attribute-Based Scoring in Cognitive Diagnostic Assessment. , 2009 .

[31]  Kikumi K. Tatsuoka,et al.  Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. , 1995 .

[32]  K. Tatsuoka.  Cognitive Assessment: An Introduction to the Rule Space Method , 2009 .

[33]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[34]  Marcel A. Croon,et al.  Latent class analysis with ordered latent classe , 1990 .

[35]  Kikumi K. Tatsuoka,et al.  ITEM CONSTRUCTION AND PSYCHOMETRIC MODELS APPROPRIATE FOR CONSTRUCTED RESPONSES , 1991 .

[36]  Matthias von Davier,et al.  FITTING THE STRUCTURED GENERAL DIAGNOSTIC MODEL TO NAEP DATA , 2008 .

[37]  K. Tatsuoka.  RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .