Consistency of Nonparametric Classification in Cognitive Diagnosis

Latent class models for cognitive diagnosis have been developed to classify examinees into one of the 2K attribute profiles arising from a K-dimensional vector of binary skill indicators. These models recognize that response patterns tend to deviate from the ideal responses that would arise if skills and items generated item responses through a purely deterministic conjunctive process. An alternative to employing these latent class models is to minimize the distance between observed item response patterns and ideal response patterns, in a nonparametric fashion that utilizes no stochastic terms for these deviations. Theorems are presented that show the consistency of this approach, when the true model is one of several common latent class models for cognitive diagnosis. Consistency of classification is independent of sample size, because no model parameters need to be estimated. Simultaneous consistency for a large group of subjects can also be shown given some conditions on how sample size and test length grow with one another.

[1]  Kikumi K. Tatsuoka,et al.  Bug distribution and statistical pattern classification , 1987 .

[2]  C. Mitchell Dayton,et al.  The Use of Probabilistic Models in the Assessment of Mastery , 1977 .

[3]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[4]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[5]  Kikumi K. Tatsuoka,et al.  Architecture of knowledge structures and cognitive diagnosis: A statistical pattern recognition and classification approach. , 1995 .

[6]  Rebecca Nugent,et al.  Skill set profile clustering based on student capability vectors computed from online tutoring data , 2008 .

[7]  Chia-Yi Chiu,et al.  A Nonparametric Approach to Cognitive Diagnosis by Proximity to Ideal Response Patterns , 2013, J. Classif..

[8]  Chia-Yi Chiu,et al.  Cluster Analysis for Cognitive Diagnosis: Theory and Applications , 2009 .

[9]  Kikumi K. Tatsuoka,et al.  Bug Distribution and Pattern Classification. , 1985 .

[10]  John T. Willse,et al.  Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables , 2009 .

[11]  Kikumi K. Tatsuoka,et al.  A Probabilistic Model for Diagnosing Misconceptions By The Pattern Classification Approach , 1985 .

[12]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions , and Connections with Nonparametric IRT , 2001 .

[13]  J. D. L. Torre,et al.  The Generalized DINA Model Framework. , 2011 .

[14]  Y. Park.  Diagnostic cluster analysis of mathematics skills , 2011 .

[15]  Matthias von Davier,et al.  A General Diagnostic Model Applied to Language Testing Data. Research Report. ETS RR-05-16. , 2005 .

[16]  Tiffany Barnes.  Novel Derivation and Application of Skill Matrices: The q-Matrix Method , 2010 .

[17]  J. Templin,et al.  Skills Diagnosis Using IRT-Based Latent Class Models , 2007 .

[18]  Louis Roussos,et al.  THE FUSION MODEL FOR SKILLS DIAGNOSIS: BLENDING THEORY WITH PRACTICALITY , 2008 .

[19]  K. Tatsuoka.  RULE SPACE: AN APPROACH FOR DEALING WITH MISCONCEPTIONS BASED ON ITEM RESPONSE THEORY , 1983 .