The Random‐Effect DINA Model

The DINA (deterministic input, noisy, and gate) model has been widely used in cognitive diagnosis tests and in the process of test development. The outcomes known as slip and guess are included in the DINA model function representing the responses to the items. This study aimed to extend the DINA model by using the random-effect approach to allow examinees to have different probabilities of slipping and guessing. Two extensions of the DINA model were developed and tested to represent the random components of slipping and guessing. The first model assumed that a random variable can be incorporated in the slipping parameters to allow examinees to have different levels of caution. The second model assumed that the examinees’ ability may increase the probability of a correct response if they have not mastered all of the required attributes of an item. The results of a series of simulations based on Markov chain Monte Carlo methods showed that the model parameters and attribute-mastery profiles can be recovered relatively accurately from the generating models and that neglect of the random effects produces biases in parameter estimation. Finally, a fraction subtraction test was used as an empirical example to demonstrate the application of the new models.

[1]  Curtis Tatsuoka,et al.  Data analytic methods for latent partially ordered classification models , 2002 .

[2]  Young-sun Lee,et al.  A Cognitive Diagnostic Modeling of Attribute Mastery in Massachusetts, Minnesota, and the U.S. National Sample Using the TIMSS 2007 , 2011 .

[3]  Wen-Chung Wang,et al.  A Generalized Model With Internal Restrictions on Item Difficulty for Polytomous Items , 2010 .

[4]  H. Akaike.  A new look at the statistical model identification , 1974 .

[5]  Jeffrey A Douglas,et al.  Higher-order latent trait models for cognitive diagnosis , 2004 .

[6]  Jean-Paul Fox,et al.  Bayesian Item Response Modeling , 2010 .

[7]  Daniel M. Bolt,et al.  Estimation of Compensatory and Noncompensatory Multidimensional Item Response Models Using Markov Chain Monte Carlo , 2003 .

[8]  Donald Geman,et al.  Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images , 1984, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[9]  B. Junker,et al.  Cognitive Assessment Models with Few Assumptions, and Connections with Nonparametric Item Response Theory , 2001 .

[10]  Jimmy de la Torre,et al.  Model Evaluation and Multiple Strategies in Cognitive Diagnosis: An Analysis of Fraction Subtraction Data , 2008 .

[11]  Jimmy de la Torre,et al.  Factors Affecting the Item Parameter Estimation and Classification Accuracy of the DINA Model , 2010 .

[12]  S. Chib,et al.  Understanding the Metropolis-Hastings Algorithm , 1995 .

[13]  Wen-Chung Wang,et al.  Modeling Randomness in Judging Rating Scales with a Random‐Effects Rating Scale Model , 2006 .

[14]  Valen E. Johnson.  A Bayesian x2 test for goodness-of-fit , 2004 .

[15]  J. Templin,et al.  Unique Characteristics of Diagnostic Classification Models: A Comprehensive Review of the Current State-of-the-Art , 2008 .

[16]  Daniel M. Bolt,et al.  A Comparison of Alternative Models for Testlets , 2006 .

[17]  J. D. L. Torre,et al.  DINA Model and Parameter Estimation: A Didactic , 2009 .

[18]  J. D. L. Torre,et al.  A Note on the Invariance of the DINA Model Parameters. , 2010 .

[19]  Edward H. Haertel.  Using restricted latent class models to map the skill structure of achievement items , 1989 .

[20]  W. Wong,et al.  The calculation of posterior distributions by data augmentation , 1987 .

[21]  Hal S. Stern,et al.  Posterior Predictive Assessment of Item Response Theory Models , 2006 .

[22]  Jing Cao,et al.  Bayesian IRT Guessing Models for Partial Guessing Behaviors , 2008 .

[23]  J. D. L. Torre,et al.  The Generalized DINA Model Framework. , 2011 .

[24]  Jimmy de la Torre.  Erratum to: The Generalized DINA Model Framework , 2011 .

[25]  Paul De Boeck,et al.  IRT Models for Ability-Based Guessing , 2006 .

[26]  D. Rubin.  Bayesianly Justifiable and Relevant Frequency Calculations for the Applied Statistician , 1984 .

[27]  Adrian F. M. Smith,et al.  Sampling-Based Approaches to Calculating Marginal Densities , 1990 .

[28]  Andrew Gelman,et al.  General methods for monitoring convergence of iterative simulations , 1998 .

[29]  Xiao-Li Meng,et al.  POSTERIOR PREDICTIVE ASSESSMENT OF MODEL FITNESS VIA REALIZED DISCREPANCIES , 1996 .

[30]  Wen-Chung Wang,et al.  Multilevel, Two-Parameter, and Random-Weights Generalizations of a Model With Internal Restrictions on Item Difficulty , 2010 .

[31]  Sandip Sinharay,et al.  Assessing Fit of Unidimensional Item Response Theory Models Using a Bayesian Approach. , 2005 .

[32]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[33]  E. Maris.  Estimating multiple classification latent class models , 1999 .

[34]  Wen-Chung Wang,et al.  Formulation and Application of the Generalized Multilevel Facets Model , 2007 .

[35]  John T. Willse,et al.  Defining a Family of Cognitive Diagnosis Models Using Log-Linear Models with Latent Variables , 2009 .

[36]  Ying Cheng.  When Cognitive Diagnosis Meets Computerized Adaptive Testing: CD-CAT , 2009 .