Evaluating model estimation processes for diagnostic classification models

Diagnostic classification models (DCMs) are a class of models that define respondent ability on a set of predefined categorical latent variables. In recent years, the popularity of these models has begun to increase. As the community of researchers of practitioners of DCMs grow, it is important to examine the implementation of these models, including the process of model estimation. A key aspect of the estimation process that remains unexplored in the DCM literature is model reduction, or the removal of parameters from the model in order to create a simpler, more parsimonious model. The current study fills this gap in the literature by first applying several model reduction processes on a real data set, the Diagnosing Teachers’ Multiplicative Reasoning assessment (Bradshaw et al., 2014). Results from this analysis indicate that the selection of model reduction process can have large implications for the resulting parameter estimates and respondent classifications. A simulation study is then conducted to evaluate the relative performance of these various model reduction processes. The results of the simulation suggest that all model reduction processes are able to provide quality estimates of the item parameters and respondent masteries, if the model is able to converge. The findings also show that if the full model does not converge, then reducing the structural model provides the best opportunities for achieving a converged solution. Implications of this study and directions for future research are discussed.