Representing probabilistic models of knowledge space theory by multinomial processing tree models

Abstract Knowledge Space Theory (KST) aims at modeling the hierarchical relations between items or skills in a learning process. For example, when studying mathematics in school, students first need to master the rules of summation before being able to learn multiplication. In KST, the knowledge states of individuals are represented by means of partially ordered latent classes. In probabilistic KST models, conditional probability parameters are introduced to model transitions from latent knowledge states to observed response patterns. Since these models account for discrete data by assuming a finite number of latent states, they can be represented by Multinomial Processing Tree (MPT) models (i.e., binary decision trees with parameters referring to the conditional probabilities of entering different states). We prove that standard probabilistic models of KST such as the Basic Local Independence Model (BLIM) and the Simple Learning Model (SLM) can be represented as specific instances of MPT models. Given this close link, MPT methods may be applied to address theoretical and practical issues in KST. By highlighting the MPT–KST link and its implications for modeling violations of local stochastic independence in Item Response Theory (IRT), we hope to facilitate an exchange of theoretical results, statistical methods, and software across these different domains of mathematical psychology and psychometrics.

[1]  E. Wagenmakers,et al.  Bayesian Estimation of Multinomial Processing Tree Models with Heterogeneity in Participants and Items , 2013, Psychometrika.

[2]  William H. Batchelder,et al.  Representing parametric order constraints in multi-trial applications of multinomial processing tree models , 2004 .

[3]  M. R. Novick,et al.  Statistical Theories of Mental Test Scores. , 1971 .

[4]  Edgar Erdfelder,et al.  Extending multinomial processing tree models to measure the relative speed of cognitive processes , 2016, Psychonomic bulletin & review.

[5]  Andrea Spoto,et al.  A class of k-modes algorithms for extracting knowledge structures from data , 2016, Behavior Research Methods.

[6]  David M. Riefer,et al.  Multinomial Modeling and the Measurement of Cognitive Processes , 2001 .

[7]  William H. Batchelder,et al.  Separation of storage and retrieval factors in free recall of clusterable pairs. , 1980 .

[8]  William A. Link,et al.  Bayesian Multimodel Inference by RJMCMC: A Gibbs Sampling Approach , 2013 .

[9]  Thorsten Meiser,et al.  Analyzing stochastic dependence of cognitive processes in multidimensional source recognition. , 2014, Experimental psychology.

[10]  David Eppstein,et al.  Knowledge Spaces, Applications in Education , 2013 .

[11]  L. Stefanutti A logistic approach to knowledge structures , 2006 .

[12]  Andrea Spoto,et al.  Considerations about the identification of forward- and backward-graded knowledge structures , 2013 .

[13]  David M. Riefer,et al.  Multinomial processing models of source monitoring. , 1990 .

[14]  Eric Ras,et al.  Learning Spaces , 2022, The SAGE Encyclopedia of Higher Education.

[15]  Ali Ünlü,et al.  Inductive item tree analysis: Corrections, improvements, and comparisons , 2009, Math. Soc. Sci..

[16]  Jean-Claude Falmagne Probabilistic Knowledge Spaces: A Review , 1989 .

[17]  M. Villano,et al.  Introduction to knowledge spaces: How to build, test, and search them , 1990 .

[18]  Hao Wu,et al.  On the Minimum Description Length Complexity of Multinomial Processing Tree Models. , 2010, Journal of mathematical psychology.

[19]  G. Schwarz Estimating the Dimension of a Model , 1978 .

[20]  Morten Moshagen,et al.  multiTree: A computer program for the analysis of multinomial processing tree models , 2010, Behavior research methods.

[21]  A. Ünlü Nonparametric item response theory axioms and properties under nonlinearity and their exemplification with knowledge space theory , 2007 .

[22]  Wh Batchelder,et al.  Multinomial models of some standard memory paradigms , 1988 .

[23]  W. Batchelder,et al.  The statistical analysis of general processing tree models with the EM algorithm , 1994 .

[24]  Edgar Erdfelder,et al.  Conjoint measurement of disorder prevalence, test sensitivity, and test specificity: notes on Botella, Huang, and Suero's multinomial model , 2013, Front. Psychol..

[25]  Andrea Spoto,et al.  On a Generalization of Local Independence in Item Response Theory Based on Knowledge Space Theory , 2018, Psychometrika.

[26]  Jorma Rissanen,et al.  Fisher information and stochastic complexity , 1996, IEEE Trans. Inf. Theory.

[27]  Jean-Claude Falmagne,et al.  Languages for the assessment of knowledge , 1986 .

[28]  Georg Rasch,et al.  Probabilistic Models for Some Intelligence and Attainment Tests , 1981, The SAGE Encyclopedia of Research Design.

[29]  Jean-Claude Falmagne,et al.  Knowledge spaces , 1998 .

[30]  Jürgen Heller,et al.  Assessing the local identifiability of probabilistic knowledge structures , 2012, Behavior research methods.

[31]  Jürgen Heller,et al.  On the Link between Cognitive Diagnostic Models and Knowledge Space Theory , 2015, Psychometrika.

[32]  Rudolf Wille,et al.  Knowledge Spaces and Formal Concept Analysis , 1996 .

[33]  David M. Riefer,et al.  Cognitive psychometrics: assessing storage and retrieval deficits in special populations with multinomial processing tree models. , 2002, Psychological assessment.

[34]  B. Hilbig,et al.  Multinomial processing tree models: A review of the literature. , 2009 .

[35]  Jean-Claude Falmagne,et al.  Spaces for the Assessment of Knowledge , 1985, Int. J. Man Mach. Stud..

[36]  William H. Batchelder,et al.  Cognitive psychometrics: Using multinomial processing tree models as measurement tools. , 2010 .

[37]  Andrea Spoto,et al.  On the unidentifiability of a certain class of skill multi map based probabilistic knowledge structures , 2012 .

[38]  A. Ünlü A Note on the Connection Between Knowledge Structures and Latent Class Models , 2011 .

[39]  Daniel W. Heck,et al.  TreeBUGS: An R package for hierarchical multinomial-processing-tree modeling , 2017, Behavior Research Methods.

[40]  W. Batchelder,et al.  A context-free language for binary multinomial processing tree models , 2009 .

[41]  Keam-Claude Falmagne,et al.  A latent trait theory via a stochastic learning theory for a knowledge space , 1989 .

[42]  William H. Batchelder,et al.  The statistical analysis of a model for storage and retrieval processes in human memory , 1986 .

[43]  K. C. Klauer Hierarchical Multinomial Processing Tree Models: A Latent-Trait Approach , 2006 .

[44]  Morten Moshagen,et al.  Model selection by minimum description length : Lower-bound sample sizes for the Fisher information approximation , 2014, 1808.00212.

[45]  Martin Schrepp,et al.  About the Connection Between Knowledge Structures and Latent Class Models , 2005 .

[46]  A. Bröder,et al.  Recognition ROCs are curvilinear-or are they? On premature arguments against the two-high-threshold model of recognition. , 2009, Journal of experimental psychology. Learning, memory, and cognition.

[47]  David M. Riefer,et al.  Theoretical and empirical review of multinomial process tree modeling , 1999, Psychonomic bulletin & review.

[48]  I. W. Molenaar,et al.  Rasch models: foundations, recent developments and applications , 1995 .

[49]  Luca Stefanutti,et al.  Recovering a Probabilistic Knowledge Structure by Constraining its Parameter Space , 2009 .

[50]  D. Heck,et al.  A New Model for Acquiescence at the Interface of Psychometrics and Cognitive Psychology , 2018, Multivariate behavioral research.

[51]  Andrea Spoto,et al.  An iterative procedure for extracting skill maps from data , 2015, Behavior Research Methods.

[52]  Jürgen Heller,et al.  Minimum Discrepancy Estimation in Probabilistic Knowledge Structures , 2013, Electron. Notes Discret. Math..

[53]  Andrea Spoto,et al.  Detecting and explaining BLIM’s unidentifiability: Forward and backward parameter transformation groups , 2018 .

[54]  I. J. Myung,et al.  Applying Occam’s razor in modeling cognition: A Bayesian approach , 1997 .

[55]  Martin Schrepp Extracting knowledge structures from observed data , 1999 .

[56]  J. Heller Identifiability in probabilistic knowledge structures , 2017 .