Measuring Model Flexibility With Parameter Space Partitioning: An Introduction and Application Example

A primary criterion on which models of cognition are evaluated is their ability to fit empirical data. To understand the reason why a model yields a good or poor fit, it is necessary to determine the data-fitting potential (i.e., flexibility) of the model. In the first part of this article, methods for comparing models and studying their flexibility are reviewed, with a focus on parameter space partitioning (PSP), a general-purpose method for analyzing and comparing all classes of cognitive models. PSP is then demonstrated in the second part of the article in which two connectionist models of speech perception (TRACE and ARTphone) are compared to learn how design differences affect model flexibility.

[1]  R. Shiffrin,et al.  A model for recognition memory: REM—retrieving effectively from memory , 1997, Psychonomic bulletin & review.

[2]  H. Akaike,et al.  Information Theory and an Extension of the Maximum Likelihood Principle , 1973 .

[3]  G. Schwarz.  Estimating the Dimension of a Model , 1978 .

[4]  Prahlad Gupta,et al.  Does neighborhood density influence repetition latency for nonwords? Separating the effects of density and duration , 2004 .

[5]  I. J. Myung,et al.  When a good fit can be bad , 2002, Trends in Cognitive Sciences.

[6]  Jorma Rissanen,et al.  Minimum Description Length Principle , 2010, Encyclopedia of Machine Learning.

[7]  I. J. Myung,et al.  Applying Occam’s razor in modeling cognition: A Bayesian approach , 1997 .

[8]  D Norris,et al.  Merging information in speech recognition: Feedback is never necessary , 2000, Behavioral and Brain Sciences.

[9]  Jay I. Myung,et al.  Modeling the word recognition data of Vitevitch and Luce (1998): Is it ARTful? , 2007, Psychonomic bulletin & review.

[10]  P. Luce,et al.  When Words Compete: Levels of Processing in Perception of Spoken Words , 1998 .

[11]  Jay I. Myung,et al.  Global model analysis by parameter space partitioning. , 2019, Psychological review.

[12]  G. Miller,et al.  Cognitive science. , 1981, Science.

[13]  Michael S Vitevitch,et al.  The influence of sublexical and lexical representations on the processing of spoken words in English , 2003, Clinical linguistics & phonetics.

[14]  Mark A. Pitt,et al.  Advances in Minimum Description Length: Theory and Applications , 2005 .

[15]  L. Wasserman,et al.  Computing Bayes Factors by Combining Simulation and Asymptotic Approximations , 1997 .

[16]  P. Luce,et al.  Increases in phonotactic probability facilitate spoken nonword repetition. , 2005 .

[17]  James L. McClelland,et al.  The TRACE model of speech perception , 1986, Cognitive Psychology.

[18]  Ted J. Strauss,et al.  jTRACE: A reimplementation and extension of the TRACE model of speech perception and spoken word recognition , 2007, Behavior research methods.

[19]  I. J. Myung,et al.  GUEST EDITORS' INTRODUCTION: Special Issue on Model Selection , 2000 .

[20]  I. J. Myung,et al.  Tutorial on maximum likelihood estimation , 2003 .

[21]  Ron Sun,et al.  Learning, action and consciousness: a hybrid approach toward modelling consciousness , 1997, Neural Networks.

[22]  Amy Wenzel,et al.  One hundred years of forgetting: A quantitative description of retention , 1996 .

[23]  Peter Green,et al.  Markov chain Monte Carlo in Practice , 1996 .

[24]  P. Luce,et al.  Probabilistic Phonotactics and Neighborhood Activation in Spoken Word Recognition , 1999 .

[25]  S. Grossberg,et al.  Neural dynamics of variable-rate speech categorization. , 1997, Journal of experimental psychology. Human perception and performance.

[26]  J. Townsend,et al.  Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. , 1993, Psychological review.

[27]  Safa R. Zaki,et al.  Exemplar and prototype models revisited: response strategies, selective attention, and stimulus generalization. , 2002, Journal of experimental psychology. Learning, memory, and cognition.

[28]  I. J. Myung,et al.  Counting probability distributions: Differential geometry and model selection , 2000, Proc. Natl. Acad. Sci. USA.

[29]  I. J. Myung,et al.  Toward a method of selecting among computational models of cognition. , 2002, Psychological review.

[30]  Michael S Vitevitch,et al.  A Web-based interface to calculate phonotactic probability for words and nonwords in English , 2004, Behavior research methods, instruments, & computers : a journal of the Psychonomic Society, Inc.

[31]  C. Lebiere,et al.  The Atomic Components of Thought , 1998 .