Bayesian Models of Cognition Revisited: Setting Optimality Aside and Letting Data Drive Psychological Theory

Recent debates in the psychological literature have raised questions about the assumptions that underpin Bayesian models of cognition and what inferences they license about human cognition. In this paper we revisit this topic, arguing that there are 2 qualitatively different ways in which a Bayesian model could be constructed. The most common approach uses a Bayesian model as a normative standard upon which to license a claim about optimality. In the alternative approach, a descriptive Bayesian model need not correspond to any claim that the underlying cognition is optimal or rational, and is used solely as a tool for instantiating a substantive psychological theory. We present 3 case studies in which these 2 perspectives lead to different computational models and license different conclusions about human cognition. We demonstrate how the descriptive Bayesian approach can be used to answer different sorts of questions than the optimal approach, especially when combined with principled tools for model evaluation and model selection. More generally we argue for the importance of making a clear distinction between the 2 perspectives. Considerable confusion results when descriptive models and optimal models are conflated, and if Bayesians are to avoid contributing to this confusion it is important to avoid making normative claims when none are intended.

[1]  M. Lee Three case studies in the Bayesian analysis of cognitive models , 2008, Psychonomic bulletin & review.

[2]  Amy Perfors,et al.  Leaping to Conclusions: Why Premise Relevance Affects Argument Strength , 2016, Cogn. Sci..

[3]  J. Tenenbaum A Bayesian framework for concept learning , 1999 .

[4]  B. D. Finetti,et al.  Foresight: Its Logical Laws, Its Subjective Sources , 1992 .

[5]  John R. Anderson,et al.  The Adaptive Character of Thought , 1990 .

[6]  Gary F. Marcus,et al.  Still Searching for Principles , 2015, Psychological science.

[7]  Refractor Vision , 2000, The Lancet.

[8]  M. Lee,et al.  Bayesian Cognitive Modeling: A Practical Course , 2014 .

[9]  Thomas L. Griffiths,et al.  One and Done? Optimal Decisions From Very Few Samples , 2014, Cogn. Sci..

[10]  R. Baierlein Probability Theory: The Logic of Science , 2004 .

[11]  Amy Perfors,et al.  Hypothesis generation, sparse categories, and the positive test strategy. , 2011, Psychological review.

[12]  Jay I. Myung,et al.  Model selection by Normalized Maximum Likelihood , 2006 .

[13]  E. Wagenmakers A practical solution to the pervasive problems ofp values , 2007, Psychonomic bulletin & review.

[14]  Richard M. Shiffrin,et al.  Model Selection, Data Distributions, and Reproducibility , 2016 .

[15]  Scott D Brown,et al.  Using alien coins to test whether simple inference is Bayesian. , 2016, Journal of experimental psychology. Learning, memory, and cognition.

[16]  J. Tenenbaum,et al.  From mere coincidences to meaningful discoveries , 2007, Cognition.

[17]  John K Kruschke,et al.  Bayesian data analysis. , 2010, Wiley interdisciplinary reviews. Cognitive science.

[18]  D. Tadin,et al.  Illusory Movement of Stationary Stimuli in the Visual Periphery: Evidence for a Strong Centrifugal Prior in Motion Processing , 2013, The Journal of Neuroscience.

[19]  E. Davis,et al.  How Robust Are Probabilistic Models of Higher-Level Cognition? , 2013, Psychological science.

[20]  Joshua B. Tenenbaum,et al.  Learning the Structure of Similarity , 1995, NIPS.

[21]  A. Tversky,et al.  The hot hand in basketball: On the misperception of random sequences , 1985, Cognitive Psychology.

[22]  Jeffrey N. Rouder,et al.  Default Bayes factors for ANOVA designs , 2012 .

[23]  David B. Dunson,et al.  Bayesian Data Analysis , 2010 .

[24]  Pernille Hemmer,et al.  Moving beyond qualitative evaluations of Bayesian models of cognition , 2015, Psychonomic bulletin & review.

[25]  M. Lengyel,et al.  Mind Reading by Machine Learning: A Doubly Bayesian Method for Inferring Mental Representations , 2010 .

[26]  Michael C. Frank,et al.  Relevant and Robust , 2015, Psychological science.

[27]  Luigi Acerbi,et al.  A Framework for Testing Identifiability of Bayesian Models of Perception , 2014, NIPS.

[28]  Michael D. Lee Generating Additive Clustering Models with Minimal Stochastic Complexity , 2002, J. Classif..

[29]  Joshua B. Tenenbaum,et al.  A probabilistic model of cross-categorization , 2011, Cognition.

[30]  Paul R. Schrater,et al.  How Haptic Size Sensations Improve Distance Perception , 2011, PLoS Comput. Biol..

[31]  Jay I. Myung,et al.  Global model analysis by parameter space partitioning. , 2019, Psychological review.

[32]  R. Shepard,et al.  Toward a universal law of generalization for psychological science. , 1987, Science.

[33]  Paul Teller,et al.  Conditionalization and observation , 1973, Synthese.

[34]  John R. Anderson,et al.  Reflections of the Environment in Memory Form of the Memory Functions , 2022 .

[35]  Jonathan D. Cohen,et al.  Sequential effects: Superstition or rational behavior? , 2008, NIPS.

[36]  Charles Kemp,et al.  The discovery of structural form , 2008, Proceedings of the National Academy of Sciences.

[37]  J. Tenenbaum,et al.  Optimal Predictions in Everyday Cognition , 2006, Psychological science.

[38]  Luigi Acerbi,et al.  Internal Representations of Temporal Statistics and Feedback Calibrate Motor-Sensory Interval Timing , 2012, PLoS Comput. Biol..

[39]  Eero P. Simoncelli,et al.  Noise characteristics and prior expectations in human visual speed perception , 2006, Nature Neuroscience.

[40]  I. J. Myung,et al.  The Importance of Complexity in Model Selection. , 2000, Journal of mathematical psychology.

[41]  Amy Perfors,et al.  How do people learn from negative evidence? Non-monotonic generalizations and sampling assumptions in inductive reasoning , 2015, Cognitive Psychology.

[42]  J. Tenenbaum,et al.  Generalization, similarity, and Bayesian inference. , 2001, The Behavioral and brain sciences.

[43]  Harold Pashler,et al.  Optimal Predictions in Everyday Cognition: The Wisdom of Individuals or Crowds? , 2008, Cogn. Sci..

[44]  Craig R. M. McKenzie,et al.  Rational models as theories – not standards – of behavior , 2003, Trends in Cognitive Sciences.

[45]  I. J. Myung,et al.  Toward a method of selecting among computational models of cognition. , 2002, Psychological review.

[46]  J. Tenenbaum,et al.  A tutorial introduction to Bayesian models of cognitive development , 2011, Cognition.

[47]  Jeffrey S. Bowers,et al.  Is that what Bayesians believe? reply to Griffiths, Chater, Norris, and Pouget (2012). , 2012, Psychological bulletin.

[48]  B. Love,et al.  The myth of computational level theory and the vacuity of rational analysis , 2011, Behavioral and Brain Sciences.

[49]  Daniel J. Navarro,et al.  Seeing is believing: Priors, trust, and base rate neglect , 2012 .

[50]  Wei Ji Ma,et al.  A detailed comparison of optimality and simplicity in perceptual decision making. , 2016, Psychological review.

[51]  L. Cosmides,et al.  Are humans good intuitive statisticians after all? Rethinking some conclusions from the literature on judgment under uncertainty , 1996, Cognition.

[52]  D. Wolpert,et al.  Cognitive Tomography Reveals Complex, Task-Independent Mental Representations , 2013, Current Biology.

[53]  J. Tenenbaum,et al.  Infants consider both the sample and the sampling process in inductive generalization , 2010, Proceedings of the National Academy of Sciences.

[54]  Joshua B. Tenenbaum,et al.  Bayesian Models of Inductive Generalization , 2002, NIPS.

[55]  David J. Weiss,et al.  Conservatism in a Simple Probability Inference Task , 2008 .

[56]  Adam N. Sanborn,et al.  Categorization as nonparametric Bayesian density estimation , 2008 .

[57]  Nick Chater,et al.  A rational analysis of the selection task as optimal data selection. , 1994 .

[58]  W. Richards,et al.  Perception as Bayesian Inference , 2008 .

[59]  Ben R. Newell,et al.  Learning and choosing in an uncertain world: An investigation of the explore–exploit dilemma in static and dynamic environments , 2016, Cognitive Psychology.

[60]  Konrad Paul Körding,et al.  The loss function of sensorimotor learning. , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[61]  Browne,et al.  Cross-Validation Methods. , 2000, Journal of mathematical psychology.

[62]  Edward E. Smith,et al.  Category-Based Induction , 1990 .

[63]  T. Griffiths,et al.  Modeling individual differences using Dirichlet processes , 2006 .

[64]  P. Groenen,et al.  Modern Multidimensional Scaling: Theory and Applications , 1999 .

[65]  Michael D. Lee,et al.  Sampling Assumptions in Inductive Generalization , 2012, Cogn. Sci..

[66]  J. Tenenbaum,et al.  Theory-based Bayesian models of inductive learning and reasoning , 2006, Trends in Cognitive Sciences.

[67]  J. Tenenbaum,et al.  Special issue on “Probabilistic models of cognition , 2022 .

[68]  Thomas L. Griffiths,et al.  Probabilistic Topic Models , 2007 .

[69]  J. Tenenbaum,et al.  The learnability of abstract syntactic principles , 2011, Cognition.

[70]  Joseph L. Zinnes,et al.  Theory and Methods of Scaling. , 1958 .

[71]  David Danks,et al.  Rational Analyses, Instrumentalism, and Implementations , 2008 .

[72]  Nick Chater,et al.  The Generalized Universal Law of Generalization , 2001, ArXiv.

[73]  Chris L. Baker,et al.  Action understanding as inverse planning , 2009, Cognition.

[74]  Noah D. Goodman,et al.  Title : The imaginary fundamentalists : The unshocking truth about Bayesian cognitive science , 2011 .

[75]  David M. Riefer,et al.  Multinomial processing models of source monitoring. , 1990 .

[76]  Thomas L. Griffiths,et al.  Latent Features in Similarity Judgments: A Nonparametric Bayesian Approach , 2008, Neural Computation.

[77]  Eleanor Rosch,et al.  Principles of Categorization , 1978 .

[78]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[79]  M. Landy,et al.  It's that time again , 2010, Nature Neuroscience.

[80]  Martyn Plummer,et al.  JAGS: A program for analysis of Bayesian graphical models using Gibbs sampling , 2003 .

[81]  Wasserman,et al.  Bayesian Model Selection and Model Averaging. , 2000, Journal of mathematical psychology.

[82]  Roger N. Shepard,et al.  Additive clustering: Representation of similarities as combinations of discrete overlapping properties. , 1979 .

[83]  Bradley C. Love,et al.  Pinning down the theoretical commitments of Bayesian cognitive models , 2011, Behavioral and Brain Sciences.

[84]  E. Wagenmakers,et al.  Why psychologists must change the way they analyze their data: the case of psi: comment on Bem (2011). , 2011, Journal of personality and social psychology.

[85]  J. Bowers,et al.  Bayesian just-so stories in psychology and neuroscience. , 2012, Psychological bulletin.

[86]  Scott D. Brown,et al.  The power law repealed: The case for an exponential law of practice , 2000, Psychonomic bulletin & review.

[87]  W. Estes The problem of inference from curves based on group data. , 1956, Psychological bulletin.

[88]  Richard M. Shiffrin,et al.  Extending Bayesian induction , 2016 .

[89]  M. Lee,et al.  Modeling individual differences in cognition , 2005, Psychonomic bulletin & review.

[90]  Evan Heit,et al.  A Bayesian Analysis of Some Forms of Inductive Reasoning , 1998 .

[91]  N. Chater,et al.  The probabilistic mind: prospects for Bayesian cognitive science , 2008 .

[92]  Estes Wk The problem of inference from curves based on group data. , 1956 .

[93]  Eero P. Simoncelli,et al.  Cardinal rules: Visual orientation perception reflects knowledge of environmental statistics , 2011, Nature Neuroscience.

[94]  Neil A. Macmillan,et al.  Detection Theory: A User's Guide , 1991 .

[95]  J. Tenenbaum,et al.  Structure and strength in causal induction , 2005, Cognitive Psychology.

[96]  D. Norris,et al.  How the Bayesians got their beliefs (and what those beliefs actually are): comment on Bowers and Davis (2012). , 2012, Psychological bulletin.

[97]  Konrad Paul Kording,et al.  Bayesian integration in sensorimotor learning , 2004, Nature.