The Perception of Probability

We present a computational model to explain the results from experiments in which subjects estimate the hidden probability parameter of a stepwise nonstationary Bernoulli process outcome by outcome. The model captures the following results qualitatively and quantitatively, with only 2 free parameters: (a) Subjects do not update their estimate after each outcome; they step from one estimate to another at irregular intervals. (b) The joint distribution of step widths and heights cannot be explained on the assumption that a threshold amount of change must be exceeded in order for them to indicate a change in their perception. (c) The mapping of observed probability to the median perceived probability is the identity function over the full range of probabilities. (d) Precision (how close estimates are to the best possible estimate) is good and constant over the full range. (e) Subjects quickly detect substantial changes in the hidden probability parameter. (f) The perceived probability sometimes changes dramatically from one observation to the next. (g) Subjects sometimes have second thoughts about a previous change perception, after observing further outcomes. (h) The frequency with which they perceive changes moves in the direction of the true frequency over sessions. (Explaining this finding requires 2 additional parametric assumptions.) The model treats the perception of the current probability as a by-product of the construction of a compact encoding of the experienced sequence in terms of its change points. It illustrates the why and the how of intermittent Bayesian belief updating and retrospective revision in simple perception. It suggests a reinterpretation of findings in the recent literature on the neurobiology of decision making.

[1]  R. Hertwig.  Decisions from Experience , 2015 .

[2]  William K. Estes,et al.  Probability Learning and Sequence Learning , 2014 .

[3]  Joshua I. Gold,et al.  A Healthy Fear of the Unknown: Perspectives on the Interpretation of Parameter Fits from Computational Models in Neuroscience , 2013, PLoS Comput. Biol..

[4]  C. Gallistel,et al.  Mice take calculated risks , 2012, Proceedings of the National Academy of Sciences.

[5]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[6]  J. Bowers,et al.  Bayesian just-so stories in psychology and neuroscience. , 2012, Psychological bulletin.

[7]  M. Carandini.  From circuits to behavior: a bridge too far? , 2012, Nature Neuroscience.

[8]  Hang Zhang,et al.  Ubiquitous Log Odds: A Common Representation of Probability and Frequency Distortion in Perception, Action, and Cognition , 2012, Front. Neurosci..

[9]  Aleksey S. Polunchenko,et al.  State-of-the-Art in Sequential Change-Point Detection , 2011, 1109.2938.

[10]  Jonathan D. Cohen,et al.  Optimal Temporal Risk Assessment , 2011, Front. Integr. Neurosci..

[11]  Karl J. Friston,et al.  A Bayesian Foundation for Individual Learning Under Uncertainty , 2011, Front. Hum. Neurosci..

[12]  Robert C. Wilson,et al.  An Approximately Bayesian Delta-Rule Model Explains the Dynamics of Belief Updating in a Changing Environment , 2010, The Journal of Neuroscience.

[13]  Joshua I. Gold,et al.  Bayesian Online Learning of the Hazard Rate in Change-Point Problems , 2010, Neural Computation.

[14]  Peter N. C. Mohr,et al.  Genetic variation in dopaminergic neuromodulation influences the ability to rapidly and flexibly adapt decisions , 2009, Proceedings of the National Academy of Sciences.

[15]  J. Feldman,et al.  Bayes and the Simplicity Principle in Perception Simplicity versus Likelihood Principles in Perception , 2022 .

[16]  Craig R. Fox,et al.  Information asymmetry in decision from description versus decision from experience , 2009, Judgment and Decision Making.

[17]  Y. Niv.  Reinforcement learning in the brain , 2009 .

[18]  L. Maloney,et al.  Economic decision-making compared with an equivalent motor task , 2009, Proceedings of the National Academy of Sciences.

[19]  N. Chater,et al.  Are Probabilities Overweighted or Underweighted When Rare Outcomes Are Experienced (Rarely)? , 2009, Psychological science.

[20]  C. Gallistel,et al.  The Importance of Proving the Null , 2022 .

[21]  C. Gallistel,et al.  Risk assessment in man and mouse , 2009, Proceedings of the National Academy of Sciences.

[22]  Scott D. Brown,et al.  Detecting and predicting changes , 2009, Cognitive Psychology.

[23]  P. Glimcher.  Choice: Towards a Standard Back-pocket Model , 2009 .

[24]  P. Dayan,et al.  Decision theory, reinforcement learning, and the brain , 2008, Cognitive, affective & behavioral neuroscience.

[25]  Colin Camerer,et al.  A framework for studying the neurobiology of value-based decision making , 2008, Nature Reviews Neuroscience.

[26]  P. Dayan,et al.  Reinforcement learning: The Good, The Bad and The Ugly , 2008, Current Opinion in Neurobiology.

[27]  Colin Camerer,et al.  Neuroeconomics: decision making and the brain , 2008 .

[28]  J. Cacioppo,et al.  Handbook Of Psychophysiology , 2019 .

[29]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.

[30]  C. Gallistel,et al.  Nonverbal arithmetic in humans: Light from noise , 2007, Perception & psychophysics.

[31]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[32]  P. Fearnhead,et al.  On‐line inference for multiple changepoint problems , 2007 .

[33]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[34]  K. Preuschoff,et al.  Adding Prediction Risk to the Theory of Reward Learning , 2007, Annals of the New York Academy of Sciences.

[35]  C. Gallistel,et al.  Is matching innate? , 2007, Journal of the experimental analysis of behavior.

[36]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[37]  Craig R. Fox,et al.  “Decisions from experience” = sampling error + prospect theory: Reconsidering Hertwig, Barron, Weber & Erev (2004) , 2006, Judgment and Decision Making.

[38]  Aaron C. Courville,et al.  Bayesian theories of conditioning in a changing world , 2006, Trends in Cognitive Sciences.

[39]  W. Schultz.  Behavioral theories and the neurophysiology of reward. , 2006, Annual review of psychology.

[40]  Scott D. Brown,et al.  Prediction and Change Detection , 2005, NIPS.

[41]  H. Seung,et al.  JOURNAL OF THE EXPERIMENTAL ANALYSIS OF BEHAVIOR 2005, 84, 581–617 NUMBER 3(NOVEMBER) LINEAR-NONLINEAR-POISSON MODELS OF PRIMATE CHOICE DYNAMICS , 2022 .

[42]  W. Estes,et al.  Risks of drawing inferences about cognitive processes from model fits to individual versus average performance , 2005, Psychonomic bulletin & review.

[43]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[44]  W. Newsome,et al.  Choosing the greater of two goods: neural currencies for valuation and decision making , 2005, Nature Reviews Neuroscience.

[45]  R. Hertwig,et al.  Decisions from Experience and the Effect of Rare Events in Risky Choice , 2004, Psychological science.

[46]  Peter Dayan,et al.  Temporal difference models describe higher-order learning in humans , 2004, Nature.

[47]  Corey J Bohil,et al.  Probability matching, accuracy maximization, and a test of the optimal classifier’s independence assumption in perceptual categorization , 2004, Perception & psychophysics.

[48]  David J. C. MacKay,et al.  Information Theory, Inference, and Learning Algorithms , 2004, IEEE Transactions on Information Theory.

[49]  David S. Touretzky,et al.  Model Uncertainty in Classical Conditioning , 2003, NIPS.

[50]  P. Glimcher.  The neurobiology of visual-saccadic decision making. , 2003, Annual review of neuroscience.

[51]  W. Schultz,et al.  Discrete Coding of Reward Probability and Uncertainty by Dopamine Neurons , 2003, Science.

[52]  P. Glimcher.  Decisions, Uncertainty, and the Brain: The Science of Neuroeconomics , 2003 .

[53]  W. Estes,et al.  Traps in the route to models of memory and decision , 2002, Psychonomic bulletin & review.

[54]  S. Kakade,et al.  Acquisition and extinction in autoshaping. , 2002, Psychological review.

[55]  Rochel Gelman,et al.  Variability signatures distinguish verbal from nonverbal counting for both large and small numbers , 2001, Psychonomic bulletin & review.

[56]  C. Gallistel,et al.  The rat approximates an ideal detector of changes in rates of reward: implications for the law of effect. , 2001, Journal of experimental psychology. Animal behavior processes.

[57]  Refractor.  Vision , 2000, The Lancet.

[58]  Nir Vulkan.  An Economist's Perspective on Probability Matching , 2000 .

[59]  R. Luce.  Utility of Gains and Losses: Measurement-Theoretical and Experimental Approaches , 2000 .

[60]  A. Dickinson,et al.  Neuronal coding of prediction errors. , 2000, Annual review of neuroscience.

[61]  Richard Gonzalez,et al.  On the Shape of the Probability Weighting Function , 1999, Cognitive Psychology.

[62]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[63]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[64]  P. Dayan,et al.  A framework for mesencephalic dopamine systems based on predictive Hebbian learning , 1996, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[65]  Dieter Vaitl,et al.  Interoception , 1996, Biological Psychology.

[66]  G. Gigerenzer.  Why the distinction between single-event probabilities and frequencies is important for psychology (and vice versa). , 1994 .

[67]  B. Dworkin.  Learning and Physiological Regulation , 1993 .

[68]  W. Estes,et al.  Global and local control of choice behavior by cyclically varying outcome probabilities. , 1984, Journal of experimental psychology. Learning, memory, and cognition.

[69]  David V. Budescu,et al.  Encoding subjective probabilities: A psychological and psychometric review , 1983 .

[70]  J. B. Brooke,et al.  Error patterns in the judgment and production of numerical proportions , 1977 .

[71]  T. Wallsten,et al.  Individual Decision Behavior , 1972 .

[72]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[73]  L. Beach,et al.  Man as an Intuitive Statistician , 2022 .

[74]  G. H. Robinson.  CONTINUOUS ESTIMATION OF A TIME-VARYING PROBABILITY , 1964 .

[75]  W. Edwards.  Behavioral decision theory. , 1961, Annual review of psychology.

[76]  W. K. Estes,et al.  Theory of learning with constant, variable, or contingent probabilities of reinforcement , 1957 .

[77]  Estes Wk.  The problem of inference from curves based on group data. , 1956 .

[78]  W. Estes.  The problem of inference from curves based on group data. , 1956, Psychological bulletin.

[79]  F. Attneave.  Psychological probability as a function of experienced frequency. , 1953, Journal of experimental psychology.

[80]  N. Campbell,et al.  Scientific Inference , 1931, Nature.