Stochastic satisficing account of confidence in uncertain value-based decisions

Every day we make choices under uncertainty; choosing what route to work or which queue in a supermarket to take, for example. It is unclear how outcome variance, e.g. waiting time in a queue, affects decisions when outcome is stochastic and continuous. For example, how does one choose between an option with unreliable but high expected reward, and an option with more certain but lower expected reward? Here we used an experimental design where two choices’ payoffs took continuous values, to examine the effect of outcome variance on decisions and confidence. Inconsistent with expected utility predictions, our participants’ probability of choosing the good option decreased when both better and worse options’ payoffs were more variable. Confidence ratings were affected by outcome variability only when choosing the good option. Inspired by the satisficing heuristic, we propose a “stochastic satisficing” (SSAT) model for choosing between options with continuous uncertain outcomes. In this model, decisions are made by comparing the available options’ probability of exceeding an acceptability threshold and confidence reports scale with the chosen option’s satisficing probability. The SSAT model best explained choice behaviour and most successfully predicted confidence ratings. We further tested the model’s prediction in a second experiment where choice and confidence behaviours were found to be consistent with the SSAT simulations. Our model and experimental results generalize the cognitive heuristic of satisficing to stochastic contexts and thus provide an account of bounded rationality in the face of uncertainty.

[1]  Geraint Rees,et al.  Relating Introspective Accuracy to Individual Differences in Brain Structure , 2010, Science.

[2]  J. Kruschke Bayesian estimation supersedes the t test. , 2013, Journal of experimental psychology. General.

[3]  Vaibhav Srivastava,et al.  Modeling Human Decision Making in Generalized Gaussian Multiarmed Bandits , 2013, Proceedings of the IEEE.

[4]  H. Simon,et al.  Rational choice and the structure of the environment. , 1956, Psychological review.

[5]  Sidney G. Winter,et al.  The satisficing principle in capability learning , 2000 .

[6]  J. Kruschke Chapter 8 – JAGS , 2015 .

[7]  Alessandro Lazaric,et al.  Risk-Aversion in Multi-armed Bandits , 2012, NIPS.

[8]  Pei-Ju Liao,et al.  Macroeconomic Theory I , 2012 .

[9]  P. Tetlock,et al.  Accounting for the effects of accountability. , 1999, Psychological bulletin.

[10]  P. Latham,et al.  The idiosyncratic nature of confidence , 2017, Nature Human Behaviour.

[11]  P. Glimcher Indeterminacy in brain and behavior. , 2005, Annual review of psychology.

[12]  Bahador Bahrami,et al.  Post-decisional accounts of biases in confidence , 2016, Current Opinion in Behavioral Sciences.

[13]  Naomi Ehrich Leonard,et al.  Satisficing in Gaussian bandit problems , 2014, 53rd IEEE Conference on Decision and Control.

[14]  R. Dolan,et al.  Prefrontal Contributions to Metacognition in Perceptual Decision Making , 2012, The Journal of Neuroscience.

[15]  Adam N. Sanborn,et al.  Bayesian Brains without Probabilities , 2016, Trends in Cognitive Sciences.

[16]  Mark W Woolrich,et al.  Associative learning of social value , 2008, Nature.

[17]  A. Tversky,et al.  Judgment under Uncertainty: Heuristics and Biases , 1974, Science.

[18]  A. Tversky,et al.  Prospect theory: an analysis of decision under risk — Source link , 2007 .

[19]  Jon Driver,et al.  A Regret-Induced Status Quo Bias , 2011, The Journal of Neuroscience.

[20]  Sumio Watanabe,et al.  Asymptotic Equivalence of Bayes Cross Validation and Widely Applicable Information Criterion in Singular Learning Theory , 2010, J. Mach. Learn. Res..

[21]  Antonio Pietrabissa,et al.  A distributed multi-path algorithm for wireless ad-hoc networks based on Wardrop routing , 2013, 21st Mediterranean Conference on Control and Automation.

[22]  W. F. Prokasy,et al.  Classical conditioning II: Current research and theory. , 1972 .

[23]  John K. Kruschke,et al.  Doing Bayesian Data Analysis: A Tutorial with R, JAGS, and Stan , 2014 .

[24]  Florent Meyniel,et al.  The Sense of Confidence during Probabilistic Learning: A Normative Account , 2015, PLoS Comput. Biol..

[25]  Vaibhav Srivastava,et al.  Satisficing in Multi-Armed Bandit Problems , 2015, IEEE Transactions on Automatic Control.

[26]  Nicole Propst,et al.  Classical Conditioning Ii Current Research And Theory , 2016 .

[27]  Todd M. Gureckis,et al.  CUNY Academic , 2016 .

[28]  V. Michel,et al.  An Automatic Valuation System in the Human Brain: Evidence from Functional Neuroimaging , 2009, Neuron.

[29]  J. Daunizeau,et al.  Automatic integration of confidence in the brain valuation signal , 2015, Nature Neuroscience.

[30]  Joshua I. Sanders,et al.  Signatures of a Statistical Computation in the Human Sense of Confidence , 2016, Neuron.

[31]  Jiqiang Guo,et al.  Stan: A Probabilistic Programming Language. , 2017, Journal of statistical software.

[32]  William Samuelson,et al.  Status quo bias in decision making , 1988 .

[33]  I. Erev,et al.  On adaptation, maximization, and reinforcement learning among cognitive strategies. , 2005, Psychological review.

[34]  Dan Bang,et al.  Making better decisions in groups , 2017, Royal Society Open Science.

[35]  Alexandre Pouget,et al.  Confidence and certainty: distinct probabilistic quantities for different goals , 2016, Nature Neuroscience.

[36]  R. Rescorla,et al.  A theory of Pavlovian conditioning : Variations in the effectiveness of reinforcement and nonreinforcement , 1972 .

[37]  P. Dayan,et al.  Neural Prediction Errors Reveal a Risk-Sensitive Reinforcement-Learning Process in the Human Brain , 2012, The Journal of Neuroscience.

[38]  H. Simon,et al.  Administrative Behavior: A Study of Decision-Making Processes in Administrative Organization. , 1959 .

[39]  H. Simon Models of Bounded Rationality: Empirically Grounded Economic Reason , 1997 .

[40]  J. Neumann,et al.  Theory of games and economic behavior , 1945, 100 Years of Math Milestones.

[41]  Sidney G. Winter,et al.  Satisficing, Selection, and the Innovating Remnant , 1971 .

[42]  Ulrike Goldschmidt Advances In Behavioral Economics , 2016 .

[43]  A. Tversky,et al.  Prospect theory: analysis of decision under risk , 1979 .

[44]  R. Dolan,et al.  Confidence in value-based choice , 2012, Nature Neuroscience.

[45]  Raymond J. Dolan,et al.  Deconstructing risk: Separable encoding of variance and skewness in the brain , 2011, NeuroImage.

[46]  Christopher Summerfield,et al.  Metacognition in human decision-making: confidence and error monitoring , 2012, Philosophical Transactions of the Royal Society B: Biological Sciences.

[47]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[48]  Wei Ji Ma,et al.  Neural coding of uncertainty and probability. , 2014, Annual review of neuroscience.

[49]  C. Heyes,et al.  Supra-personal cognitive control and metacognition , 2014, Trends in Cognitive Sciences.