Stochastic Choice and Optimal Sequential Sampling

We model the joint distribution of choice probabilities and decision times in binary choice tasks as the solution to a problem of optimal sequential sampling, where the agent is uncertain of the utility of each action and pays a constant cost per unit time for gathering information. In the resulting optimal policy, the agent's choices are more likely to be correct when the agent chooses to decide quickly, provided that the agent's prior beliefs are correct. For this reason it better matches the observed correlation between decision time and choice probability than does the classical drift-diffusion model, where the agent is uncertain which of two actions is best but knows the utility difference between them.

[1]  Alexander Kukush,et al.  Statistics of stochastic processes , 2010 .

[2]  Jay Lu,et al.  Random Choice and Private Information , 2014 .

[3]  J. Miguel Villas-Boas,et al.  Search for Information on Multiple Products , 2015, Manag. Sci..

[4]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Optimal stopping rules , 1977 .

[5]  P. Moerbeke.  On optimal stopping and free boundary problems , 1973, Advances in Applied Probability.

[6]  E. Fehr,et al.  Neuroeconomic Foundations of Economic Choice—Recent Advances , 2011 .

[7]  A. Pouget,et al.  The Cost of Accumulating Evidence in Perceptual Decision Making , 2012, The Journal of Neuroscience.

[8]  John D. Hey.  Does Repetition Improve Consistency? , 2001 .

[9]  J. Andel.  Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[10]  Donald Laming,et al.  Information theory of choice-reaction times , 1968 .

[11]  Herman Chernoff,et al.  Sequential Tests for the Mean of a Normal Distribution , 1965 .

[12]  Richard G. Swensson,et al.  The elusive tradeoff: Speed vs accuracy in visual discrimination tasks , 1972 .

[13]  W. Edwards.  Optimal strategies for seeking information: Models for statistics, choice reaction times, and human information processing ☆ , 1965 .

[14]  J. Townsend,et al.  Fundamental derivations from decision field theory , 1992 .

[15]  Jörg Rieskamp,et al.  Models of Deferred Decision Making , 2014, CogSci.

[16]  Andrew Caplin,et al.  Search, choice, and revealed preference , 2011 .

[17]  M. Shadlen,et al.  Decision Making as a Window on Cognition , 2013, Neuron.

[18]  R. Ratcliff,et al.  A comparison of macaque behavior and superior colliculus neuronal activity to predictions from models of two-choice decisions. , 2003, Journal of neurophysiology.

[19]  Paulo Natenzon,et al.  Random Choice and Learning , 2019, Journal of Political Economy.

[20]  Rajesh P. N. Rao,et al.  Bayesian brain : probabilistic approaches to neural coding , 2006 .

[21]  N. Wilcox,et al.  Decisions, Error and Heterogeneity , 1997 .

[22]  Thomas L. Griffiths,et al.  One and Done? Optimal Decisions From Very Few Samples , 2014, Cogn. Sci..

[23]  M. Stone.  Models for choice-reaction time , 1960 .

[24]  J. Schall,et al.  Neural Control of Voluntary Movement Initiation , 1996, Science.

[25]  D. Vickers,et al.  Evidence for an accumulator model of psychophysical discrimination. , 1970, Ergonomics.

[26]  Dan Rosen,et al.  An Integrated Market and Credit Risk Portfolio Model , 1999 .

[27]  Anna Popova,et al.  Experimental evidence on rational inattention , 2011 .

[28]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[29]  Christof Koch,et al.  The drift diffusion model can account for value-based choice response times under high and low time pressure , 2010 .

[30]  A. Rubinstein.  Instinctive and Cognitive Reasoning: A Study of Response Times , 2006 .

[31]  M. A. Girshick,et al.  Bayes and minimax solutions of sequential decision problems , 1949 .

[32]  Drew Fudenberg,et al.  Stochastic Choice and Revealed Perturbed Utility , 2015 .

[33]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[34]  Colin Camerer,et al.  The Attentional Drift-Diffusion Model Extends to Simple Purchasing Decisions , 2012, Front. Psychology.

[35]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[36]  Michael N. Shadlen,et al.  The Speed and Accuracy of a Simple Perceptual Decision: A Mathematical Primer. , 2007 .

[37]  Philip L. Smith,et al.  Stochastic Dynamic Models of Response Time and Accuracy: A Foundational Primer. , 2000, Journal of mathematical psychology.

[38]  David G. Rand,et al.  Spontaneous giving and calculated greed , 2012, Nature.

[39]  John D. Hey.  Does Repetition Improve Consistency , 2001 .

[40]  Jerome R. Busemeyer,et al.  Computational Models of Decision Making , 2003 .

[41]  Antonio Rangel,et al.  Combining Response Times and Choice Data Using A Neuroeconomic Model of the Decision Process Improves Out-of-Sample Predictions ∗ , 2013 .

[42]  David I. Laibson,et al.  Bounded Rationality and Directed Cognition , 2005 .

[43]  Eric J. Johnson,et al.  Detecting Failures of Backward Induction: Monitoring Information Search in Sequential Bargaining , 2002, J. Econ. Theory.

[44]  R. Nagel,et al.  Search Dynamics in Consumer Choice under Time Pressure: An Eye-Tracking Study , 2011 .

[45]  Christof Koch,et al.  The Drift Diffusion Model Can Account for the Accuracy and Reaction Time of Value-Based Choices Under High and Low Time Pressure , 2010, Judgment and Decision Making.

[46]  Ernst Fehr,et al.  Rethinking fast and slow based on a critique of reaction-time reverse inference , 2015, Nature Communications.

[47]  Michael Woodford,et al.  An Optimizing Neuroeconomic Model of Discrete Choice , 2014 .

[48]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[49]  S. Link,et al.  A sequential theory of psychological discrimination , 1975 .

[50]  Andrew Caplin,et al.  The Dual‐Process Drift Diffusion Model: Evidence from Response Times , 2016 .

[51]  A. Rangel,et al.  Multialternative drift-diffusion model predicts the relationship between visual fixations and choice in value-based decisions , 2011, Proceedings of the National Academy of Sciences.

[52]  John D. Hey,et al.  Experimental Investigations of Errors in Decision Making under Risk Experimental investigations of errors in decision making under risk , 1995 .

[53]  Miguel A. Costa-Gomes,et al.  Cognition and Behavior in Normal-Form Games: An Experimental Study , 1998 .

[54]  Ian Krajbich,et al.  Visual fixations and the computation and comparison of value in simple choice , 2010, Nature Neuroscience.

[55]  Jochen Ditterich,et al.  Stochastic models of decisions about motion direction: Behavior and physiology , 2006, Neural Networks.

[56]  Peter I. Frazier,et al.  Sequential Hypothesis Testing under Stochastic Deadlines , 2007, NIPS.

[57]  M. Yor,et al.  Continuous martingales and Brownian motion , 1990 .

[58]  Philipp Strack,et al.  Inverse Optimal Stopping , 2015 .

[59]  Dudley,et al.  Real Analysis and Probability: Measurability: Borel Isomorphism and Analytic Sets , 2002 .

[60]  Goran Peskir,et al.  On Integral Equations Arising in the First-Passage Problem for Brownian Motion , 2002 .

[61]  Colin Camerer.  An experimental test of several generalized utility theories , 1989 .

[62]  Philipp Strack,et al.  Optimal Stopping with Private Information , 2014, J. Econ. Theory.

[63]  J. A. Bather,et al.  Bayes procedures for deciding the Sign of a normal mean , 1962 .

[64]  Tze Leung Lai,et al.  Optimal stopping for Brownian motion with applications to sequential analysis and option pricing , 2003 .

[65]  M. Shadlen,et al.  Decision-making with multiple alternatives , 2008, Nature Neuroscience.

[66]  J. Miguel Villas-Boas,et al.  Optimal Search for Product Information , 2010, Manag. Sci..

[67]  Albert N. Shiryaev,et al.  Optimal Stopping Rules , 2011, International Encyclopedia of Statistical Science.