The Cost of Accumulating Evidence in Perceptual Decision Making

Decision making often involves the accumulation of information over time, but acquiring information typically comes at a cost. Little is known about the cost incurred by animals and humans for acquiring additional information from sensory variables due, for instance, to attentional efforts. Through a novel integration of diffusion models and dynamic programming, we were able to estimate the cost of making additional observations per unit of time from two monkeys and six humans in a reaction time (RT) random-dot motion discrimination task. Surprisingly, we find that the cost is neither zero nor constant over time, but for the animals and humans features a brief period in which it is constant but increases thereafter. In addition, we show that our theory accurately matches the observed reaction time distributions for each stimulus condition, the time-dependent choice accuracy both conditional on stimulus strength and independent of it, and choice accuracy and mean reaction times as a function of stimulus strength. The theory also correctly predicts that urgency signals in the brain should be independent of the difficulty, or stimulus strength, at each trial.

[1]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[2]  D. M. Green,et al.  Signal detection theory and psychophysics , 1966 .

[3]  David R. Cox,et al.  The Theory of Stochastic Processes , 1967, The Mathematical Gazette.

[4]  Donald Laming,et al.  Information theory of choice-reaction times , 1968 .

[5]  J. Andel Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[6]  H. Akaike A new look at the statistical model identification , 1974 .

[7]  S. Link,et al.  A sequential theory of psychological discrimination , 1975 .

[8]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[9]  D. Vickers Decision processes in visual perception , 1979 .

[10]  J. Elgin The Fokker-Planck Equation: Methods of Solution and Applications , 1984 .

[11]  T. Lai Nearly Optimal Sequential Tests of Composite Hypotheses , 1988 .

[12]  K. H. Britten,et al.  Neuronal correlates of a perceptual decision , 1989, Nature.

[13]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[14]  Dimitri P. Bertsekas,et al.  Dynamic Programming and Optimal Control, Two Volume Set , 1995 .

[15]  D. Dittmar Slice Sampling , 2000 .

[16]  Philip L. Smith,et al.  Stochastic Dynamic Models of Response Time and Accuracy: A Foundational Primer. , 2000, Journal of mathematical psychology.

[17]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[18]  J. Gold,et al.  Banburismus and the Brain Decoding the Relationship between Sensory Stimuli, Decisions, and Reward , 2002, Neuron.

[19]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[20]  Joseph B. Kadane,et al.  A Gridding Method for Bayesian Sequential Decision Problems , 2003 .

[21]  M. Shadlen,et al.  A role for neural integrators in perceptual decision making. , 2003, Cerebral cortex.

[22]  S. Walker Invited comment on the paper "Slice Sampling" by Radford Neal , 2003 .

[23]  Sridhar Mahadevan,et al.  Average reward reinforcement learning: Foundations, algorithms, and empirical results , 2004, Machine Learning.

[24]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[25]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[26]  M. Shadlen,et al.  The effect of stimulus strength on the speed and accuracy of a perceptual decision. , 2005, Journal of vision.

[27]  J. Ditterich Evidence for time‐variant decision making , 2006, The European journal of neuroscience.

[28]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[29]  Michael D. Lee,et al.  A Bayesian Approach to Diffusion Models of Decision-Making and Response Time , 2006, NIPS.

[30]  Jochen Ditterich,et al.  Stochastic models of decisions about motion direction: Behavior and physiology , 2006, Neural Networks.

[31]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[32]  Peter I. Frazier,et al.  Sequential Hypothesis Testing under Stochastic Deadlines , 2007, NIPS.

[33]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[34]  Timothy D. Hanks,et al.  Bounded Integration in Parietal Cortex Underlies Decisions Even When Viewing Duration Is Dictated by the Environment , 2008, The Journal of Neuroscience.

[35]  Michael D. Lee,et al.  A Bayesian approach to diffusion process models of decision-making , 2008 .

[36]  M. Shadlen,et al.  Decision-making with multiple alternatives , 2008, Nature Neuroscience.

[37]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[38]  M. Shadlen,et al.  Representation of Confidence Associated with a Decision by Neurons in the Parietal Cortex , 2009, Science.

[39]  Rajesh P. N. Rao,et al.  Decision Making Under Uncertainty: A Neural Model Based on Partially Observable Markov Decision Processes , 2010, Front. Comput. Neurosci..

[40]  James L. McClelland,et al.  Integration of Sensory and Reward Information during Perceptual Decision-Making in Lateral Intraparietal Cortex (LIP) of the Macaque Monkey , 2010, PloS one.

[41]  Rubén Moreno-Bote,et al.  Decision Confidence and Uncertainty in Diffusion Models with Partially Correlated Neuronal Integrators , 2010, Neural Computation.

[42]  A. Pouget,et al.  Marginalization in Neural Circuits with Divisive Normalization , 2011, The Journal of Neuroscience.

[43]  P. Schrimpf,et al.  Dynamic Programming , 2011 .

[44]  A. Pouget,et al.  Variance as a Signature of Neural Computations during Decision Making , 2011, Neuron.

[45]  Timothy D. Hanks,et al.  Elapsed Decision Time Affects the Weighting of Prior Probability in a Perceptual Decision Task , 2011, The Journal of Neuroscience.

[46]  Lila Davachi,et al.  Persistence of hippocampal multivoxel patterns into postencoding rest is related to memory , 2013, Proceedings of the National Academy of Sciences.