Stochastic models of evidence accumulation in changing environments

Organisms and ecological groups accumulate evidence to make decisions. Classic experiments and theoretical studies have explored this process when the correct choice is fixed during each trial. However, we live in a constantly changing world. What effect does such impermanence have on classical results about decision making? To address this question we use sequential analysis to derive a tractable model of evidence accumulation when the correct option changes in time. Our analysis shows that ideal observers discount prior evidence at a rate determined by the volatility of the environment, and the dynamics of evidence accumulation is governed by the information gained over an average environmental epoch. A plausible neural implementation of an optimal observer in a changing environment shows that, in contrast to previous models, neural populations representing alternate choices are coupled through excitation. Our work builds a bridge between statistical decision making in volatile environments and stochast...

[1]  R. K. Simpson Nature Neuroscience , 2022 .

[2]  Jonathan W. Pillow,et al.  Single-trial spike trains in parietal cortex reveal discrete steps during decision-making , 2015, Science.

[3]  Alʹbert Nikolaevich Shiri︠a︡ev,et al.  Optimal stopping rules , 1977 .

[4]  Robert C. Wilson,et al.  Rational regulation of learning dynamics by pupil–linked arousal systems , 2012, Nature Neuroscience.

[5]  KongFatt Wong-Lin,et al.  Neural Circuit Dynamics Underlying Accumulation of Time-Varying Evidence During Perceptual Decision Making , 2007, Frontiers Comput. Neurosci..

[6]  W. Newsome,et al.  Matching Behavior and the Representation of Value in the Parietal Cortex , 2004, Science.

[7]  Tim Kovacs,et al.  On optimal decision-making in brains and social insect colonies , 2009, Journal of The Royal Society Interface.

[8]  Sophie Denève,et al.  Bayesian Spiking Neurons I: Inference , 2008, Neural Computation.

[9]  J. Gold,et al.  Banburismus and the Brain Decoding the Relationship between Sensory Stimuli, Decisions, and Reward , 2002, Neuron.

[10]  S. Swain Handbook of Stochastic Methods for Physics, Chemistry and the Natural Sciences , 1984 .

[11]  J. Zabczyk,et al.  Stochastic Equations in Infinite Dimensions , 2008 .

[12]  R. Rosenfeld Nature , 2009, Otolaryngology--head and neck surgery : official journal of American Academy of Otolaryngology-Head and Neck Surgery.

[13]  Ryan P. Adams,et al.  Bayesian Online Changepoint Detection , 2007, 0710.3742.

[14]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[15]  P. Holmes,et al.  The dynamics of choice among multiple alternatives , 2006 .

[16]  I. Couzin,et al.  Effective leadership and decision-making in animal groups on the move , 2005, Nature.

[17]  Philip L. Smith,et al.  Psychology and neurobiology of simple decisions , 2004, Trends in Neurosciences.

[18]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[19]  W. Marsden I and J , 2012 .

[20]  Angela J. Yu Optimal Change-Detection and Spiking Neurons , 2006, NIPS.

[21]  R. Cabeza,et al.  Frontiers in Human Neuroscience , 2009 .

[22]  J. Brown Behavioral Ecology and Sociobiology , 2019, Encyclopedia of Animal Behavior.

[23]  T. Seeley,et al.  Collective decision-making in honey bees: how colonies choose among nectar sources , 1991, Behavioral Ecology and Sociobiology.

[24]  Xiao-Jing Wang,et al.  Probabilistic Decision Making by Slow Reverberation in Cortical Circuits , 2002, Neuron.

[25]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[26]  M. V. Rossum,et al.  In Neural Computation , 2022 .

[27]  Venugopal V. Veeravalli,et al.  Multihypothesis sequential probability ratio tests - Part I: Asymptotic optimality , 1999, IEEE Trans. Inf. Theory.

[28]  J. Gillis,et al.  Probability and Related Topics in Physical Sciences , 1960 .

[29]  E. S. Pearson,et al.  On the Problem of the Most Efficient Tests of Statistical Hypotheses , 1933 .

[30]  Kevin N. Gurney,et al.  The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions , 2007, Neural Computation.

[31]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[32]  Christian Kuehn,et al.  Large Deviations for Nonlocal Stochastic Neural Fields , 2013, Journal of mathematical neuroscience.

[33]  Richard A. Silverman,et al.  Probability Theory: A Concise Course , 1977 .

[34]  Marius Usher,et al.  Extending a biologically inspired model of choice: multi-alternatives, nonlinearity and value-based multidimensional choice , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[35]  Michael I. Jordan,et al.  Advances in Neural Information Processing Systems 30 , 1995 .

[36]  Stefan J. Kiebel,et al.  Perceptual decision making: drift-diffusion model is equivalent to a Bayesian model , 2014, Front. Hum. Neurosci..

[37]  Robert B. Ash,et al.  Information Theory , 2020, The SAGE International Encyclopedia of Mass Media and Society.

[38]  S. Grossberg,et al.  Psychological Review , 2003 .

[39]  P. Laguna,et al.  Signal Processing , 2002, Yearbook of Medical Informatics.

[40]  K. Vahala Handbook of stochastic methods for physics, chemistry and the natural sciences , 1986, IEEE Journal of Quantum Electronics.

[41]  Albert N. Shiryaev,et al.  Optimal Stopping Rules , 2011, International Encyclopedia of Statistical Science.

[42]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[43]  Ranulfo Romo,et al.  Flexible Control of Mutual Inhibition: A Neural Model of Two-Interval Discrimination , 2005, Science.

[44]  O. Faugeras,et al.  Stochastic neural field equations: a rigorous footing , 2013, Journal of mathematical biology.

[45]  Constant D. Beugré,et al.  The neural basis of decision making , 2018 .

[46]  Joshua I. Gold,et al.  Bayesian Online Learning of the Hazard Rate in Change-Point Problems , 2010, Neural Computation.

[47]  Bingni W. Brunton,et al.  Rats and Humans Can Optimally Accumulate Evidence for Decision-Making , 2013, Science.

[48]  N. Franks,et al.  A Mechanism for Value-Sensitive Decision-Making , 2013, PloS one.

[49]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[50]  James L. McClelland,et al.  Loss aversion and inhibition in dynamical models of multialternative choice. , 2004, Psychological review.

[51]  Thomas M. Cover,et al.  Elements of Information Theory , 2005 .

[52]  L. Christophorou Science , 2018, Emerging Dynamics: Science, Energy, Society and Values.

[53]  A. Oudenaarden,et al.  Cellular Decision Making and Biological Noise: From Microbes to Mammals , 2011, Cell.

[54]  Vaibhav Srivastava,et al.  A Theory of Decision Making Under Dynamic Context , 2015, NIPS.

[55]  J. Townsend,et al.  Decision field theory: a dynamic-cognitive approach to decision making in an uncertain environment. , 1993, Psychological review.

[56]  P. Armitage Sequential Analysis with More than Two Alternative Hypotheses, and its Relation to Discriminant Function Analysis , 1950 .

[57]  T. Seeley,et al.  Swarm cognition in honey bees , 2007, Behavioral Ecology and Sociobiology.

[58]  Qing Zhao,et al.  Quickest Detection in Multiple On–Off Processes , 2010, IEEE Transactions on Signal Processing.

[59]  J. Norris Appendix: probability and measure , 1997 .

[60]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[61]  Domenici,et al.  The kinematics and performance of fish fast-start swimming , 1997, The Journal of experimental biology.

[62]  Joseph W Kable,et al.  Normative evidence accumulation in unpredictable environments , 2015, eLife.

[63]  L. Tippett,et al.  Applied Statistics. A Journal of the Royal Statistical Society , 1952 .

[64]  L. Goddard Information Theory , 1962, Nature.

[65]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[66]  D. Wilkin,et al.  Neuron , 2001, Brain Research.

[67]  Philip Holmes,et al.  Can Monkeys Choose Optimally When Faced with Noisy Stimuli and Unequal Rewards? , 2009, PLoS Comput. Biol..

[68]  Joseph T. McGuire,et al.  Functionally Dissociable Influences on Learning Rate in a Dynamic Environment , 2014, Neuron.