Optimal Decision Making on the Basis of Evidence Represented in Spike Trains

Experimental data indicate that perceptual decision making involves integration of sensory evidence in certain cortical areas. Theoretical studies have proposed that the computation in neural decision circuits approximates statistically optimal decision procedures (e.g., sequential probability ratio test) that maximize the reward rate in sequential choice tasks. However, these previous studies assumed that the sensory evidence was represented by continuous values from gaussian distributions with the same variance across alternatives. In this article, we make a more realistic assumption that sensory evidence is represented in spike trains described by the Poisson processes, which naturally satisfy the mean-variance relationship observed in sensory neurons. We show that for such a representation, the neural circuits involving cortical integrators and basal ganglia can approximate the optimal decision procedures for two and multiple alternative choice tasks.

[1]  H. Saunders,et al.  Probability, Random Variables and Stochastic Processes (2nd Edition) , 1989 .

[2]  Venugopal V. Veeravalli,et al.  Multihypothesis sequential probability ratio tests - Part I: Asymptotic optimality , 1999, IEEE Trans. Inf. Theory.

[3]  A. Pike Stochastic models of choice behaviour: response probabilities and latencies of finite Markov chain systems. , 1966, The British journal of mathematical and statistical psychology.

[4]  J. Movshon,et al.  The statistical reliability of signals in single neurons in cat and monkey visual cortex , 1983, Vision Research.

[5]  T. Zandt,et al.  Time-dependent Poisson counter models of response latency in simple judgment. , 2000, The British journal of mathematical and statistical psychology.

[6]  A. Diederich,et al.  Modeling the effects of payoff on response bias in a perceptual discrimination task: Bound-change, drift-rate-change, or two-stage-processing hypothesis , 2006, Perception & psychophysics.

[7]  Wei Ji Ma,et al.  Bayesian inference with probabilistic population codes , 2006, Nature Neuroscience.

[8]  D C Van Essen,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. I. Selectivity for stimulus direction, speed, and orientation. , 1983, Journal of neurophysiology.

[9]  S. Treue,et al.  The response of neurons in areas V1 and MT of the alert rhesus monkey to moving random dot patterns , 2005, Experimental Brain Research.

[10]  Timothy D. Hanks,et al.  Probabilistic Population Codes for Bayesian Decision Making , 2008, Neuron.

[11]  Roger Ratcliff,et al.  The Diffusion Decision Model: Theory and Data for Two-Choice Decision Tasks , 2008, Neural Computation.

[12]  R Ratcliff,et al.  Continuous versus discrete information processing modeling accumulation of partial information. , 1988, Psychological review.

[13]  J. Wolfowitz,et al.  Optimum Character of the Sequential Probability Ratio Test , 1948 .

[14]  Xiao-Jing Wang,et al.  A Recurrent Network Mechanism of Time Integration in Perceptual Decisions , 2006, The Journal of Neuroscience.

[15]  Venugopal V. Veeravalli,et al.  A sequential procedure for multihypothesis testing , 1994, IEEE Trans. Inf. Theory.

[16]  Philip Holmes,et al.  Rapid decision threshold modulation by reward rate in a neural network , 2006, Neural Networks.

[17]  M. Shadlen,et al.  A role for neural integrators in perceptual decision making. , 2003, Cerebral cortex.

[18]  Philip L. Smith,et al.  A comparison of sequential sampling models for two-choice reaction time. , 2004, Psychological review.

[19]  J. Andel Sequential Analysis , 2022, The SAGE Encyclopedia of Research Design.

[20]  Donald Laming,et al.  Information theory of choice-reaction times , 1968 .

[21]  G. A. Barnard,et al.  Sequential Tests in Industrial Statistics , 1946 .

[22]  Jonathan D. Cohen,et al.  The physics of optimal decision making: a formal analysis of models of performance in two-alternative forced-choice tasks. , 2006, Psychological review.

[23]  O. L. Smith,et al.  ON THE CUMULANTS OF RENEWAL PROCESSES , 1959 .

[24]  W. Newsome,et al.  A selective impairment of motion perception following lesions of the middle temporal visual area (MT) , 1988, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[25]  J. Movshon,et al.  The analysis of visual motion: a comparison of neuronal and psychophysical performance , 1992, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[26]  M. Shadlen,et al.  Neural correlates of a decision in the dorsolateral prefrontal cortex of the macaque , 1999, Nature Neuroscience.

[27]  K. R. Ridderinkhof,et al.  Striatum and pre-SMA facilitate decision-making under time pressure , 2008, Proceedings of the National Academy of Sciences.

[28]  Ehud Zohary,et al.  Correlated neuronal discharge rate and its implications for psychophysical performance , 1994, Nature.

[29]  J. Kenney Mathematics of statistics , 1940 .

[30]  R. Duncan Luce,et al.  Response Times: Their Role in Inferring Elementary Mental Organization , 1986 .

[31]  Rafal Bogacz,et al.  Optimal decision-making theories , 2009 .

[32]  Venugopal V. Veeravalli,et al.  Multihypothesis sequential probability ratio tests - Part II: Accurate asymptotic expansions for the expected sample size , 2000, IEEE Trans. Inf. Theory.

[33]  J. Gold,et al.  The neural basis of decision making. , 2007, Annual review of neuroscience.

[34]  T. Prescott,et al.  Modelling Natural Action Selection: Index , 2011 .

[35]  Charles J. Wilson,et al.  A model of reverse spike frequency adaptation and repetitive firing of subthalamic nucleus neurons. , 2004, Journal of neurophysiology.

[36]  A. Destexhe,et al.  Synaptic background activity controls spike transfer from thalamus to cortex , 2005, Nature Neuroscience.

[37]  Jonathan D. Cohen,et al.  The Quarterly Journal of Experimental Psychology Do Humans Produce the Speed–accuracy Trade-off That Maximizes Reward Rate? , 2022 .

[38]  Michael N. Shadlen,et al.  The Speed and Accuracy of a Simple Perceptual Decision: A Mathematical Primer. , 2007 .

[39]  J. Gold,et al.  Banburismus and the Brain Decoding the Relationship between Sensory Stimuli, Decisions, and Reward , 2002, Neuron.

[40]  K. H. Britten,et al.  Responses of neurons in macaque MT to stochastic motion signals , 1993, Visual Neuroscience.

[41]  Timothy D. Hanks,et al.  Bounded Integration in Parietal Cortex Underlies Decisions Even When Viewing Duration Is Dictated by the Environment , 2008, The Journal of Neuroscience.

[42]  John H. R. Maunsell,et al.  Functional properties of neurons in middle temporal visual area of the macaque monkey. II. Binocular interactions and sensitivity to binocular disparity. , 1983, Journal of neurophysiology.

[43]  W. Newsome,et al.  The Variable Discharge of Cortical Neurons: Implications for Connectivity, Computation, and Information Coding , 1998, The Journal of Neuroscience.

[44]  Jochen Ditterich,et al.  Perceptual Decisions between Multiple Directions of Visual Motion , 2008, The Journal of Neuroscience.

[45]  Marius Usher,et al.  Extending a biologically inspired model of choice: multi-alternatives, nonlinearity and value-based multidimensional choice , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[46]  William R. Softky,et al.  The highly irregular firing of cortical cells is inconsistent with temporal integration of random EPSPs , 1993, The Journal of neuroscience : the official journal of the Society for Neuroscience.

[47]  D. Vickers,et al.  Evidence for an accumulator model of psychophysical discrimination. , 1970, Ergonomics.

[48]  James L. McClelland,et al.  The time course of perceptual choice: the leaky, competing accumulator model. , 2001, Psychological review.

[49]  M. Shadlen,et al.  Decision-making with multiple alternatives , 2008, Nature Neuroscience.

[50]  Athanasios Papoulis,et al.  Probability, Random Variables and Stochastic Processes , 1965 .

[51]  Kevin N. Gurney,et al.  The Basal Ganglia and Cortex Implement Optimal Decision Making Between Alternative Actions , 2007, Neural Computation.

[52]  Roger Ratcliff,et al.  A Theory of Memory Retrieval. , 1978 .

[53]  Niels Taatgen,et al.  RACE/A: An Architectural Account of the Interactions Between Learning, Task Control, and Retrieval Dynamics , 2012, Cogn. Sci..

[54]  Anthony J. Movshon,et al.  Optimal representation of sensory information by neural populations , 2006, Nature Neuroscience.

[55]  Modelling Natural Action Selection: Extending a biologically inspired model of choice: multi-alternatives, nonlinearity, and value-based multidimensional choice , 2011 .

[56]  S. Zeki The response properties of cells in the middle temporal area (area MT) of owl monkey visual cortex , 1980, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[57]  Rajesh P. N. Rao,et al.  Bayesian brain : probabilistic approaches to neural coding , 2006 .

[58]  Charles J. Wilson,et al.  Apamin-Sensitive Small Conductance Calcium-Activated Potassium Channels, through their Selective Coupling to Voltage-Gated Calcium Channels, Are Critical Determinants of the Precision, Pace, and Pattern of Action Potential Generation in Rat Subthalamic Nucleus Neurons In Vitro , 2003, The Journal of Neuroscience.

[59]  Jeffrey D. Schall,et al.  Neural basis of deciding, choosing and acting , 2001, Nature Reviews Neuroscience.

[60]  J. Kenney,et al.  Mathematics of statistics , 1940 .

[61]  P. Holmes,et al.  The dynamics of choice among multiple alternatives , 2006 .

[62]  Philip Holmes,et al.  Simple Neural Networks that Optimize Decisions , 2005, Int. J. Bifurc. Chaos.

[63]  Jonathan D. Cohen,et al.  Reward rate optimization in two-alternative decision making: empirical tests of theoretical predictions. , 2009, Journal of experimental psychology. Human perception and performance.

[64]  Rafal Bogacz,et al.  Integration of Reinforcement Learning and Optimal Decision-Making Theories of the Basal Ganglia , 2011, Neural Computation.

[65]  M. Shadlen,et al.  Response of Neurons in the Lateral Intraparietal Area during a Combined Visual Discrimination Reaction Time Task , 2002, The Journal of Neuroscience.

[66]  J. Gold,et al.  Neural computations that underlie decisions about sensory stimuli , 2001, Trends in Cognitive Sciences.

[67]  W. Newsome,et al.  Neural basis of a perceptual decision in the parietal cortex (area LIP) of the rhesus monkey. , 2001, Journal of neurophysiology.

[68]  Jeffrey D Schall,et al.  The neural selection and control of saccades by the frontal eye field. , 2002, Philosophical transactions of the Royal Society of London. Series B, Biological sciences.

[69]  R. Llinás,et al.  Electrophysiology of globus pallidus neurons in vitro. , 1994, Journal of neurophysiology.

[70]  S. Zeki,et al.  Response properties and receptive fields of cells in an anatomically defined region of the superior temporal sulcus in the monkey. , 1971, Brain research.