Multiple timescales of normalized value coding underlie adaptive choice behavior

Adaptation is a fundamental process crucial for the efficient coding of sensory information. Recent evidence suggests that similar coding principles operate in decision-related brain areas, where neural value coding adapts to recent reward history. However, the circuit mechanism for value adaptation is unknown, and the link between changes in adaptive value coding and choice behavior is unclear. Here we show that choice behavior in nonhuman primates varies with the statistics of recent rewards. Consistent with efficient coding theory, decision-making shows increased choice sensitivity in lower variance reward environments. Both the average adaptation effect and across-session variability are explained by a novel multiple timescale dynamical model of value representation implementing divisive normalization. The model predicts empirical variance-driven changes in behavior despite having no explicit knowledge of environmental statistics, suggesting that distributional characteristics can be captured by dynamic model architectures. These findings highlight the importance of treating decision-making as a dynamic process and the role of normalization as a unifying computation for contextual phenomena in choice.Previous work has shown that the neural representation of value adapts to the recent history of rewards. Here, the authors report that a computational model based on divisive normalization over multiple timescales can explain changes in value coding driven by changes in the reward statistics.

[1]  William Bialek,et al.  Adaptive Rescaling Maximizes Information Transmission , 2000, Neuron.

[2]  D. Heeger Normalization of cell responses in cat striate cortex , 1992, Visual Neuroscience.

[3]  Karl J. Friston,et al.  Neural processes mediating contextual influences on human choice behaviour , 2016, Nature Communications.

[4]  Jan Zimmermann,et al.  Oculomatic: High speed, reliable, and accurate open-source eye tracking for humans and non-human primates , 2016, Journal of Neuroscience Methods.

[5]  J. Cowan,et al.  Wilson–Cowan Equations for Neocortical Dynamics , 2016, Journal of mathematical neuroscience.

[6]  M. Carandini,et al.  Normalization as a canonical neural computation , 2011, Nature Reviews Neuroscience.

[7]  I. Simonson,et al.  Choice Based on Reasons: The Case of Attraction and Compromise Effects , 1989 .

[8]  P. Glimcher,et al.  The Temporal Dynamics of Cortical Normalization Models of Decision-making , 2014 .

[9]  C. Padoa-Schioppa Neurobiology of economic choice: a good-based model. , 2011, Annual review of neuroscience.

[10]  Peter Dayan,et al.  A Neural Substrate of Prediction and Reward , 1997, Science.

[11]  Timothy E. J. Behrens,et al.  Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world , 2011, Annals of the New York Academy of Sciences.

[12]  C. Padoa-Schioppa,et al.  Neurons in the orbitofrontal cortex encode economic value , 2006, Nature.

[13]  P. Glimcher,et al.  Dynamic Divisive Normalization Predicts Time-Varying Value Coding in Decision-Related Circuits , 2014, The Journal of Neuroscience.

[14]  D. Heeger,et al.  The Normalization Model of Attention , 2009, Neuron.

[15]  H. Kennedy,et al.  A Large-Scale Circuit Mechanism for Hierarchical Dynamical Processing in the Primate Cortex , 2015, Neuron.

[16]  W. Schultz,et al.  Relative reward preference in primate orbitofrontal cortex , 1999, Nature.

[17]  P. Glimcher,et al.  Reward Value-Based Gain Control: Divisive Normalization in Parietal Cortex , 2011, The Journal of Neuroscience.

[18]  C. Padoa-Schioppa,et al.  The representation of economic value in the orbitofrontal cortex is invariant for changes of menu , 2008, Nature Neuroscience.

[19]  A. Kohn Visual adaptation: physiology, mechanisms, and functional benefits. , 2007, Journal of neurophysiology.

[20]  Alireza Soltani,et al.  A Range-Normalization Model of Context-Dependent Choice: A New Model and Evidence , 2012, PLoS Comput. Biol..

[21]  Christopher P. Puto,et al.  Adding Asymmetrically Dominated Alternatives: Violations of Regularity & the Similarity Hypothesis. , 1981 .

[22]  Valentin Dragoi,et al.  Adaptive coding of visual information in neural populations , 2008, Nature.

[23]  R. Wise Dopamine, learning and motivation , 2004, Nature Reviews Neuroscience.

[24]  Tatsuo K Sato,et al.  An excitatory basis for divisive normalization in visual cortex , 2016, Nature Neuroscience.

[25]  C. Enroth-Cugell,et al.  Chapter 9 Visual adaptation and retinal gain controls , 1984 .

[26]  David J. Freedman,et al.  A hierarchy of intrinsic timescales across primate cortex , 2014, Nature Neuroscience.

[27]  Ryan Webb,et al.  Adaptive neural coding: from biological to behavioral decision-making , 2015, Current Opinion in Behavioral Sciences.

[28]  James L. McClelland,et al.  Integration of Sensory and Reward Information during Perceptual Decision-Making in Lateral Intraparietal Cortex (LIP) of the Macaque Monkey , 2010, PloS one.

[29]  S. Solomon,et al.  Moving Sensory Adaptation beyond Suppressive Effects in Single Neurons , 2014, Current Biology.

[30]  H. Seo,et al.  A reservoir of time constants for memory traces in cortical neurons , 2011, Nature Neuroscience.

[31]  Aldo Rustichini,et al.  Optimal coding and neuronal adaptation in economic decisions , 2017, Nature Communications.

[32]  C. Padoa-Schioppa Neuronal Origins of Choice Variability in Economic Decisions , 2013, Neuron.

[33]  K. Zilles,et al.  Synaptic patterning and the timescales of cortical dynamics , 2017, Current Opinion in Neurobiology.

[34]  Shawn R. Olsen,et al.  Divisive Normalization in Olfactory Population Codes , 2010, Neuron.

[35]  M. Carandini,et al.  Visual cortex: Fatigue and adaptation , 2000, Current Biology.

[36]  A. Tversky Elimination by aspects: A theory of choice. , 1972 .

[37]  Joseph J. Atick,et al.  Towards a Theory of Early Visual Processing , 1990, Neural Computation.

[38]  Mel W. Khaw,et al.  Normalization is a general neural mechanism for context-dependent decision making , 2013, Proceedings of the National Academy of Sciences.

[39]  C. H. Donahue,et al.  Metaplasticity as a Neural Substrate for Adaptive Learning and Choice under Uncertainty , 2017, Neuron.

[40]  C. Padoa-Schioppa Range-Adapting Representation of Economic Value in the Orbitofrontal Cortex , 2009, The Journal of Neuroscience.

[41]  J. Serences,et al.  Value-based attentional capture influences context-dependent decision-making. , 2015, Journal of neurophysiology.

[42]  Joseph J. Paton,et al.  A Scalable Population Code for Time in the Striatum , 2015, Current Biology.

[43]  A. Angelucci,et al.  Circuits and Mechanisms for Surround Modulation in Visual Cortex. , 2017, Annual review of neuroscience.

[44]  Raymond J. Dolan,et al.  The influence of contextual reward statistics on risk preference , 2016, NeuroImage.

[45]  Xiao-Jing Wang,et al.  A diversity of localized timescales in network activity , 2014, eLife.

[46]  Rufin Vogels,et al.  Divisive Normalization Predicts Adaptation-Induced Response Changes in Macaque Inferior Temporal Cortex , 2016, The Journal of Neuroscience.

[47]  W. Schultz,et al.  Adaptation of Reward Sensitivity in Orbitofrontal Neurons , 2010, The Journal of Neuroscience.

[48]  P. Glimcher,et al.  Annals of the New York Academy of Sciences Efficient Coding and the Neural Representation of Value , 2022 .

[49]  A. Rangel,et al.  Value normalization in decision making: theory and evidence , 2012, Current Opinion in Neurobiology.

[50]  M. Subrahmanyam Theory and Evidence , 2013 .

[51]  J. Cowan,et al.  Excitatory and inhibitory interactions in localized populations of model neurons. , 1972, Biophysical journal.

[52]  O. Schwartz,et al.  Adaptation in the visual cortex: a case for probing neuronal populations with natural stimuli , 2017, F1000Research.

[53]  Paul W. Glimcher,et al.  Absence of Spatial Tuning in the Orbitofrontal Cortex , 2014, PloS one.

[54]  Michael J. Berry,et al.  Adaptation of retinal processing to image contrast and spatial scale , 1997, Nature.

[55]  Timothy E. J. Behrens,et al.  Hierarchical competitions subserving multi-attribute choice , 2014, Nature Neuroscience.

[56]  W. Schultz,et al.  Adaptive Coding of Reward Value by Dopamine Neurons , 2005, Science.

[57]  Eero P. Simoncelli,et al.  Natural image statistics and neural representation. , 2001, Annual review of neuroscience.

[58]  Paul Cisek,et al.  Neural Correlates of Biased Competition in Premotor Cortex , 2011, The Journal of Neuroscience.

[59]  H. Adesnik,et al.  Input normalization by global feedforward inhibition expands cortical dynamic range , 2009, Nature Neuroscience.

[60]  H. B. Barlow,et al.  Possible Principles Underlying the Transformations of Sensory Messages , 2012 .

[61]  P. Dayan,et al.  Space and time in visual context , 2007, Nature Reviews Neuroscience.

[62]  Emad N. Eskandar,et al.  A flexible software tool for temporally-precise behavioral control in Matlab , 2008, Journal of Neuroscience Methods.

[63]  Ayzerman,et al.  Theory of choice , 1995 .

[64]  M. Khamassi,et al.  Contextual modulation of value signals in reward and punishment learning , 2015, Nature Communications.

[65]  Timothy E. J. Behrens,et al.  Learning the value of information in an uncertain world , 2007, Nature Neuroscience.

[66]  S. Laughlin A Simple Coding Procedure Enhances a Neuron's Information Capacity , 1981, Zeitschrift fur Naturforschung. Section C, Biosciences.

[67]  Zachary M. Westrick,et al.  Pattern Adaptation and Normalization Reweighting , 2016, The Journal of Neuroscience.

[68]  A. Fairhall,et al.  Sensory adaptation , 2007, Current Opinion in Neurobiology.

[69]  H. Wilson,et al.  Spatial frequency mechanisms with short-wavelength-sensitive cone inputs , 1992, Vision Research.