Modeling the sensory roles of noradrenaline in action selection

Noradrenaline participates in the neuromodulation of brain activity to modify the trade-off between exploration and exploitation when sensory contingencies have changed. Accordingly, attentional models of noradrenaline acting on sensory representations have been proposed. In this paper, we explore another possible action of this neuromodulator in the decision making process and report simulation results that illustrate that its role is concerned with different aspects of sensory processing. This is made possible by the extension of a classical model of action selection, to render it able to detect and to adapt to sudden changes in sensory contingencies, which is a major characteristic of autonomous learning.

[1]  M. Kringelbach.  The human orbitofrontal cortex: linking reward to hedonic experience , 2005, Nature Reviews Neuroscience.

[2]  N. Daw,et al.  Multiplicity of control in the basal ganglia: computational roles of striatal subregions , 2011, Current Opinion in Neurobiology.

[3]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[4]  S. Killcross,et al.  Coordination of actions and habits in the medial prefrontal cortex of rats. , 2003, Cerebral cortex.

[5]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.

[6]  P. Dayan,et al.  Uncertainty-based competition between prefrontal and dorsolateral striatal systems for behavioral control , 2005, Nature Neuroscience.

[7]  B. Knowlton,et al.  Learning and memory functions of the Basal Ganglia. , 2002, Annual review of neuroscience.

[8]  Thomas E. Hazy,et al.  PVLV: the primary value and learned value Pavlovian learning algorithm. , 2007, Behavioral neuroscience.

[9]  S. T. Mason,et al.  An investigation of the role of cortical and cerebellar noradrenaline in associative motor learning in the rat , 1977, Brain Research.

[10]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[11]  S. Sara,et al.  Orienting and Reorienting: The Locus Coeruleus Mediates Cognition through Arousal , 2012, Neuron.

[12]  M. Khamassi,et al.  Dopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia , 2012, Front. Neurosci..

[13]  Philip Holmes,et al.  Simple Neural Networks that Optimize Decisions , 2005, Int. J. Bifurc. Chaos.

[14]  Eytan Ruppin,et al.  Actor-critic models of the basal ganglia: new anatomical and computational perspectives , 2002, Neural Networks.

[15]  P. Dayan,et al.  Decision theory, reinforcement learning, and the brain , 2008, Cognitive, affective & behavioral neuroscience.

[16]  S. Mizumori,et al.  Neural systems analysis of decision making during goal-directed navigation , 2012, Progress in Neurobiology.

[17]  A. Pouget,et al.  Probabilistic brains: knowns and unknowns , 2013, Nature Neuroscience.

[18]  B. Everitt,et al.  Emotion and motivation: the role of the amygdala, ventral striatum, and prefrontal cortex , 2002, Neuroscience & Biobehavioral Reviews.

[19]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[20]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[21]  Jonathan D. Cohen,et al.  Conflict monitoring and anterior cingulate cortex: an update , 2004, Trends in Cognitive Sciences.

[22]  B. Balleine,et al.  The integrative function of the basal ganglia in instrumental conditioning , 2009, Behavioural Brain Research.

[23]  B. Balleine,et al.  Parallel incentive processing: an integrated view of amygdala function , 2006, Trends in Neurosciences.

[24]  R. Malenka,et al.  Modulation of synaptic transmission by dopamine and norepinephrine in ventral but not dorsal striatum. , 1998, Journal of neurophysiology.

[25]  Andrew M. Wikenheiser,et al.  Toward a theoretical role for tonic norepinephrine in the orbitofrontal cortex in facilitating flexible learning , 2017, Neuroscience.

[26]  T. Boraud,et al.  Interaction between cognitive and motor cortico-basal ganglia loops during decision making: a computational study. , 2013, Journal of neurophysiology.

[27]  K. Gurney,et al.  The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis , 2013, Front. Behav. Neurosci..

[28]  T. Robbins,et al.  Lesions of the dorsal noradrenergic bundle impair attentional set‐shifting in the rat , 2007, The European journal of neuroscience.

[29]  Y. Niv.  Reinforcement learning in the brain , 2009 .

[30]  Wolfgang M. Pauli,et al.  Attentional control of associative learning—A possible role of the central cholinergic system , 2008, Brain Research.

[31]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[32]  Jonathan D. Cohen,et al.  An exploration-exploitation model based on norepinepherine and dopamine activity , 2005, NIPS.

[33]  G. Aston-Jones,et al.  Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task , 1997, Neuroscience.

[34]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.