Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures

One of the most critical properties of a versatile intelligent agent is its capacity to adapt autonomously to any change in the environment without overly complexifying its cognitive architecture. In this paper, we propose that understanding the role of neuromodulation in the brain is of central interest for this purpose. More precisely, we propose that an accurate estimation of the nature of uncertainty present in the environment is performed by specific brain regions and broadcast throughout the cerebral network by neuromodulators, resulting in appropriate changes in cerebral functioning and learning modes. Better understanding the principles of these mechanisms in the brain might tremendously inspire the field of Artificial General Intelligence. The original contribution of this paper is to relate the four major neuromodulators to four fundamental dimensions of uncertainty.

[1]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[2]  Angela J. Yu,et al.  Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration , 2007, Philosophical Transactions of the Royal Society B: Biological Sciences.

[3]  Nikolaus R. McFarland,et al.  Striatonigrostriatal Pathways in Primates Form an Ascending Spiral from the Shell to the Dorsolateral Striatum , 2000, The Journal of Neuroscience.

[4]  Kenji Doya,et al.  Metalearning and neuromodulation , 2002, Neural Networks.

[5]  K. Berridge,et al.  What is the role of dopamine in reward: hedonic impact, reward learning, or incentive salience? , 1998, Brain Research Reviews.

[6]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[7]  K. Gurney,et al.  The nucleus accumbens as a nexus between values and goals in goal-directed behavior: a review and a new hypothesis , 2013, Front. Behav. Neurosci..

[8]  Wolfgang M. Pauli,et al.  Expectancy, Ambiguity, and Behavioral Flexibility: Separable and Complementary Roles of the Orbital Frontal Cortex and Amygdala in Processing Reward Expectancies , 2012, Journal of Cognitive Neuroscience.

[9]  Jonathan D. Cohen,et al.  An integrative theory of locus coeruleus-norepinephrine function: adaptive gain and optimal performance. , 2005, Annual review of neuroscience.

[10]  Wolfgang M. Pauli,et al.  Attentional control of associative learning—A possible role of the central cholinergic system , 2008, Brain Research.

[11]  Jonathan D. Cohen,et al.  Role of locus coeruleus in attention and behavioral flexibility , 1999, Biological Psychiatry.

[12]  S. Sara,et al.  Orienting and Reorienting: The Locus Coeruleus Mediates Cognition through Arousal , 2012, Neuron.

[13]  M. Khamassi,et al.  Dopaminergic Control of the Exploration-Exploitation Trade-Off via the Basal Ganglia , 2012, Front. Neurosci..

[14]  S. Sara,et al.  Network reset: a simplified overarching theory of locus coeruleus noradrenaline function , 2005, Trends in Neurosciences.

[15]  Brian R. Tietz,et al.  Deciding Which Way to Go: How Do Insects Alter Movements to Negotiate Barriers? , 2012, Front. Neurosci..

[16]  Mitsuo Kawato,et al.  Multiple Model-Based Reinforcement Learning , 2002, Neural Computation.

[17]  Frédéric Alexandre,et al.  A pavlovian model of the amygdala and its influence within the medial temporal lobe , 2015, Front. Syst. Neurosci..

[18]  Matthew Thompson,et al.  The Predictive Value of the NICE “Red Traffic Lights” in Acutely Ill Children , 2014, PloS one.

[19]  Qiang Yang,et al.  Lifelong Machine Learning Systems: Beyond Learning Algorithms , 2013, AAAI Spring Symposium: Lifelong Machine Learning.

[20]  Oded Berger-Tal,et al.  The Exploration-Exploitation Dilemma: A Multidisciplinary Framework , 2014, PloS one.

[21]  Stephen Grossberg,et al.  Adaptive Resonance Theory: How a brain learns to consciously attend, learn, and recognize a changing world , 2013, Neural Networks.

[22]  Balaraman Ravindran,et al.  An extended reinforcement learning model of basal ganglia to understand the contributions of serotonin and dopamine in risk-based decision making, reward prediction, and punishment learning , 2014, Front. Comput. Neurosci..

[23]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[24]  G. Aston-Jones,et al.  Conditioned responses of monkey locus coeruleus neurons anticipate acquisition of discriminative behavior in a vigilance task , 1997, Neuroscience.

[25]  Jonathan D. Cohen,et al.  An exploration-exploitation model based on norepinepherine and dopamine activity , 2005, NIPS.

[26]  W. Schultz.  Predictive reward signal of dopamine neurons. , 1998, Journal of neurophysiology.

[27]  Y. Niv.  Cost, Benefit, Tonic, Phasic , 2007, Annals of the New York Academy of Sciences.

[28]  Karl J. Friston.  Functional integration and inference in the brain , 2002, Progress in Neurobiology.

[29]  N. Mons,et al.  Extracellular Hippocampal Acetylcholine Level Controls Amygdala Function and Promotes Adaptive Conditioned Emotional Response , 2006, The Journal of Neuroscience.

[30]  N. Daw,et al.  Serotonin and Dopamine: Unifying Affective, Activational, and Decision Functions , 2011, Neuropsychopharmacology.

[31]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[32]  P. Dayan.  Twenty-Five Lessons from Computational Neuromodulation , 2012, Neuron.

[33]  Sham M. Kakade,et al.  Opponent interactions between serotonin and dopamine , 2002, Neural Networks.

[34]  P. Dayan,et al.  Cortical substrates for exploratory decisions in humans , 2006, Nature.