Different Learning Architectures Contribute to Value Prediction in Human Cognition

Pavlovian conditioning plays a fundamental role in our cog-nitive architecture, by its capacity to bind values to stimuli. Due to the multifarious characteristics of this learning mode, many approaches in machine learning have been proposed to implement it in artificial intelligence models. Considering the complementary properties of these models and inspired by biological evidences, we propose not to select the best model but rather to combine them, thereby forming a cognitive architecture. From a functional point of view, we report the good properties and performances of this architecture. From a methodological point of view, this work highlights the interest of defining a cognitive function at the algorithmic level, binding its general properties to its implementation details. It is also proposed that it is a fruitful approach to decipher and organize many information extracted by modern approaches in neu-roscience, towards the definition of a global cognitive architecture.

[1]  Frédéric Alexandre,et al.  A Modular Network Architecture Resolving Memory Interference Through Inhibition , 2015, IJCCI.

[2]  M. Hasselmo The role of acetylcholine in learning and memory , 2006, Current Opinion in Neurobiology.

[3]  M. Pelley The Role of Associative History in Models of Associative Learning: A Selective Review and a Hybrid Model: , 2004 .

[4]  James L. McClelland,et al.  Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. , 1995, Psychological review.

[5]  Frédéric Alexandre,et al.  Modeling Neuromodulation as a Framework to Integrate Uncertainty in General Cognitive Architectures , 2016, AGI.

[6]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[7]  Pieter R. Roelfsema,et al.  Attention-Gated Reinforcement Learning of Internal Representations for Classification , 2005, Neural Computation.

[8]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[9]  Michael B. Stadler,et al.  Encoding of conditioned fear in central amygdala inhibitory circuits , 2010, Nature.

[10]  Frédéric Alexandre,et al.  Biological Inspiration for Multiple Memories Implementation and Cooperation , 2000 .

[11]  N. Schmajuk,et al.  Stimulus configuration, classical conditioning, and hippocampal function. , 1992, Psychological review.

[12]  Michael Davis,et al.  The amygdala , 2000, Current Biology.

[13]  L. Allan Human contingency judgments: rule based or associative? , 1993, Psychological bulletin.

[14]  Zenon W. Pylyshyn,et al.  Computation and Cognition: Toward a Foundation for Cognitive Science , 1984 .

[15]  Marvin Minsky,et al.  Perceptrons: An Introduction to Computational Geometry , 1969 .

[16]  Frédéric Alexandre,et al.  Feature, Configuration, History - A Bio-inspired Framework for Information Representation in Neural Networks , 2014, IJCCI.

[17]  Frédéric Alexandre,et al.  Integration of exteroceptive and interoceptive information within the hippocampus: a computational study , 2015, Front. Syst. Neurosci..

[18]  Johan Håstad,et al.  On the power of small-depth threshold circuits , 1990, Proceedings [1990] 31st Annual Symposium on Foundations of Computer Science.

[19]  Robert M. French,et al.  Semi-distributed Representations and Catastrophic Forgetting in Connectionist Networks , 1992 .

[20]  R. O’Reilly,et al.  Computational Explorations in Cognitive Neuroscience: Understanding the Mind by Simulating the Brain , 2000 .

[21]  D. Blei,et al.  Context, learning, and extinction. , 2010, Psychological review.

[22]  Frédéric Alexandre,et al.  Modeling the sensory roles of noradrenaline in action selection , 2016, 2016 Joint IEEE International Conference on Development and Learning and Epigenetic Robotics (ICDL-EpiRob).

[23]  R. Rescorla A theory of pavlovian conditioning: The effectiveness of reinforcement and non-reinforcement , 1972 .

[24]  John J. Hopfield,et al.  Neural networks and physical systems with emergent collective computational abilities , 1999 .

[25]  Joseph J. Paton,et al.  Expectation Modulates Neural Responses to Pleasant and Aversive Stimuli in Primate Amygdala , 2007, Neuron.

[26]  Rich Caruana,et al.  Do Deep Nets Really Need to be Deep? , 2013, NIPS.

[27]  Frédéric Alexandre,et al.  A pavlovian model of the amygdala and its influence within the medial temporal lobe , 2015, Front. Syst. Neurosci..

[28]  N. Mons,et al.  Extracellular Hippocampal Acetylcholine Level Controls Amygdala Function and Promotes Adaptive Conditioned Emotional Response , 2006, The Journal of Neuroscience.

[29]  G. McNally,et al.  The conditions that promote fear learning: Prediction error and Pavlovian fear conditioning , 2014, Neurobiology of Learning and Memory.

[30]  John R Anderson,et al.  An integrated theory of the mind. , 2004, Psychological review.

[31]  A. Lüthi,et al.  Switching on and off fear by distinct neuronal circuits , 2008, Nature.

[32]  G. Roth,et al.  Evolution of the Amygdala: New Insights from Studies in Amphibians , 2006, Brain, Behavior and Evolution.

[33]  Gregory Ashby,et al.  A neuropsychological theory of multiple systems in category learning. , 1998, Psychological review.

[34]  Frédéric Alexandre,et al.  A biologically inspired neuronal model of reward prediction error computation , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[35]  Wolfgang M. Pauli,et al.  Attentional control of associative learning—A possible role of the central cholinergic system , 2008, Brain Research.

[36]  S. Thorpe,et al.  Seeking Categories in the Brain , 2001, Science.

[37]  Stephen Maren,et al.  Contextual and auditory fear conditioning are mediated by the lateral, basal, and central amygdaloid nuclei in rats. , 2001, Learning & memory.

[38]  R. Weale Vision. A Computational Investigation Into the Human Representation and Processing of Visual Information. David Marr , 1983 .

[39]  D. Paré,et al.  Measuring correlations and interactions among four simultaneously recorded brain regions during learning. , 2009, Journal of Neurophysiology.

[40]  Angela J. Yu,et al.  Uncertainty, Neuromodulation, and Attention , 2005, Neuron.

[41]  L. Squire Declarative and Nondeclarative Memory: Multiple Brain Systems Supporting Learning and Memory , 1992, Journal of Cognitive Neuroscience.