Neuromorphic motivated systems

Although reinforcement learning has been extensively modeled, few agent models that incorporate values use biologically plausible neural networks as a uniform computational architecture. We call biologically plausible neural network architecture neuromorphic. This paper discusses some theoretical constraints on neuromorphic intrinsic value systems [3]. By intrinsic, we mean a value system that is likely programmed by the genes, whose value bias has already taken a shape at the birth time. Such an intrinsic value system plays an important role in developing extrinsic values through the agent's own experience during its life span. Based on our theoretical constraints, we model two types of neurotransmitters, serotonin and dopamine, to construct a neuromorpic intrinsic value system based on a uniform neural network architecture. Serotonin represents punishment and stress, while dopamine represents reward and pleasure. Experimentally, this model allows our simulated robot to develop an attachment to one entity and fear another.

[1]  A. Maslow A Theory of Human Motivation , 1943 .

[2]  W. K. Purves Life: The Science of Biology , 1985 .

[3]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[4]  M. Alexander,et al.  Principles of Neural Science , 1981 .

[5]  T. Robbins,et al.  Neurobehavioural mechanisms of reward and motivation , 1996, Current Opinion in Neurobiology.

[6]  Juyang Weng,et al.  2008 Special issue , 2008 .

[7]  Jeffrey L. Krichmar,et al.  The Neuromodulatory System: A Framework for Survival and Adaptive Behavior in a Challenging World , 2008, Adapt. Behav..

[8]  Juyang Weng,et al.  A 5-chunk developmental brain-mind network model for multiple events in complex backgrounds , 2010, The 2010 International Joint Conference on Neural Networks (IJCNN).

[9]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[10]  Juyang Weng,et al.  Dually Optimal Neuronal Layers: Lobe Component Analysis , 2009, IEEE Transactions on Autonomous Mental Development.

[11]  R. Solomon,et al.  An opponent-process theory of motivation. I. Temporal dynamics of affect. , 1974, Psychological review.

[12]  Lee Spector,et al.  Genetic Programming for Reward Function Search , 2010, IEEE Transactions on Autonomous Mental Development.

[13]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[14]  Kathryn E. Merrick,et al.  A Comparative Study of Value Systems for Self-Motivated Exploration and Learning by Robots , 2010, IEEE Transactions on Autonomous Mental Development.

[15]  Jeffrey L. Krichmar,et al.  Neuromodulation as a robot controller , 2009, IEEE Robotics & Automation Magazine.

[16]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[17]  Richard L. Lewis,et al.  Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective , 2010, IEEE Transactions on Autonomous Mental Development.

[18]  Olaf Sporns,et al.  Plasticity in Value Systems and its Role in Adaptive Behavior , 2000, Adapt. Behav..

[19]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[20]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[21]  R. Solomon,et al.  An Opponent-Process Theory of Motivation , 1978 .

[22]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination , 2004 .

[23]  Howard S. Hoffman,et al.  An opponent-process theory of motivation: III. Some affective dynamics in imprinting , 1974 .

[24]  A. Maslow Motivation and Personality , 1954 .

[25]  G. Edelman,et al.  Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. , 1998, Cerebral cortex.

[26]  Andrew W. Moore,et al.  Reinforcement Learning: A Survey , 1996, J. Artif. Intell. Res..

[27]  M. Domjan The principles of learning and behavior , 1982 .

[28]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[29]  M. Sur,et al.  Patterning and Plasticity of the Cerebral Cortex , 2005, Science.

[30]  Juyang Weng,et al.  Inherent Value Systems for Autonomous Mental Development , 2007, Int. J. Humanoid Robotics.

[31]  L. Wolpert Developmental Biology , 1968, Nature.

[32]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .