Inherent Value Systems for Autonomous Mental Development

The inherent value system of a developmental agent enables autonomous mental development to take place right after the agent's "birth." Biologically, it is not clear what basic components constitute a value system. In the computational model introduced here, we propose that inherent value systems should have at least three basic components: punishment, reward and novelty with decreasing weights from the first component to the last. Punishments and rewards are temporally sparse but novelty is temporally dense. We present a biologically inspired computational architecture that guides development of sensorimotor skills through real-time interactions with the environments, driven by an inborn value system. The inherent value system has been successfully tested on an artificial agent in a simulation environment and a robot in the real world.

[1]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[2]  Rodney A. Brooks,et al.  Humanoid robots , 2002, CACM.

[3]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[4]  M Rucci,et al.  Registration of Neural Maps through Value-Dependent Learning: Modeling the Alignment of Auditory and Visual Maps in the Barn Owl’s Optic Tectum , 1997, The Journal of Neuroscience.

[5]  G. Lynch,et al.  The neurobiology of learning and memory , 1989, Cognition.

[6]  Peter Norvig,et al.  Artificial Intelligence: A Modern Approach , 1995 .

[7]  M. Domjan.  The principles of learning and behavior , 1982 .

[8]  N. Bayley.  Bayley Scales of Infant Development , 1999 .

[9]  Juyang Weng,et al.  An Incremental Learning Algorithm with Automatically Derived Discriminating Features , 2000 .

[10]  Xiao Huang,et al.  Novelty and Reinforcement Learning in the Value System of Developmental Robots , 2002 .

[11]  James L. McClelland,et al.  Autonomous Mental Development by Robots and Animals , 2001, Science.

[12]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[13]  Juyang Weng,et al.  From neural networks to the brain: autonomous mental development , 2006, IEEE Computational Intelligence Magazine.

[14]  Pierre-Yves Oudeyer,et al.  The Playground Experiment: Task-Independent Development of a Curious Robot , 2005 .

[15]  Minoru Asada,et al.  Purposive Behavior Acquisition for a Real Robot by Vision-Based Reinforcement Learning , 2005, Machine Learning.

[16]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[17]  Pierre-Yves Oudeyer,et al.  Intelligent Adaptive Curiosity: a source of Self-Development , 2004 .

[18]  S Ullman,et al.  Shifts in selective visual attention: towards the underlying neural circuitry. , 1985, Human neurobiology.

[19]  Peter Dayan,et al.  Dopamine: generalization and bonuses , 2002, Neural Networks.

[20]  G. Edelman,et al.  Behavioral constraints in the development of neuronal properties: a cortical model embedded in a real-world device. , 1998, Cerebral cortex.

[21]  Sebastian Thrun,et al.  Exploration in active learning , 1998 .

[22]  Peter Dayan,et al.  Dopamine Bonuses , 2000, NIPS.

[23]  Minoru Asada,et al.  Purposive behavior acquisition for a real robot by vision-based reinforcement learning , 1995, Machine Learning.

[24]  A G Barto,et al.  Toward a modern theory of adaptive networks: expectation and prediction. , 1981, Psychological review.

[25]  J. C. Stanley.  Computer simulation of a model of habituation , 1976, Nature.

[26]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[27]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[28]  P. L. Adams.  THE ORIGINS OF INTELLIGENCE IN CHILDREN , 1976 .

[29]  J. W. Rudy,et al.  Habituation, sensitization, and infant visual attention , 1990 .

[30]  W. Schultz.  Multiple reward signals in the brain , 2000, Nature Reviews Neuroscience.

[31]  D. Batens,et al.  Theory and Experiment , 1988 .

[32]  Juyang Weng,et al.  Developmental Robotics: Theory and Experiments , 2004, Int. J. Humanoid Robotics.

[33]  Minoru Asada,et al.  Cognitive developmental robotics as a new paradigm for the design of humanoid robots , 2001, Robotics Auton. Syst..

[34]  Olaf Sporns,et al.  Plasticity in Value Systems and its Role in Adaptive Behavior , 2000, Adapt. Behav..

[35]  Wesley R. Elsberry,et al.  Optimality in Biological and Artificial Networks , 1997 .

[36]  L. Steels,et al.  Grounding adaptive language games in robotic agents , 2006, AAAI 2012.

[37]  Brian Scassellati,et al.  Theory of Mind for a Humanoid Robot , 2002, Auton. Robots.

[38]  M. Sur,et al.  Rewiring cortex: the role of patterned activity in development and plasticity of neocortical circuits. , 1999, Journal of neurobiology.

[39]  H. J. Mclaughlin,et al.  Learn , 2002 .