Intrinsically motivated learning systems based on biologically-inspired novelty detection

Intrinsic motivations play an important role in human learning, particularly in the early stages of childhood development, and ideas from this research field have influenced robotic learning and adaptability. In this paper we investigate one specific type of intrinsic motivation, that of novelty detection and we discuss the reasons that make it a powerful facility for continuous learning. We formulate and present one original type of biologically inspired novelty detection architecture and implement it on a robotic system engaged in a perceptual classification task. The results of real-world robot experiments we conducted show how this original architecture conforms to behavioural observations and demonstrate its effectiveness in terms of focusing the system's attention in areas that are potential for effective learning. We proposed novelty detection as intrinsic motivation for autonomous learning.Novelty detection model is bio-inspired by the behavioural phenomenon of habituation.We demonstrated the effectiveness of the model in a real-world experiment.We showed how habituation is used as an intrinsic motivation for robot learning.

[1]  Pierre-Yves Oudeyer,et al.  Intrinsically Motivated Machines , 2006, 50 Years of Artificial Intelligence.

[2]  R. W. White.  Motivation reconsidered: the concept of competence. , 1959, Psychological review.

[3]  L. Festinger,et al.  A Theory of Cognitive Dissonance , 2017 .

[4]  S. Marsland.  Novelty Detection in Learning Systems , 2008 .

[5]  Marco Mirolli,et al.  Deciding Which Skill to Learn When: Temporal-Difference Competence-Based Intrinsic Motivation (TD-CB-IM) , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[6]  Sameer Singh,et al.  Novelty detection: a review - part 1: statistical approaches , 2003, Signal Process..

[7]  Gianluca Baldassarre,et al.  What are intrinsic motivations? A biological perspective , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[8]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[9]  Andrew Zisserman,et al.  Efficient Visual Search of Videos Cast as Text Retrieval , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[11]  T. Kohonen.  Self-organized formation of topographically correct feature maps , 1982 .

[12]  J. C. Stanley.  Computer simulation of a model of habituation , 1976, Nature.

[13]  Hugo Vieira Neto,et al.  Real-time Automated Visual Inspection using Mobile Robots , 2007, J. Intell. Robotic Syst..

[14]  T. Martin McGinnity,et al.  Online unsupervised cumulative learning for life-long robot operation , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[15]  Marco Mirolli,et al.  Functions and Mechanisms of Intrinsic Motivations , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[16]  Pierre-Yves Oudeyer,et al.  R-IAC: Robust Intrinsically Motivated Exploration and Active Learning , 2009, IEEE Transactions on Autonomous Mental Development.

[17]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[18]  J. Lisman,et al.  The Hippocampal-VTA Loop: Controlling the Entry of Information into Long-Term Memory , 2005, Neuron.

[19]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[20]  R. Aslin.  What's in a look? , 2007, Developmental science.

[21]  Donald A. Wilson,et al.  Habituation revisited: An updated and revised description of the behavioral characteristics of habituation , 2009, Neurobiology of Learning and Memory.

[22]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[23]  R. F. Thompson,et al.  Habituation: a model phenomenon for the study of neuronal substrates of behavior. , 1966, Psychological review.

[24]  Stephen Marsland,et al.  Using habituation in machine learning , 2009, Neurobiology of Learning and Memory.

[25]  Emrah Düzel,et al.  The Hippocampal-VTA Loop: The Role of Novelty and Motivation in Controlling the Entry of Information into Long-Term Memory , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[26]  Stephen R. Marsland,et al.  A self-organising network that grows when required , 2002, Neural Networks.

[27]  Stephen R. Marsland,et al.  On-line novelty detection for autonomous mobile robots , 2005, Robotics Auton. Syst..

[28]  Francesco Mannella,et al.  Intrinsically motivated action-outcome learning and goal-based action recall: a system-level bio-constrained computational model. , 2013, Neural networks : the official journal of the International Neural Network Society.

[29]  Hugo Vieira Neto,et al.  Visual novelty detection with automatic scale selection , 2007, Robotics Auton. Syst..

[30]  R. Krauzlis,et al.  Superior colliculus and visual spatial attention. , 2013, Annual review of neuroscience.

[31]  T. Abrams,et al.  Insights into a molecular switch that gates sensory neuron synapses during habituation in Aplysia , 2009, Neurobiology of Learning and Memory.

[32]  Richard F. Thompson.  Habituation: A history , 2009, Neurobiology of Learning and Memory.

[33]  Catharine H. Rankin,et al.  Introduction to special issue of neurobiology of learning and memory on habituation , 2009, Neurobiology of Learning and Memory.

[34]  J S Werner,et al.  Habituation, response to novelty, and dishabituation in human infants: tests of a dual-process theory of visual attention. , 1986, Journal of experimental child psychology.

[35]  Teuvo Kohonen,et al.  Self-organized formation of topologically correct feature maps , 2004, Biological Cybernetics.

[36]  J. Colombo,et al.  Infant visual habituation , 2009, Neurobiology of Learning and Memory.

[37]  Chun-Fang Wu,et al.  Neurogenetic approaches to habituation and dishabituation in Drosophila , 2009, Neurobiology of Learning and Memory.