Novelty and Beyond: Towards Combined Motivation Models and Integrated Learning Architectures

For future intrinsically motivated agents to combine multiple intrinsic motivation or behavioural components, there is a need to identify fundamental units of motivation models that can be reused and combined to produce more complex agents. This chapter reviews three existing models of intrinsic motivation, novelty, interest and competence-seeking motivation, that are based on the neural network framework of a real-time novelty detector. Four architectures are discussed that combine basic units of the intrinsic motivation functions in different ways. This chapter concludes with a discussion of future directions for combined motivation models and integrated learning architectures.

[1]  P. Hartvigsen.  The Computer for the 21st Century (1991) , 2014 .

[2]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[3]  G. Baldassarre,et al.  Functions and Mechanisms of Intrinsic Motivations The Knowledge Versus Competence Distinction , 2012 .

[4]  Douglas B. Lenat,et al.  AM, an artificial intelligence approach to discovery in mathematics as heuristic search , 1976 .

[5]  Pierre-Yves Oudeyer,et al.  Intrinsically Motivated Machines , 2006, 50 Years of Artificial Intelligence.

[6]  Ethem Alpaydin,et al.  Simplified ART: A new class of ART algorithms , 1998 .

[7]  Mary Lou and Gero John S. Maher,et al.  Agent Models of 3D Virtual Worlds , 2002 .

[8]  Kathryn E. Merrick,et al.  Modeling Behavior Cycles as a Value System for Developmental Robots , 2010, Adapt. Behav..

[9]  John S. Gero,et al.  Creativity, emergence and evolution in design , 1996, Knowl. Based Syst..

[10]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[11]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[12]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[13]  Kathryn E. Merrick,et al.  A curious agent for network anomaly detection , 2011, AAMAS.

[14]  Rob Saunders,et al.  Curious Design Agents and Artificial Creativity - A Synthetic Approach to the Study of Creative Behaviour , 2001 .

[15]  R. W. White.  Motivation reconsidered: the concept of competence. , 1959, Psychological review.

[16]  Nuttapong Chentanez,et al.  Intrinsically Motivated Learning of Hierarchical Collections of Skills , 2004 .

[17]  Marco Mirolli,et al.  Evolution and Learning in an Intrinsically Motivated Reinforcement Learning Robot , 2007, ECAL.

[18]  L. Steels.  Self-organising vocabularies , 1996 .

[19]  Teuvo Kohonen,et al.  Self-Organization and Associative Memory , 1988 .

[20]  Peter Dayan,et al.  Exploration and Curiosity in Robot Learning and Inference (Dagstuhl Seminar 11131) , 2011, Dagstuhl Reports.

[21]  Kathryn E. Merrick,et al.  Motivated Reinforcement Learning - Curious Characters for Multiuser Games , 2009 .

[22]  Kathryn E. Merrick,et al.  Achievement, affiliation, and power: Motive profiles for artificial agents , 2011, Adapt. Behav..

[23]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[24]  D. Berlyne.  Conflict, arousal, and curiosity , 2014 .

[25]  Wolfgang Banzhaf,et al.  Advances in Artificial Life , 2003, Lecture Notes in Computer Science.

[26]  Jürgen Schmidhuber,et al.  Formal Theory of Fun and Creativity , 2010, ECML/PKDD.

[27]  Rob Saunders,et al.  Achieving Creative Behavior Using Curious Learning Agents , 2008, AAAI Spring Symposium: Creative Intelligent Systems.

[28]  Kathryn E. Merrick.  Designing Toys That Come Alive: Curious Robots for Creative Play , 2008, ICEC.

[29]  J. C. Stanley.  Computer simulation of a model of habituation , 1976, Nature.

[30]  Kathryn E. Merrick,et al.  A Comparative Study of Value Systems for Self-Motivated Exploration and Learning by Robots , 2010, IEEE Transactions on Autonomous Mental Development.

[31]  Gianluca Baldassarre,et al.  What are intrinsic motivations? A biological perspective , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[32]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination in Human Behavior , 1975, Perspectives in Social Psychology.

[33]  Peter Dayan,et al.  Q-learning , 1992, Machine Learning.

[34]  Peter Dayan,et al.  Technical Note: Q-Learning , 2004, Machine Learning.

[35]  Pierre-Yves Oudeyer,et al.  Intrinsic Motivation Systems for Autonomous Mental Development , 2007, IEEE Transactions on Evolutionary Computation.

[36]  Jutta Heckhausen,et al.  Motivation and action (2nd ed.). , 2008 .

[37]  Marco Mirolli,et al.  Functions and Mechanisms of Intrinsic Motivations , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[38]  Edward L. Deci,et al.  Intrinsic Motivation and Self-Determination , 2004 .

[39]  Marco Mirolli,et al.  Intrinsically Motivated Learning in Natural and Artificial Systems , 2013 .

[40]  Mark Weiser.  The computer for the 21st century , 1991 .

[41]  Kathryn E. Merrick.  Modelling Affordances for the Control and Evaluation of Intrinsically Motivated Robots , 2009 .

[42]  T. Martin McGinnity,et al.  Novelty Detection as an Intrinsic Motivation for Cumulative Learning Robots , 2013, Intrinsically Motivated Learning in Natural and Artificial Systems.

[43]  Jürgen Schmidhuber,et al.  Curious model-building control systems , 1991, [Proceedings] 1991 IEEE International Joint Conference on Neural Networks.

[44]  Stephen R. Marsland,et al.  A Real-Time Novelty Detector for a Mobile Robot , 2000, ArXiv.

[45]  Pierre-Yves Oudeyer,et al.  Intelligent Adaptive Curiosity: a source of Self-Development , 2004 .

[46]  Antonio Amodeo,et al.  Nano- and microrobotics: how far is the reality? , 2008, Expert review of anticancer therapy.

[47]  Salvatore J. Stolfo,et al.  A framework for constructing features and models for intrusion detection systems , 2000, TSEC.

[48]  Kathryn E. Merrick,et al.  Agent Models for Self‐Motivated Home‐Assistant Bots , 2010 .

[49]  Jutta Heckhausen,et al.  Motivation and action , 1991 .

[50]  Bernd Fritzke.  Incremental Learning of Local Linear Mappings , 1995 .

[51]  W. Wundt,et al.  Principles of physiological psychology , 2015 .

[52]  Ken E. Whelan,et al.  The Automation of Science , 2009, Science.