Motivational Engine with Sub-goal Identification in Neuroevolution Based Cognitive Robotics

A first approach towards a new motivational system for an autonomous robot that can learn chains of sub-goals leading to a final reward is proposed in this paper. The motivational system provides the motivation that guides the robot operation according to its knowledge of its sensorial space so that rewards are maximized during its lifetime. In order to do this, a motivational engine progressively and interactively creates an internal model of expected future reward (value function) for areas of the robot’s state space, through a neuroevolutionary process, over samples obtained in the sensorial (state space) traces followed by the robot whenever it obtained a reward. To improve this modelling process, a strategy is proposed to decompose the global value function leading to the reward or goal into several more local ones, thus discovering sub-goals that simplify the whole learning process and that can be reused in the future. The motivational engine is tested in a simulated experiment with very promising results.

[1]  Christoph Salge,et al.  Approximation of Empowerment in the continuous Domain , 2013, Adv. Complex Syst..

[2]  E. Deci,et al.  Intrinsic and Extrinsic Motivations: Classic Definitions and New Directions. , 2000, Contemporary educational psychology.

[3]  Richard S. Sutton,et al.  Reinforcement learning architectures for animats , 1991 .

[4]  Shaul Markovitch,et al.  Learning Novel Domains Through Curiosity and Conjecture , 1989, IJCAI.

[5]  Sergiu-Mihai Dascalu,et al.  Virtual Neurorobotics (VNR) to Accelerate Development of Plausible Neuromorphic Brain Architectures , 2007, Frontiers in neurorobotics.

[6]  Douglas B. Lenat,et al.  AM, an artificial intelligence approach to discovery in mathematics as heuristic search , 1976 .

[7]  Andrew G. Barto,et al.  Automatic Discovery of Subgoals in Reinforcement Learning using Diverse Density , 2001, ICML.

[8]  Shie Mannor,et al.  Q-Cut - Dynamic Discovery of Sub-goals in Reinforcement Learning , 2002, ECML.

[9]  Pierre-Yves Oudeyer,et al.  What is Intrinsic Motivation? A Typology of Computational Approaches , 2007, Frontiers Neurorobotics.

[10]  Andrés Faiña,et al.  Multilevel Darwinist Brain (MDB): Artificial Evolution in a Cognitive Architecture for Real Robots , 2010, IEEE Transactions on Autonomous Mental Development.

[11]  G. Konidaris,et al.  Sensorimotor abstraction selection for efficient, autonomous robot skill acquisition , 2008, 2008 7th IEEE International Conference on Development and Learning.

[12]  Shie Mannor,et al.  Dynamic abstraction in reinforcement learning via clustering , 2004, ICML.