Two Perspectives on Learning Rich Representations from Robot Experience

This position paper describes two approaches towards the representations that a robot can learn from its experience. In the first approach, the robot learns models for reasoning about human-interpretable aspects of the environment, for example models of space and objects. In the second approach, the robot incrementally learns predictions for the consequences of performing policies, where a policy is any experimental procedure that the robot can perform. These two approaches correspond closely to the ideas of a scientific model and an experimental prediction, and ideally the benefits of both can be accessible to a robot.

[1]  Luc Steels,et al.  The Origins of Syntax in Visually Grounded Robotic Agents , 1997, IJCAI.

[2]  Richard S. Sutton,et al.  GQ(lambda): A general gradient algorithm for temporal-difference prediction learning with eligibility traces , 2010, Artificial General Intelligence.

[3]  Patrick M. Pilarski,et al.  Horde: a scalable real-time architecture for learning knowledge from unsupervised sensorimotor interaction , 2011, AAMAS.

[4]  A. Gopnik The scientist in the crib , 1999 .

[5]  Alborz Geramifard,et al.  Dyna-Style Planning with Linear Function Approximation and Prioritized Sweeping , 2008, UAI.

[6]  Doina Precup,et al.  Between MDPs and Semi-MDPs: A Framework for Temporal Abstraction in Reinforcement Learning , 1999, Artif. Intell..

[7]  R. Sutton The Grand Challenge of Predictive Empirical Abstract Knowledge , 2009 .

[8]  R. Sutton,et al.  GQ(λ): A general gradient algorithm for temporal-difference prediction learning with eligibility traces , 2010 .

[9]  Joseph Modayil,et al.  Discovering sensor space: Constructing spatial embeddings that explain sensor correlations , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[10]  Benjamin Kuipers,et al.  Map Learning with Uninterpreted Sensors and Effectors , 1995, Artif. Intell..

[11]  Richard S. Sutton,et al.  Multi-timescale nexting in a reinforcement learning robot , 2011, Adapt. Behav..

[12]  Richard S. Sutton,et al.  Learning to predict by the methods of temporal differences , 1988, Machine Learning.

[13]  Benjamin Kuipers,et al.  Autonomous Development of a Grounded Object Ontology by a Learning Robot , 2007, AAAI.

[14]  Benjamin Kuipers,et al.  The Spatial Semantic Hierarchy , 2000, Artif. Intell..