Learning diffeomorphism models of robotic sensorimotor cascades

The problem of bootstrapping consists in designing agents that can learn from scratch the model of their sensorimotor cascade (the series of robot actuators, the external world, and the robot sensors) and use it to achieve useful tasks. In principle, we would want to design agents that can work for any robot dynamics and any robot sensor(s). One of the difficulties of this problem is the fact that the observations are very high dimensional, the dynamics is nonlinear, and there is a wide range of “representation nuisances” to which we would want the agent to be robust. In this paper, we model the dynamics of sensorimotor cascades using diffeomorphisms of the sensel space. We show that this model captures the dynamics of camera and range-finder data, that it can be used for long-term predictions, and that it can capture nonlinear phenomena such as a limited field of view. Moreover, by analyzing the learned diffeomorphisms it is possible to recover the “linear structure” of the dynamics independently of the commands representation.

[1]  M. Hallett,et al.  Functional relevance of cross-modal plasticity in blind humans , 1997, Nature.

[2]  Benjamin Kuipers,et al.  Map Learning with Uninterpreted Sensors and Effectors , 1995, Artif. Intell..

[3]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[4]  Bernhard Schölkopf,et al.  A Unifying View of Wiener and Volterra Theory and Polynomial Kernel Regression , 2006, Neural Computation.

[5]  Jefferson Provost and Benjamin J. Kuipers and Risto Miikkulainen.  Self-Organizing Distinctive State Abstraction Using Options , 2007 .

[6]  Benjamin Kuipers,et al.  An Intellectual History of the Spatial Semantic Hierarchy , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[7]  Giuseppe De Nicolao,et al.  A new kernel-based approach for system identification , 2008, 2008 American Control Conference.

[8]  Benjamin Kuipers,et al.  The initial development of object knowledge by a learning robot , 2008, Robotics Auton. Syst..

[9]  B. Kuipers,et al.  From pixels to policies: A bootstrapping agent , 2008, 2008 7th IEEE International Conference on Development and Learning.

[10]  M. Lassonde,et al.  Cross-modal plasticity for the spatial processing of sounds in visually deprived subjects , 2008, Experimental Brain Research.

[11]  Roland Siegwart,et al.  Characterization of the compact Hokuyo URG-04LX 2D laser range scanner , 2009, 2009 IEEE International Conference on Robotics and Automation.

[12]  Benjamin Kuipers,et al.  Learning the sensorimotor structure of the foveated retina , 2009, EpiRob.

[13]  Benjamin Kuipers,et al.  Sensor Map Discovery for Developing Robots , 2009, AAAI Fall Symposium: Manifold Learning and Its Applications.

[14]  Richard L. Lewis,et al.  Intrinsically Motivated Reinforcement Learning: An Evolutionary Perspective , 2010, IEEE Transactions on Autonomous Mental Development.

[15]  Byron Boots,et al.  Predictive State Temporal Difference Learning , 2010, NIPS.

[16]  Byron Boots,et al.  Closing the learning-planning loop with predictive state representations , 2009, Int. J. Robotics Res..

[17]  Joseph Modayil,et al.  Discovering sensor space: Constructing spatial embeddings that explain sensor correlations , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[18]  Risto Miikkulainen,et al.  Learning geometry from sensorimotor experience , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[19]  Richard M. Murray,et al.  Bootstrapping sensorimotor cascades: A group-theoretic perspective , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  Richard M. Murray,et al.  Bootstrapping bilinear models of robotic sensorimotor cascades , 2011, 2011 IEEE International Conference on Robotics and Automation.

[21]  Evan S. Gawlik,et al.  Geometric, variational discretization of continuum theories , 2010, 1010.4851.

[22]  Richard M. Murray,et al.  Uncertain semantics, representation nuisances, and necessary invariance properties of bootstrapping agents , 2011, 2011 IEEE International Conference on Development and Learning (ICDL).

[23]  Richard M. Murray,et al.  Fault detection and isolation from uninterpreted data in robotic sensorimotor cascades , 2012, 2012 IEEE International Conference on Robotics and Automation.