Bootstrapping bilinear models of Simple Vehicles

Learning and adaptivity will play a large role in robotics in the future. Two questions are open: (1) in principle, how much it is possible to learn; and (2) in practice, how much should an agent be able to learn. The bootstrapping scenario describes the extreme case in which agents need to learn “everything” from scratch, including a torque-to-pixels model for its robotic body. This paper considers the bootstrapping problem for a subset of the set of all robots. The Simple Vehicles are an idealization of mobile robots equipped with a set of “canonical” exteroceptive sensors: the camera, the range finder and the field sampler. The sensorimotor dynamics of these sensors are derived and shown to be surprising similar. These sensorimotor dynamics are well approximated by a class of nonlinear systems that assume an instantaneous bilinear relation among observations, commands, and changes in the observations. The bilinear approximation is sufficient to guarantee success in the task of generalized “servoing”: driving the observations to a given goal snapshot. Simulations and experiments substantiate the theoretical results. This is the first instance of a bootstrapping agent that can learn the model of the dynamics of a relatively large universe of systems and use the models to solve well-defined tasks, with no parameter tuning or hand-designed features.

[1]  W. Ashby,et al.  Every Good Regulator of a System Must Be a Model of That System , 1970 .

[2]  Bruce A. Francis,et al.  The internal model principle of control theory , 1976, Autom..

[3]  R. W. Brockett,et al.  Asymptotic stability and feedback stabilization , 1982 .

[4]  V. Braitenberg.  Vehicles, Experiments in Synthetic Psychology , 1984 .

[5]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[6]  Benjamin Kuipers,et al.  Navigation and Mapping in Large Scale Space , 1988, AI Mag..

[7]  Benjamin Kuipers,et al.  A robot exploration and mapping strategy based on a semantic hierarchy of spatial representations , 1991, Robotics Auton. Syst..

[8]  D. Kendall,et al.  The Riemannian Structure of Euclidean Shape Spaces: A Novel Environment for Statistics , 1993 .

[9]  Christopher M. Bishop,et al.  Neural Network for Pattern Recognition , 1995 .

[10]  Yuandan Lin,et al.  A Smooth Converse Lyapunov Theorem for Robust Stability , 1996 .

[11]  M. Hallett,et al.  Functional relevance of cross-modal plasticity in blind humans , 1997, Nature.

[12]  Benjamin Kuipers,et al.  Map Learning with Uninterpreted Sensors and Effectors , 1995, Artif. Intell..

[13]  Bart De Moor,et al.  Subspace identification of bilinear systems subject to white inputs , 1999, IEEE Trans. Autom. Control..

[14]  S. Sastry.  Nonlinear Systems: Analysis, Stability, and Control , 1999 .

[15]  Benjamin Kuipers,et al.  The Spatial Semantic Hierarchy , 2000, Artif. Intell..

[16]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[17]  Richard S. Sutton,et al.  Predictive Representations of State , 2001, NIPS.

[18]  Yiannis Aloimonos,et al.  Polydioptric camera design and 3D motion estimation , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[19]  D. Mumford,et al.  Riemannian Geometries on Spaces of Plane Curves , 2003, math/0312384.

[20]  Giulio Sandini,et al.  Developmental robotics: a survey , 2003, Connect. Sci..

[21]  Benjamin Kuipers,et al.  Towards a general theory of topological maps , 2004, Artif. Intell..

[22]  Michael R. James,et al.  Predictive State Representations: A New Theory for Modeling Dynamical Systems , 2004, UAI.

[23]  Shree K. Nayar,et al.  The Raxel Imaging Model and Ray-Based Calibration , 2005, International Journal of Computer Vision.

[24]  S. Foss,et al.  AN OVERVIEW OF SOME STOCHASTIC STABILITY METHODS( Network Design, Control and Optimization) , 2004 .

[25]  Cédric Pradalier,et al.  Perceptual navigation around a sensori-motor trajectory , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[26]  Heriot-Watt University.  AN OVERVIEW OF SOME STOCHASTIC STABILITY METHODS , 2004 .

[27]  Vincent Verdult,et al.  Kernel methods for subspace identification of multivariable LPV and bilinear systems , 2005, Autom..

[28]  片山 徹.  Subspace methods for system identification , 2005 .

[29]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[30]  François Chaumette,et al.  Visual servo control. I. Basic approaches , 2006, IEEE Robotics & Automation Magazine.

[31]  Benjamin Kuipers,et al.  Bootstrap learning of foundational representations , 2006, Connect. Sci..

[32]  Yee Whye Teh,et al.  A Fast Learning Algorithm for Deep Belief Nets , 2006, Neural Computation.

[33]  Petros A. Ioannou,et al.  Adaptive Control Tutorial (Advances in Design and Control) , 2006 .

[34]  Bernhard Schölkopf,et al.  A Unifying View of Wiener and Volterra Theory and Polynomial Kernel Regression , 2006, Neural Computation.

[35]  S. Benhamou.  HOW MANY ANIMALS REALLY DO THE LÉVY WALK , 2007 .

[36]  Geoffrey E. Hinton,et al.  Unsupervised Learning of Image Transformations , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Pieter Abbeel,et al.  Hierarchical Apprenticeship Learning with Application to Quadruped Locomotion , 2007, NIPS.

[38]  Simon Benhamou,et al.  How many animals really do the Lévy walk? , 2008, Ecology.

[39]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[40]  Jefferson Provost and Benjamin J. Kuipers and Risto Miikkulainen.  Self-Organizing Distinctive State Abstraction Using Options , 2007 .

[41]  Geoffrey E. Hinton,et al.  The Recurrent Temporal Restricted Boltzmann Machine , 2008, NIPS.

[42]  Benjamin Kuipers,et al.  An Intellectual History of the Spatial Semantic Hierarchy , 2008, Robotics and Cognitive Approaches to Spatial Mapping.

[43]  Christophe Collewet,et al.  Visual servoing set free from image processing , 2008, 2008 IEEE International Conference on Robotics and Automation.

[44]  Benjamin Kuipers,et al.  The initial development of object knowledge by a learning robot , 2008, Robotics Auton. Syst..

[45]  Wisama Khalil,et al.  Model Identification , 2019, Springer Handbook of Robotics, 2nd Ed..

[46]  Benjamin Kuipers,et al.  Drinking from the firehose of experience , 2008, Artif. Intell. Medicine.

[47]  B. Kuipers,et al.  From pixels to policies: A bootstrapping agent , 2008, 2008 7th IEEE International Conference on Development and Learning.

[48]  Andy Reynolds,et al.  How many animals really do the Lévy walk? Comment. , 2008, Ecology.

[49]  Aude Billard,et al.  Online Learning of the Body Schema , 2008, Int. J. Humanoid Robotics.

[50]  Michel Verhaegen,et al.  Subspace identification of Bilinear and LPV systems for open- and closed-loop data , 2009, Autom..

[51]  Antonio Bicchi,et al.  Visual Servoing in the Large , 2009, Int. J. Robotics Res..

[52]  G. Chirikjian.  Stochastic Models, Information Theory, and Lie Groups, Volume 1 , 2009 .

[53]  Tobi Delbrück,et al.  Getting to Know Your Neighbors: Unsupervised Learning of Topography from Real-World, Event-Based Input , 2009, Neural Computation.

[54]  Roland Siegwart,et al.  Characterization of the compact Hokuyo URG-04LX 2D laser range scanner , 2009, 2009 IEEE International Conference on Robotics and Automation.

[55]  Alcherio Martinoli,et al.  Theoretical analysis of three bio-inspired plume tracking algorithms , 2009, 2009 IEEE International Conference on Robotics and Automation.

[56]  Alexander Stoytchev,et al.  Some Basic Principles of Developmental Robotics , 2009, IEEE Transactions on Autonomous Mental Development.

[57]  Carl E. Rasmussen,et al.  Gaussian processes for machine learning , 2005, Adaptive computation and machine learning.

[58]  Benjamin Kuipers,et al.  Learning the sensorimotor structure of the foveated retina , 2009, EpiRob.

[59]  D. Elliott.  Bilinear Control Systems: Matrices in Action , 2009 .

[60]  Benjamin Kuipers,et al.  Sensor Map Discovery for Developing Robots , 2009, AAAI Fall Symposium: Manifold Learning and Its Applications.

[61]  David L. Elliott,et al.  Bilinear Control Systems , 2009 .

[62]  Dieter Fox,et al.  Learning GP-BayesFilters via Gaussian process latent variable models , 2009, Auton. Robots.

[63]  Masaki Ogino,et al.  Cognitive Developmental Robotics: A Survey , 2009, IEEE Transactions on Autonomous Mental Development.

[64]  Wolfram Burgard,et al.  Body schema learning for robotic manipulators from visual self-perception , 2009, Journal of Physiology - Paris.

[65]  E. Bai,et al.  Block Oriented Nonlinear System Identification , 2010 .

[66]  Geoffrey E. Hinton,et al.  Learning to combine foveal glimpses with a third-order Boltzmann machine , 2010, NIPS.

[67]  Geoffrey E. Hinton,et al.  Learning to Represent Spatial Transformations with Factored Higher-Order Boltzmann Machines , 2010, Neural Computation.

[68]  Jens-Steffen Gutmann,et al.  Vector field SLAM , 2010, 2010 IEEE International Conference on Robotics and Automation.

[69]  David J. Fleet,et al.  Dynamical binary latent variable models for 3D human pose tracking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[70]  Yann LeCun,et al.  Convolutional Learning of Spatio-temporal Features , 2010, ECCV.

[71]  Richard M. Murray,et al.  A bio-plausible design for visual pose stabilization , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[72]  Byron Boots,et al.  Reduced-Rank Hidden Markov Models , 2009, AISTATS.

[73]  Manuel Lopes,et al.  Body schema acquisition through active learning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[74]  Alejandro Hernández Arieta,et al.  Body Schema in Robotics: A Review , 2010, IEEE Transactions on Autonomous Mental Development.

[75]  Francesco Orabona,et al.  Discrete camera calibration from pixel streams , 2010, Comput. Vis. Image Underst..

[76]  Byron Boots,et al.  Predictive State Temporal Difference Learning , 2010, NIPS.

[77]  Geoffrey E. Hinton,et al.  Modeling pixel means and covariances using factorized third-order boltzmann machines , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[78]  Joseph Modayil,et al.  Discovering sensor space: Constructing spatial embeddings that explain sensor correlations , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[79]  Changhai Xu,et al.  Towards the Object Semantic Hierarchy , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[80]  Richard M. Murray,et al.  Bootstrapping sensorimotor cascades: A group-theoretic perspective , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[81]  Dieter Fox,et al.  Learning GP-BayesFilters via Gaussian process latent variable models , 2009, Auton. Robots.

[82]  L. Rosasco.  THE COMPUTATIONAL MAGIC OF THE VENTRAL STREAM , 2011 .

[83]  Richard M. Murray,et al.  Bootstrapping bilinear models of robotic sensorimotor cascades , 2011, 2011 IEEE International Conference on Robotics and Automation.

[84]  Bart De Moor,et al.  Subspace Identification for Linear Systems: Theory ― Implementation ― Applications , 2011 .

[85]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[86]  Stefano Soatto,et al.  Steps Towards a Theory of Visual Information: Active Perception, Signal-to-Symbol Conversion and the Interplay Between Sensing and Control , 2011, ArXiv.

[87]  John D. Lafferty,et al.  Learning image representations from the pixel level via hierarchical sparse coding , 2011, CVPR 2011.

[88]  Richard M. Murray,et al.  Learning diffeomorphism models of robotic sensorimotor cascades , 2012, 2012 IEEE International Conference on Robotics and Automation.

[89]  Nitish Srivastava,et al.  Multimodal learning with deep Boltzmann machines , 2012, J. Mach. Learn. Res..

[90]  Wolfram Burgard,et al.  Body Schema Learning , 2012, Towards Service Robots for Everyday Environments.

[91]  Richard M. Murray,et al.  Fault detection and isolation from uninterpreted data in robotic sensorimotor cascades , 2012, 2012 IEEE International Conference on Robotics and Automation.

[92]  Dong Yu,et al.  Tensor Deep Stacking Networks , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[93]  Davide Scaramuzza,et al.  Calibration by Correlation Using Metric Embedding from Nonmetric Similarities , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[94]  Giuseppe Oriolo,et al.  Simultaneous Calibration of Odometry and Sensor Parameters for Mobile Robots , 2013, IEEE Transactions on Robotics.

[95]  Jan Peters,et al.  Reinforcement learning in robotics: A survey , 2013, Int. J. Robotics Res..

[96]  Andrea Censi,et al.  Bootstrapping vehicles : a formal approach to unsupervised sensorimotor learning based on invariance , 2013 .

[97]  S. Shankar Sastry,et al.  Provably safe and robust learning-based model predictive control , 2011, Autom..

[98]  Eric E. Thomson,et al.  Perceiving Invisible Light through a Somatosensory Cortical Prosthesis , 2013, Nature Communications.

[99]  Richard M. Murray,et al.  Motion planning in observations space with learned diffeomorphism models , 2013, 2013 IEEE International Conference on Robotics and Automation.

[100]  Éric Marchand,et al.  Photometric visual servoing for omnidirectional cameras , 2013, Auton. Robots.

[101]  Dean Alderucci.  A SPECTRAL ALGORITHM FOR LEARNING HIDDEN MARKOV MODELS THAT HAVE SILENT STATES , 2015 .