On-line regression algorithms for learning mechanical models of robots: A survey

With the emergence of more challenging contexts for robotics, the mechanical design of robots is becoming more and more complex. Moreover, their missions often involve unforeseen physical interactions with the environment. To deal with these difficulties, endowing the controllers of the robots with the capability to learn a model of their kinematics and dynamics under changing circumstances is becoming mandatory. This emergent necessity has given rise to a significant amount of research in the Machine Learning community, generating algorithms that address more and more sophisticated on-line modeling questions. In this paper, we provide a survey of the corresponding literature with a focus on the methods rather than on the results. In particular, we provide a unified view of all recent algorithms that outlines their distinctive features and provides a framework for their combination. Finally, we give a prospective account of the evolution of the domain towards more challenging questions.

[1]  Helge J. Ritter,et al.  Self-Organizing Feature Maps for Modeling and Control of Robotic Manipulators , 2003, J. Intell. Robotic Syst..

[2]  R. Schiffer,et al.  INTRODUCTION , 1988, Neurology.

[3]  G. Oriolo,et al.  Robotics: Modelling, Planning and Control , 2008 .

[4]  Laxmidhar Behera,et al.  Visual motor control of a 7DOF redundant manipulator using redundancy preserving learning network , 2009, Robotica.

[5]  Martin V. Butz,et al.  A comparative study: function approximation with LWPR and XCSF , 2010, GECCO '10.

[6]  Jan Peters,et al.  Local Gaussian process regression for real-time model-based robot control , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[7]  Christopher K. I. Williams Prediction with Gaussian Processes: From Linear Regression to Linear Prediction and Beyond , 1999, Learning in Graphical Models.

[8]  Stewart W. Wilson Classifiers that approximate functions , 2002, Natural Computing.

[9]  Jane Wilhelms,et al.  A 'Notion' for interactive behavioral animation control , 1990, IEEE Computer Graphics and Applications.

[10]  F ROSENBLATT,et al.  The perceptron: a probabilistic model for information storage and organization in the brain. , 1958, Psychological review.

[11]  Lars Eldén,et al.  Partial least-squares vs. Lanczos bidiagonalization - I: analysis of a projection method for multiple regression , 2004, Comput. Stat. Data Anal..

[12]  Dana Kulic,et al.  Online Incremental Learning of Inverse Dynamics Incorporating Prior Knowledge , 2011, AIS.

[13]  Manuel Lopes,et al.  Jacobian Learning Methods for Tasks Sequencing in Visual Servoing , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Manuel Lopes,et al.  A learning framework for generic sensory-motor maps , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[15]  Stefan Schaal,et al.  Local Dimensionality Reduction , 1997, NIPS.

[16]  Riccardo Poli,et al.  Genetic and Evolutionary Computation , 2006, Intelligenza Artificiale.

[17]  Brian Scassellati,et al.  Reaching through learned forward model , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[18]  Sethu Vijayakumar,et al.  Methods for Learning Control Policies from Variable-Constraint Demonstrations , 2010, From Motor Learning to Interaction Learning in Robots.

[19]  Aude Billard,et al.  Online Learning of the Body Schema , 2008, Int. J. Humanoid Robotics.

[20]  Antonio Bicchi,et al.  Hands for dexterous manipulation and robust grasping: a difficult road toward simplicity , 2000, IEEE Trans. Robotics Autom..

[21]  Martin V. Butz,et al.  Context-dependent predictions and cognitive arm control with XCSF , 2008, GECCO '08.

[22]  Olivier Sigaud,et al.  Control of redundant robots using learned models: An operational space control approach , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Sethu Vijayakumar,et al.  Adaptive Optimal Control for Redundantly Actuated Arms , 2008, SAB.

[24]  Daniel E. Whitney,et al.  Resolved Motion Rate Control of Manipulators and Human Prostheses , 1969 .

[25]  H. Wold Soft Modelling by Latent Variables: The Non-Linear Iterative Partial Least Squares (NIPALS) Approach , 1975, Journal of Applied Probability.

[26]  T. Kovacs Deletion schemes for classifier systems , 1999 .

[27]  Sukhan Lee,et al.  Robot kinematic control based on bidirectional mapping neural network , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[28]  Martin V. Butz,et al.  Analysis and Improvement of Fitness Exploitation in XCS: Bounding Models, Tournament Selection, and Bilateral Accuracy , 2003, Evolutionary Computation.

[29]  Stefan Schaal,et al.  Local dimensionality reduction for locally weighted learning , 1997, Proceedings 1997 IEEE International Symposium on Computational Intelligence in Robotics and Automation CIRA'97. 'Towards New Computational Principles for Robotics and Automation'.

[30]  Stewart W. Wilson Function approximation with a classifier system , 2001 .

[31]  Daniel H. Grollman,et al.  Sparse incremental learning for interactive robot control policy estimation , 2008, 2008 IEEE International Conference on Robotics and Automation.

[32]  Mitsuo Kawato,et al.  Internal models for motor control and trajectory planning , 1999, Current Opinion in Neurobiology.

[33]  Lennart Ljung,et al.  System Identification: Theory for the User , 1987 .

[34]  Stefan Schaal,et al.  Locally Weighted Projection Regression : An O(n) Algorithm for Incremental Real Time Learning in High Dimensional Space , 2000 .

[35]  Sandeep Gulati,et al.  Neutral learning of constrained nonlinear transformations , 1989, Computer.

[36]  Adi Ben-Israel,et al.  Generalized inverses: theory and applications , 1974 .

[37]  Stefan Schaal,et al.  A Library for Locally Weighted Projection Regression , 2008, J. Mach. Learn. Res..

[38]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[39]  F. Pourboghrat Neural networks for learning inverse-kinematics of redundant manipulators , 1989, Proceedings of the 32nd Midwest Symposium on Circuits and Systems,.

[40]  F. Flentge,et al.  Locally Weighted Interpolating Growing Neural Gas , 2006, IEEE Transactions on Neural Networks.

[41]  Jan Peters,et al.  Model Learning with Local Gaussian Process Regression , 2009, Adv. Robotics.

[42]  Richard S. Sutton,et al.  Dimensions of Reinforcement Learning , 1998 .

[43]  Pierre-Yves Oudeyer,et al.  Incremental local online Gaussian Mixture Regression for imitation learning of multiple tasks , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  Teuvo Kohonen,et al.  Self-Organizing Maps , 2010 .

[45]  Stefan Schaal,et al.  LWPR: A Scalable Method for Incremental Online Learning in High Dimensions , 2005 .

[46]  Andrew A. Goldenberg,et al.  Neural network control of mobile manipulators , 2000, Proceedings. 2000 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2000) (Cat. No.00CH37113).

[47]  Thomas C. Henderson,et al.  A Survey of General- Purpose Manipulation , 1989, Int. J. Robotics Res..

[48]  Stefan Schaal,et al.  Learning inverse kinematics , 2001, Proceedings 2001 IEEE/RSJ International Conference on Intelligent Robots and Systems. Expanding the Societal Role of Robotics in the the Next Millennium (Cat. No.01CH37180).

[49]  Christopher G. Atkeson,et al.  Using locally weighted regression for robot learning , 1991, Proceedings. 1991 IEEE International Conference on Robotics and Automation.

[50]  Manuel Lopes,et al.  Body schema acquisition through active learning , 2010, 2010 IEEE International Conference on Robotics and Automation.

[51]  Michael I. Jordan,et al.  Forward Models: Supervised Learning with a Distal Teacher , 1992, Cogn. Sci..

[52]  Joachim Hoffmann,et al.  Exploiting redundancy for flexible behavior: unsupervised learning in a modular sensorimotor control architecture. , 2007, Psychological review.

[53]  Jan Peters,et al.  Model learning for robot control: a survey , 2011, Cognitive Processing.

[54]  David A. Cohn,et al.  Active Learning with Statistical Models , 1996, NIPS.

[55]  Giorgio Metta,et al.  Incremental learning of robot dynamics using random features , 2011, 2011 IEEE International Conference on Robotics and Automation.

[56]  Stefan Schaal,et al.  Locally Weighted Projection Regression: Incremental Real Time Learning in High Dimensional Space , 2000, ICML.

[57]  Benjamin Recht,et al.  Random Features for Large-Scale Kernel Machines , 2007, NIPS.

[58]  Marc Toussaint,et al.  Learning Multiple Models of Non-linear Dynamics for Control Under Varying Contexts , 2006, ICANN.

[59]  Martin V. Butz,et al.  Learning sensorimotor control structures with XCSF: redundancy exploitation and dynamic control , 2009, GECCO '09.

[60]  Larry Bull,et al.  Foundations of Learning Classifier Systems , 2005 .

[61]  Oussama Khatib,et al.  Dynamic control of manipulator in operational space , 1983 .

[62]  M. Brfiwer A network model for the control of the movement of a redundant manipulator , 2010 .

[63]  Wolfram Burgard,et al.  Unsupervised body scheme learning through self-perception , 2008, 2008 IEEE International Conference on Robotics and Automation.

[64]  Klaus Schulten,et al.  Implementation of self-organizing neural networks for visuo-motor control of an industrial robot , 1993, IEEE Trans. Neural Networks.

[65]  Marc Toussaint,et al.  Active Learning in Motor Control , 2007 .

[66]  J. Taylor,et al.  Playing safe? , 1989, Nursing times.

[67]  Martin V. Butz,et al.  Controlling a Four Degree of Freedom Arm in 3D Using the XCSF Learning Classifier System , 2009, KI.

[68]  Martin Jagersand Image-based predictive display for high d.o.f. uncalibrated tele-manipulation using affine and intensity subspace models , 2001 .

[69]  Pierre-Yves Oudeyer,et al.  Maturationally-constrained competence-based intrinsically motivated learning , 2010, 2010 IEEE 9th International Conference on Development and Learning.

[70]  Martin V. Butz,et al.  Computational Complexity of the XCS Classifier System , 2005 .

[71]  Olivier Sigaud,et al.  From Motor Learning to Interaction Learning in Robots , 2010, From Motor Learning to Interaction Learning in Robots.

[72]  Stefan Ulbrich,et al.  Rapid learning of humanoid body schemas with Kinematic Bézier Maps , 2009, 2009 9th IEEE-RAS International Conference on Humanoid Robots.

[73]  Alan F. Murray,et al.  Synaptic Rewiring for Topographic Map Formation , 2008, ICANN.

[74]  Stefan Schaal,et al.  Learning to Control in Operational Space , 2008, Int. J. Robotics Res..

[75]  Julian F. Miller,et al.  Genetic and Evolutionary Computation — GECCO 2003 , 2003, Lecture Notes in Computer Science.

[76]  G. Lewicki,et al.  Approximation by Superpositions of a Sigmoidal Function , 2003 .

[77]  Jing Peng,et al.  SVM vs regularized least squares classification , 2004, ICPR 2004.

[78]  Andrew Y. Ng,et al.  Fast Gaussian Process Regression using KD-Trees , 2005, NIPS.

[79]  Alexander J. Smola,et al.  Support Vector Regression Machines , 1996, NIPS.

[80]  Goldberg,et al.  Genetic algorithms , 1993, Robust Control Systems with Genetic Algorithms.

[81]  Bernhard Schölkopf,et al.  Learning Inverse Dynamics: a Comparison , 2008, ESANN.

[82]  Sethu Vijayakumar,et al.  Multi-task Gaussian Process Learning of Robot Inverse Dynamics , 2008, NIPS.

[83]  Fumio Kanehiro,et al.  Development of Multi-fingered Hand for Life-size Humanoid Robots , 2008 .

[84]  Simon Osindero,et al.  An Alternative Infinite Mixture Of Gaussian Process Experts , 2005, NIPS.

[85]  Alejandro Hernández Arieta,et al.  Body Schema in Robotics: A Review , 2010, IEEE Transactions on Autonomous Mental Development.

[86]  Bernard Widrow,et al.  Adaptive switching circuits , 1988 .

[87]  Dana Kulic,et al.  Learning inverse dynamics for redundant manipulator control , 2010, 2010 International Conference on Autonomous and Intelligent Systems, AIS 2010.

[88]  John Kenneth Salisbury,et al.  Playing it safe [human-friendly robots] , 2004, IEEE Robotics & Automation Magazine.

[89]  Stefan Schaal,et al.  Scalable Techniques from Nonparametric Statistics for Real Time Robot Learning , 2002, Applied Intelligence.

[90]  Yu Zhang,et al.  Learning Inverse Dynamics by Gaussian process Begrression under the Multi-Task Learning Framework , 2009 .

[91]  Dali Wang,et al.  Solutions of kinematics of robot manipulators using a Kohonen self-organizing neural network , 1997, Proceedings of 12th IEEE International Symposium on Intelligent Control.

[92]  Ali H. Sayed,et al.  Adaptive Filters , 2008 .

[93]  Martin V. Butz,et al.  Bounding the Population Size in XCS to Ensure Reproductive Opportunities , 2003, GECCO.

[94]  Jan Peters,et al.  Using model knowledge for learning inverse dynamics , 2010, 2010 IEEE International Conference on Robotics and Automation.

[95]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[96]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[97]  James Theiler,et al.  Accurate On-line Support Vector Regression , 2003, Neural Computation.

[98]  Jean-Arcady Meyer,et al.  Fast and Incremental Method for Loop-Closure Detection Using Bags of Visual Words , 2008, IEEE Transactions on Robotics.

[99]  Carl E. Rasmussen,et al.  A Unifying View of Sparse Approximate Gaussian Process Regression , 2005, J. Mach. Learn. Res..

[100]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machines , 2002 .

[101]  Stefan Schaal,et al.  Receptive Field Weighted Regression , 1997 .

[102]  Giulio Sandini,et al.  Learning to Exploit Proximal Force Sensing: A Comparison Approach , 2010, From Motor Learning to Interaction Learning in Robots.

[103]  Jan Peters,et al.  Real-Time Local GP Model Learning , 2010, From Motor Learning to Interaction Learning in Robots.

[104]  Nicolas Schweighofer,et al.  Local Online Support Vector Regression for Learning Control , 2007, 2007 International Symposium on Computational Intelligence in Robotics and Automation.

[105]  T. Poggio,et al.  Regularized Least-Squares Classification 133 In practice , although , 2007 .

[106]  Sethu Vijayakumar,et al.  Load estimation and control using learned dynamics models , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[107]  Jan Peters,et al.  Incremental Sparsification for Real-time Online Model Learning , 2010, AISTATS.

[108]  Shie Mannor,et al.  The kernel recursive least-squares algorithm , 2004, IEEE Transactions on Signal Processing.

[109]  Alison Pease,et al.  A Formal Cognitive Model of Mathematical Metaphors , 2009, KI.

[110]  Johan A. K. Suykens,et al.  Least Squares Support Vector Machine Classifiers , 1999, Neural Processing Letters.

[111]  Giulio Sandini,et al.  Active learning for sensorimotor coordinations of autonomous robots , 2009, 2009 2nd Conference on Human System Interactions.

[112]  Brian Scassellati,et al.  A Fast and Efficient Model for Learning to Reach , 2005, Int. J. Humanoid Robotics.

[113]  Zoubin Ghahramani,et al.  Learning Nonlinear Dynamical Systems Using an EM Algorithm , 1998, NIPS.

[114]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[115]  O. Jenkins,et al.  Incremental Nonparametric Bayesian Regression , 2008 .

[116]  Stefan Schaal,et al.  Robot Programming by Demonstration , 2009, Springer Handbook of Robotics.

[117]  Pierre-Yves Oudeyer,et al.  R-IAC: Robust Intrinsically Motivated Exploration and Active Learning , 2009, IEEE Transactions on Autonomous Mental Development.

[118]  Myung-Chul Han,et al.  The estimation for forward kinematic solution of Stewart platform using the neural network , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[119]  Bernhard Schölkopf,et al.  A tutorial on support vector regression , 2004, Stat. Comput..

[120]  Khashayar Khorasani,et al.  Neural network architectures for the forward kinematics problem in robotics , 1990, 1990 IJCNN International Joint Conference on Neural Networks.

[121]  Richard S. Sutton,et al.  Introduction to Reinforcement Learning , 1998 .

[122]  Klaus Schulten,et al.  Topology-conserving maps for learning visuo-motor-coordination , 1989, Neural Networks.

[123]  Bernhard Schölkopf,et al.  New Support Vector Algorithms , 2000, Neural Computation.

[124]  H. Sadjadian,et al.  Numerical Methods for Computing the Forward Kinematics of a Redundant Parallel Manipulator , 2003 .

[125]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[126]  Stewart W. Wilson Classifier Fitness Based on Accuracy , 1995, Evolutionary Computation.

[127]  Andrew G. Barto,et al.  Reinforcement learning , 1998 .

[128]  J. Gani,et al.  Perspectives in Probability and Statistics. , 1980 .

[129]  Helge J. Ritter,et al.  Three-dimensional neural net for learning visuomotor coordination of a robot arm , 1990, IEEE Trans. Neural Networks.

[130]  Martin V. Butz,et al.  Toward a theory of generalization and learning in XCS , 2004, IEEE Transactions on Evolutionary Computation.

[131]  Li-Chen Fu,et al.  Motion Control , 2008, Springer Handbook of Robotics.

[132]  Allon Guez,et al.  On the solution to the inverse kinematic problem , 1990, Proceedings., IEEE International Conference on Robotics and Automation.