Fast motion planning from experience: trajectory prediction for speeding up movement generation

Trajectory planning and optimization is a fundamental problem in articulated robotics. Algorithms used typically for this problem compute optimal trajectories from scratch in a new situation. In effect, extensive data is accumulated containing situations together with the respective optimized trajectories—but this data is in practice hardly exploited. This article describes a novel method to learn from such data and speed up motion generation, a method we denote tajectory pediction. The main idea is to use demonstrated optimal motions to quickly predict appropriate trajectories for novel situations. These can be used to initialize and thereby drastically speed-up subsequent optimization of robotic movements. Our approach has two essential ingredients. First, to generalize from previous situations to new ones we need a situation descriptor—we construct features for such descriptors and use a sparse regularized feature selection approach to improve generalization. Second, the transfer of previously optimized trajectories to a new situation should not be made in joint angle space—we propose a more efficient task space transfer. We present extensive results in simulation to illustrate the benefits of the new method, and demonstrate it also with real robot hardware. Our experiments in diverse tasks show that we can predict good motion trajectories in new situations for which the refinement is much faster than an optimization from scratch.

[1]  M. Ciletti,et al.  The computation and theory of optimal control , 1972 .

[2]  Alberto Elfes,et al.  Using occupancy grids for mobile robot perception and navigation , 1989, Computer.

[3]  Dean Pomerleau,et al.  Efficient Training of Artificial Neural Networks for Autonomous Navigation , 1991, Neural Computation.

[4]  Christopher G. Atkeson,et al.  Using Local Trajectory Optimizers to Speed Up Global Optimization in Dynamic Programming , 1993, NIPS.

[5]  Lydia E. Kavraki,et al.  Randomized query processing in robot path planning , 1995, STOC '95.

[6]  Jianwei Zhang,et al.  An Enhanced Optimization Approach for Generating Smooth Robot Trajectories in the Presence of Obstacles , 1995 .

[7]  R. Sutton,et al.  Macro-Actions in Reinforcement Learning: An Empirical Analysis , 1998 .

[8]  Kazuo Hiraki,et al.  From Egocentric to Allocentric Spatial Behavior: A Computational Model of Spatial Development , 1998, Adapt. Behav..

[9]  M. Carpenter,et al.  Three sources of information in social learning , 2002 .

[10]  Manuela M. Veloso,et al.  Real-Time Randomized Path Planning for Robot Navigation , 2002, RoboCup.

[11]  Edwin D. de Jong,et al.  Context-based policy search: transfer of experience across problems , 2002 .

[12]  K. Dautenhahn,et al.  Imitation in Animals and Artifacts , 2002 .

[13]  Ubbo Visser,et al.  Egocentric qualitative spatial knowledge representation for physical robots , 2004, Robotics Auton. Syst..

[14]  Marc Toussaint,et al.  Rprop Using the Natural Gradient , 2005 .

[15]  E. Todorov,et al.  A generalized iterative LQG method for locally-optimal feedback control of constrained nonlinear stochastic systems , 2005, Proceedings of the 2005, American Control Conference, 2005..

[16]  Aude Billard,et al.  Recognition and reproduction of gestures using a probabilistic framework combining PCA, ICA and HMM , 2005, ICML.

[17]  Andrew G. Barto,et al.  Autonomous shaping: knowledge transfer in reinforcement learning , 2006, ICML.

[18]  Steven M. LaValle,et al.  Planning algorithms , 2006 .

[19]  Tamim Asfour,et al.  An integrated approach to inverse kinematics and path planning for redundant manipulators , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[20]  Gerd Hirzinger,et al.  Capturing robot workspace structure: representing robot capabilities , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[21]  Marcelo Kallmann,et al.  Learning humanoid reaching tasks in dynamic environments , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Rajesh P. N. Rao,et al.  Towards a Real-Time Bayesian Imitation System for a Humanoid Robot , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[23]  Sean R. Martin,et al.  Offline and Online Evolutionary Bi-Directional RRT Algorithms for Efficient Re-Planning in Dynamic Environments , 2007, 2007 IEEE International Conference on Automation Science and Engineering.

[24]  Christopher G. Atkeson,et al.  Transfer of policies based on trajectory libraries , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Florent Lamiraux,et al.  Motion planning for humanoid robots in environments modeled by vision , 2008, Humanoids 2008 - 8th IEEE-RAS International Conference on Humanoid Robots.

[26]  Ross A. Knepper,et al.  Path and trajectory diversity: Theory and algorithms , 2008, 2008 IEEE International Conference on Robotics and Automation.

[27]  James J. Kuffner,et al.  Adaptive workspace biasing for sampling-based planners , 2008, 2008 IEEE International Conference on Robotics and Automation.

[28]  Siddhartha S. Srinivasa,et al.  CHOMP: Gradient optimization techniques for efficient motion planning , 2009, 2009 IEEE International Conference on Robotics and Automation.

[29]  Brett Browning,et al.  A survey of robot learning from demonstration , 2009, Robotics Auton. Syst..

[30]  Marc Toussaint,et al.  Robot trajectory optimization using approximate inference , 2009, ICML '09.

[31]  Marc Toussaint,et al.  Trajectory prediction: learning to map situations to robot trajectories , 2009, ICML '09.

[32]  Jochen J. Steil,et al.  Automatic selection of task spaces for imitation learning , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Marc Toussaint,et al.  Trajectory prediction in cluttered voxel environments , 2010, 2010 IEEE International Conference on Robotics and Automation.

[34]  Jan Peters,et al.  Reinforcement Learning to Adjust Robot Movements to New Situations , 2010, IJCAI.

[35]  Wolfram Burgard,et al.  Robotics: Science and Systems XV , 2010 .

[36]  Jun Morimoto,et al.  Task-Specific Generalization of Discrete and Periodic Dynamic Movement Primitives , 2010, IEEE Transactions on Robotics.

[37]  Gerd Hirzinger,et al.  Trajectory planning for optimal robot catching in real-time , 2011, 2011 IEEE International Conference on Robotics and Automation.