Representation, learning, and planning algorithms for geometric task and motion planning

We present a framework for learning to guide geometric task-and-motion planning (G-TAMP). G-TAMP is a subclass of task-and-motion planning in which the goal is to move multiple objects to target regions among movable obstacles. A standard graph search algorithm is not directly applicable, because G-TAMP problems involve hybrid search spaces and expensive action feasibility checks. To handle this, we introduce a novel planner that extends basic heuristic search with random sampling and a heuristic function that prioritizes feasibility checking on promising state–action pairs. The main drawback of such pure planners is that they lack the ability to learn from planning experience to improve their efficiency. We propose two learning algorithms to address this. The first is an algorithm for learning a rank function that guides the discrete task-level search, and the second is an algorithm for learning a sampler that guides the continuous motion-level search. We propose design principles for designing data-efficient algorithms for learning from planning experience and representations for effective generalization. We evaluate our framework in challenging G-TAMP problems, and show that we can improve both planning and data efficiency.

[1]  Leslie Pack Kaelbling,et al.  Integrated Task and Motion Planning , 2020, Annu. Rev. Control. Robotics Auton. Syst..

[2]  Leslie Pack Kaelbling,et al.  CAMPs: Learning Context-Specific Abstractions for Efficient Planning in Factored MDPs , 2020, CoRL.

[3]  Jung-Su Ha,et al.  Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image , 2020, Robotics: Science and Systems.

[4]  Jung-Su Ha,et al.  Deep Visual Heuristics: Learning Feasibility of Mixed-Integer Programs for Manipulation Planning , 2020, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[5]  Caelan Reed Garrett,et al.  Online Replanning in Belief Space for Partially Observable Task and Motion Problems , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Leslie Pack Kaelbling,et al.  Adversarial Actor-Critic Method for Task and Motion Planning Problems Using Planning Experience , 2019, AAAI.

[7]  Zhiyuan Liu,et al.  Graph Neural Networks: A Review of Methods and Applications , 2018, AI Open.

[8]  Leslie Pack Kaelbling,et al.  Learning Quickly to Plan Quickly Using Modular Meta-Learning , 2018, 2019 International Conference on Robotics and Automation (ICRA).

[9]  Jessica B. Hamrick,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[10]  Leslie Pack Kaelbling,et al.  Guiding Search in Continuous State-Action Spaces by Learning an Action Sampler From Off-Target Search Experience , 2018, AAAI.

[11]  Demis Hassabis,et al.  Mastering the game of Go without human knowledge , 2017, Nature.

[12]  Léon Bottou,et al.  Wasserstein Generative Adversarial Networks , 2017, ICML.

[13]  Caelan Reed Garrett,et al.  Sample-Based Methods for Factored Task and Motion Planning , 2017, Robotics: Science and Systems.

[14]  Yiming Yang,et al.  MMD GAN: Towards Deeper Understanding of Moment Matching Network , 2017, NIPS.

[15]  Leslie Pack Kaelbling,et al.  Learning to guide task and motion planning using score-space representation , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Samuel S. Schoenholz,et al.  Neural Message Passing for Quantum Chemistry , 2017, ICML.

[17]  Aaron C. Courville,et al.  Improved Training of Wasserstein GANs , 2017, NIPS.

[18]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[19]  Leslie Pack Kaelbling,et al.  FFRob: Leveraging symbolic planning for efficient task and motion planning , 2016, Int. J. Robotics Res..

[20]  Leslie Pack Kaelbling,et al.  Learning to Rank for Synthesizing Planning Heuristics , 2016, IJCAI.

[21]  Siddhartha S. Srinivasa,et al.  Rearrangement planning using object-centric and robot-centric action spaces , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Dylan Hadfield-Menell,et al.  Guided search for task and motion plans using learned heuristics , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[23]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[24]  Yuval Tassa,et al.  Continuous control with deep reinforcement learning , 2015, ICLR.

[25]  Marc Toussaint,et al.  Logic-Geometric Programming: An Optimization-Based Approach to Combined Task and Motion Planning , 2015, IJCAI.

[26]  Kostas E. Bekris,et al.  Dealing with Difficult Instances of Object Rearrangement , 2015, Robotics: Science and Systems.

[27]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[28]  Pieter Abbeel,et al.  Motion planning with sequential convex optimization and convex collision checking , 2014, Int. J. Robotics Res..

[29]  Aaron C. Courville,et al.  Generative adversarial networks , 2014, Commun. ACM.

[30]  Kris K. Hauser,et al.  The minimum constraint removal problem with three robotics applications , 2014, Int. J. Robotics Res..

[31]  Max Welling,et al.  Auto-Encoding Variational Bayes , 2013, ICLR.

[32]  Leslie Pack Kaelbling,et al.  Integrated task and motion planning in belief space , 2013, Int. J. Robotics Res..

[33]  Siddhartha S. Srinivasa,et al.  CHOMP: Covariant Hamiltonian optimization for motion planning , 2013, Int. J. Robotics Res..

[34]  Bernhard Nebel,et al.  The FF Planning System: Fast Plan Generation Through Heuristic Search , 2011, J. Artif. Intell. Res..

[35]  Leslie Pack Kaelbling,et al.  Hierarchical task and motion planning in the now , 2011, 2011 IEEE International Conference on Robotics and Automation.

[36]  Erez Karpas,et al.  To Max or Not to Max: Online Learning for Speeding Up Optimal Planning , 2010, AAAI.

[37]  Andreas Krause,et al.  Information-Theoretic Regret Bounds for Gaussian Process Optimization in the Bandit Setting , 2009, IEEE Transactions on Information Theory.

[38]  R. Alami,et al.  A Hybrid Approach to Intricate Motion, Manipulation and Task Planning , 2009, Int. J. Robotics Res..

[39]  Tamim Asfour,et al.  Manipulation Planning Among Movable Obstacles , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[40]  Michael Fink,et al.  Online Learning of Search Heuristics , 2007, AISTATS.

[41]  Robert Givan,et al.  Learning Heuristic Functions from Relaxed Plans , 2006, ICAPS.

[42]  Malte Helmert,et al.  The Fast Downward Planning System , 2006, J. Artif. Intell. Res..

[43]  F. Scarselli,et al.  A new model for learning in graph domains , 2005, Proceedings. 2005 IEEE International Joint Conference on Neural Networks, 2005..

[44]  Thomas Hofmann,et al.  Large Margin Methods for Structured and Interdependent Output Variables , 2005, J. Mach. Learn. Res..

[45]  James J. Kuffner,et al.  Navigation among movable obstacles: real-time reasoning in complex environments , 2004, 4th IEEE/RAS International Conference on Humanoid Robots, 2004..

[46]  Steven M. LaValle,et al.  RRT-connect: An efficient approach to single-query path planning , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[47]  Alessandro Sperduti,et al.  Supervised neural networks for the classification of structures , 1997, IEEE Trans. Neural Networks.

[48]  M. Veloso,et al.  Robotics and autonomous systems , 1988, Robotics Auton. Syst..

[49]  Beomjoon Kim,et al.  Learning value functions with relational state representations for guiding task-and-motion planning , 2019, CoRL.

[50]  Philip S. Yu,et al.  A Comprehensive Survey on Graph Neural Networks , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[51]  Derong Liu,et al.  Neural Information Processing , 2017, Lecture Notes in Computer Science.

[52]  FernAlan,et al.  Learning partial policies to speedup MDP tree search via reduction to I.I.D. learning , 2017 .

[53]  Takeo Kanade,et al.  Automated Construction of Robotic Manipulation Programs , 2010 .

[54]  Ah Chung Tsoi,et al.  The Graph Neural Network Model , 2009, IEEE Transactions on Neural Networks.

[55]  Rachid Alami,et al.  aSyMov: A Planner That Deals with Intricate Symbolic and Geometric Problems , 2003, ISRR.

[56]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.