Efficient and Interpretable Robot Manipulation with Graph Neural Networks

Many manipulation tasks can be naturally cast as a sequence of spatial relationships and constraints between objects. We aim to discover and scale these task-specific spatial relationships by representing manipulation tasks as operations over graphs. To do this, we pose manipulating a large, variable number of objects as a probabilistic classification problem over actions, objects and goals, learned using graph neural networks (GNNs). Our formulation first transforms the environment into a graph representation, then applies a trained GNN policy to predict which object to manipulate towards which goal state. Our GNN policies are trained using very few expert demonstrations on simple tasks, and exhibits generalization over number and configurations of objects in the environment and even to new, more complex tasks, and provide interpretable explanations for their decision-making. We present experiments which show that a single learned GNN policy can solve a variety of blockstacking tasks in both simulation and real hardware.

[1]  Leslie Pack Kaelbling,et al.  Learning to guide task and motion planning using score-space representation , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[2]  Rachid Alami,et al.  Combining symbolic and geometric planning to synthesize human-aware plans: toward more efficient combined search. , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Brandon M. Greenwell,et al.  Interpretable Machine Learning , 2019, Hands-On Machine Learning with R.

[4]  Leslie Pack Kaelbling,et al.  Active Model Learning and Diverse Action Sampling for Task and Motion Planning , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Dylan Hadfield-Menell,et al.  Guided search for task and motion plans using learned heuristics , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[6]  Leslie Pack Kaelbling,et al.  Integrated Task and Motion Planning , 2020, Annu. Rev. Control. Robotics Auton. Syst..

[7]  Jung-Su Ha,et al.  Deep Visual Reasoning: Learning to Predict Action Sequences for Task and Motion Planning from an Initial Scene Image , 2020, Robotics: Science and Systems.

[8]  Xiaoning Qian,et al.  Variational Graph Recurrent Neural Networks , 2019, NeurIPS.

[9]  Sergey Levine,et al.  Learning Complex Dexterous Manipulation with Deep Reinforcement Learning and Demonstrations , 2017, Robotics: Science and Systems.

[10]  Abhinav Gupta,et al.  Object-centric Forward Modeling for Model Predictive Control , 2019, CoRL.

[11]  Jure Leskovec,et al.  Inductive Representation Learning on Large Graphs , 2017, NIPS.

[12]  Pietro Liò,et al.  Graph Attention Networks , 2017, ICLR.

[13]  Lydia E. Kavraki,et al.  Learning Feasibility for Task and Motion Planning in Tabletop Environments , 2019, IEEE Robotics and Automation Letters.

[14]  Humbert Fiorino,et al.  HDDL: An Extension to PDDL for Expressing Hierarchical Planning Problems , 2020, AAAI.

[15]  Jan Eric Lenssen,et al.  Fast Graph Representation Learning with PyTorch Geometric , 2019, ArXiv.

[16]  Marcin Andrychowicz,et al.  Solving Rubik's Cube with a Robot Hand , 2019, ArXiv.

[17]  Jure Leskovec,et al.  GNNExplainer: Generating Explanations for Graph Neural Networks , 2019, NeurIPS.

[18]  Razvan Pascanu,et al.  Relational inductive biases, deep learning, and graph networks , 2018, ArXiv.

[19]  Leslie Pack Kaelbling,et al.  Learning composable models of parameterized skills , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Sergey Levine,et al.  Relay Policy Learning: Solving Long-Horizon Tasks via Imitation and Reinforcement Learning , 2019, CoRL.

[21]  Allan Jabri,et al.  Towards Practical Multi-Object Manipulation using Relational Reinforcement Learning , 2019, 2020 IEEE International Conference on Robotics and Automation (ICRA).

[22]  Leslie Pack Kaelbling,et al.  Hierarchical task and motion planning in the now , 2011, 2011 IEEE International Conference on Robotics and Automation.

[23]  Leslie Pack Kaelbling,et al.  Learning compositional models of robot skills for task and motion planning , 2021, Int. J. Robotics Res..

[24]  Martin Grohe,et al.  Weisfeiler and Leman Go Neural: Higher-order Graph Neural Networks , 2018, AAAI.

[25]  Kelsey R. Allen,et al.  Learning constraint-based planning models from demonstrations , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[26]  Leslie Pack Kaelbling,et al.  From Skills to Symbols: Learning Symbolic Representations for Abstract High-Level Planning , 2018, J. Artif. Intell. Res..

[27]  Maria Fox,et al.  PDDL2.1: An Extension to PDDL for Expressing Temporal Planning Domains , 2003, J. Artif. Intell. Res..

[28]  Geoffrey J. Gordon,et al.  A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning , 2010, AISTATS.

[29]  Rafael Muñoz-Salinas,et al.  Speeded up detection of squared fiducial markers , 2018, Image Vis. Comput..

[30]  Sendai,et al.  2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) , 2020 .

[31]  Francisco José Madrid-Cuevas,et al.  Generation of fiducial marker dictionaries using Mixed Integer Linear Programming , 2016, Pattern Recognit..

[32]  Edward Johns,et al.  My House, My Rules: Learning Tidying Preferences with Graph Neural Networks , 2021, CoRL.

[33]  Jitendra Malik,et al.  Learning Visual Predictive Models of Physics for Playing Billiards , 2015, ICLR.

[34]  Silvio Savarese,et al.  Neural Task Graphs: Generalizing to Unseen Tasks From a Single Video Demonstration , 2018, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[35]  Sergey Levine,et al.  End-to-End Training of Deep Visuomotor Policies , 2015, J. Mach. Learn. Res..

[36]  Jitendra Malik,et al.  Habitat 2.0: Training Home Assistants to Rearrange their Habitat , 2021, ArXiv.

[37]  Alec Radford,et al.  Proximal Policy Optimization Algorithms , 2017, ArXiv.

[38]  Leslie Pack Kaelbling,et al.  Planning with Learned Object Importance in Large Problem Instances using Graph Neural Networks , 2020, AAAI.

[39]  Natalia Gimelshein,et al.  PyTorch: An Imperative Style, High-Performance Deep Learning Library , 2019, NeurIPS.

[40]  Oliver Kroemer,et al.  Graph-Structured Visual Imitation , 2019, CoRL.

[41]  Sergey Levine,et al.  DeepMimic , 2018, ACM Trans. Graph..

[42]  Beomjoon Kim,et al.  Learning value functions with relational state representations for guiding task-and-motion planning , 2019, CoRL.

[43]  Richard S. Zemel,et al.  Gated Graph Sequence Neural Networks , 2015, ICLR.