GPDAN: Grasp Pose Domain Adaptation Network for Sim-to-Real 6-DoF Object Grasping

In this letter, we propose a novel Grasp Pose Domain Adaptation Network (GPDAN) to achieve sim-to-real domain adaptation for 6-DoF grasp pose detection. The main task of GPDAN is to detect feasible 6-DoF grasp poses in cluttered scenes. A point-wise self-supervised domain classification module with point cloud mixture and feature fusion strategy is proposed as the auxiliary task to promote the feature alignment between the source and target domain through adversarial training. Experimental results on both simulation and real-world environments demonstrate that GPDAN outperforms other approaches in detecting 6-DoF grasps on the target domain, highlighting the effectiveness of GPDAN in improving the performance of 6-DoF grasp pose detectors trained in simulation and deployed in real-world environments without any further laborious labeling.

[1]  Manning Wang,et al.  A learnable self-supervised task for unsupervised domain adaptation on point cloud classification and segmentation , 2023, Frontiers of Computer Science.

[2]  Shuo Wang,et al.  VGPN: 6-DoF Grasp Pose Detection Network Based on Hough Voting , 2022, 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  K. Jia,et al.  Quasi-Balanced Self-Training on Noise-Aware Synthesis of Object Point Clouds for Closing Domain Gap , 2022, ECCV.

[4]  He Wang,et al.  Domain Adaptation on Point Clouds via Geometry-Aware Implicits , 2021, 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Bo Zhou,et al.  Domain adversarial transfer for cross-domain and task-constrained grasp pose detection , 2021, Robotics Auton. Syst..

[6]  Cewu Lu,et al.  Graspness Discovery in Clutters for Fast and Accurate Grasp Detection , 2021, 2021 IEEE/CVF International Conference on Computer Vision (ICCV).

[7]  Dong Xu,et al.  SRDAN: Scale-aware and Range-aware Domain Adaptation Network for Cross-dataset 3D Object Detection , 2021, 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Peng Wang,et al.  GPR: Grasp Pose Refinement Network for Cluttered Scenes , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Miao Li,et al.  Unknown Object Segmentation through Domain Adaptation , 2021, 2021 4th International Conference on Intelligent Autonomous Systems (ICoIAS).

[10]  Dieter Fox,et al.  Contact-GraspNet: Efficient 6-DoF Grasp Generation in Cluttered Scenes , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Dieter Fox,et al.  ACRONYM: A Large-Scale Grasp Dataset Based on Simulation , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Yuke Zhu,et al.  Fast Uncertainty Quantification for Deep Object Pose Estimation , 2020, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[13]  Ajay Kumar Tanwani,et al.  DIRL: Domain-Invariant Representation Learning for Sim-to-Real Transfer , 2020, CoRL.

[14]  Xuguang Lan,et al.  An Exploration of Domain Adaptation Applying to Grasp Detection Algorithm , 2020, 2020 Chinese Automation Congress (CAC).

[15]  Tomi Westerlund,et al.  Sim-to-Real Transfer in Deep Reinforcement Learning for Robotics: a Survey , 2020, 2020 IEEE Symposium Series on Computational Intelligence (SSCI).

[16]  Chek Sing Teo,et al.  Grasping Detection Network with Uncertainty Estimation for Confidence-Driven Semi-Supervised Domain Adaptation , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Cewu Lu,et al.  GraspNet-1Billion: A Large-Scale Benchmark for General Object Grasping , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Zhihai He,et al.  CLDA: an adversarial unsupervised domain adaptation method with classifier-level adaptation , 2020, Multimedia Tools and Applications.

[19]  Gal Chechik,et al.  Self-Supervised Learning for Domain Adaptation on Point Clouds , 2020, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[20]  Ronald P. A. Petrick,et al.  Learning Grasp Affordance Reasoning Through Semantic Relations , 2019, IEEE Robotics and Automation Letters.

[21]  Dieter Fox,et al.  6-DOF GraspNet: Variational Grasp Generation for Object Manipulation , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[22]  Eric Brachmann,et al.  BOP: Benchmark for 6D Object Pose Estimation , 2018, ECCV.

[23]  Andrew J. Davison,et al.  Sim-to-Real Reinforcement Learning for Deformable Object Manipulation , 2018, CoRL.

[24]  Mei Wang,et al.  Deep Visual Domain Adaptation: A Survey , 2018, Neurocomputing.

[25]  Yue Wang,et al.  Dynamic Graph CNN for Learning on Point Clouds , 2018, ACM Trans. Graph..

[26]  Anirban Sarkar,et al.  Grad-CAM++: Generalized Gradient-Based Visual Explanations for Deep Convolutional Networks , 2017, 2018 IEEE Winter Conference on Applications of Computer Vision (WACV).

[27]  Darwin G. Caldwell,et al.  AffordanceNet: An End-to-End Deep Learning Approach for Object Affordance Detection , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[28]  Arkanath Pathak,et al.  Learning 6-DOF Grasping Interaction via Deep Geometry-Aware 3D Representations , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[29]  Kate Saenko,et al.  Grasp Pose Detection in Point Clouds , 2017, Int. J. Robotics Res..

[30]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[31]  Leonidas J. Guibas,et al.  PointNet++: Deep Hierarchical Feature Learning on Point Sets in a Metric Space , 2017, NIPS.

[32]  Alexei A. Efros,et al.  Unpaired Image-to-Image Translation Using Cycle-Consistent Adversarial Networks , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[33]  Trevor Darrell,et al.  Adversarial Discriminative Domain Adaptation , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[34]  Jonathon Shlens,et al.  Conditional Image Synthesis with Auxiliary Classifier GANs , 2016, ICML.

[35]  Ming-Yu Liu,et al.  Coupled Generative Adversarial Networks , 2016, NIPS.

[36]  Mathieu Aubry,et al.  Dex-Net 1.0: A cloud-based network of 3D objects for robust grasp planning using a Multi-Armed Bandit model with correlated rewards , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[37]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[38]  Pierre Vandergheynst,et al.  Geodesic Convolutional Neural Networks on Riemannian Manifolds , 2015, 2015 IEEE International Conference on Computer Vision Workshop (ICCVW).

[39]  Victor S. Lempitsky,et al.  Unsupervised Domain Adaptation by Backpropagation , 2014, ICML.

[40]  Jianxiong Xiao,et al.  3D ShapeNets: A deep representation for volumetric shapes , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[41]  Cheng Wang,et al.  Point-Based Multilevel Domain Adaptation for Point Cloud Segmentation , 2022, IEEE Geoscience and Remote Sensing Letters.

[42]  Sergey Levine,et al.  Using Simulation and Domain Adaptation to Improve Efficiency of Deep Robotic Grasping , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).