Towards Robotic Picking of Targets with Background Distractors using Deep Reinforcement Learning

Robotic manipulation using vision-based learning algorithms has benefits many tasks where the robot needs to interact with a desired target. However, when a desired target is placed with other objects, there remains several technical challenges. Firstly, the target might be occluded by other objects in terms of the camera field of view. Secondly, prioritizing the selection of the objects changes the polices and rewards of the learning algorithm which might deteriorate the successful rates. Thirdly, the occlusion by other objects increases the difficulty of manipulation as it requires complex synergies between pushing and grasping. Therefore, we propose a value-based deep reinforcement learning with Mask R-CNN to address the issues of robotic manipulation for multiple object. In the proposed three rewards are proposed, namely: 1) success of grasping the desired target; 2) removal of the distractors; and 3) effective pushes. This method aims to enable the manipulation to grasp a desired target among distractors. Simulation has been conducted to demonstrate the effectiveness of the proposed method. The results show that the robot is able to effectively pick the desired targets out despite the physical and optical occlusion by other objects. All tasks are completed with 100% successful rate within 30 episodes.

[1]  Sergey Levine,et al.  QT-Opt: Scalable Deep Reinforcement Learning for Vision-Based Robotic Manipulation , 2018, CoRL.

[2]  Surya P. N. Singh,et al.  V-REP: A versatile and scalable robot simulation framework , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[3]  Peter K. Allen,et al.  Graspit! A versatile simulator for robotic grasping , 2004, IEEE Robotics & Automation Magazine.

[4]  Marcin Andrychowicz,et al.  Asymmetric Actor Critic for Image-Based Robot Learning , 2017, Robotics: Science and Systems.

[5]  Anca D. Dragan,et al.  Robot grasping in clutter: Using a hierarchy of supervisors for learning from demonstrations , 2016, 2016 IEEE International Conference on Automation Science and Engineering (CASE).

[6]  Peter K. Allen,et al.  An SVM learning approach to robotic grasping , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[7]  Kilian Q. Weinberger,et al.  Densely Connected Convolutional Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]  Sergey Levine,et al.  Deep Reinforcement Learning for Vision-Based Robotic Grasping: A Simulated Comparative Evaluation of Off-Policy Methods , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[9]  Douglas Chai,et al.  Review of Deep Learning Methods in Robotic Grasp Detection , 2018, Multimodal Technol. Interact..

[10]  Chyi-Yeu Lin,et al.  6D pose estimation using an improved method based on point pair features , 2018, 2018 4th International Conference on Control, Automation and Robotics (ICCAR).

[11]  Fei-Fei Li,et al.  ImageNet: A large-scale hierarchical image database , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[12]  Arpit Agarwal,et al.  Reinforcement Learning of Active Vision for Manipulating Objects under Occlusions , 2018, CoRL.

[13]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[14]  Stefan Schaal,et al.  Reinforcement learning of motor skills in high dimensions: A path integral approach , 2010, 2010 IEEE International Conference on Robotics and Automation.

[15]  Alberto Rodriguez,et al.  Learning Synergies Between Pushing and Grasping with Self-Supervised Deep Reinforcement Learning , 2018, 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  Abdeslam Boularias,et al.  Learning to Manipulate Unknown Objects in Clutter by Reinforcement , 2015, AAAI.

[17]  James M. Conrad,et al.  Design and implementation of a semi-autonomous waste segregation robot , 2016, SoutheastCon 2016.

[18]  Pietro Perona,et al.  Microsoft COCO: Common Objects in Context , 2014, ECCV.

[19]  Sergey Levine,et al.  Visual Foresight: Model-Based Deep Reinforcement Learning for Vision-Based Robotic Control , 2018, ArXiv.

[20]  Wang Kai,et al.  Vision-based Robotic Grasping from Object Localization, Pose Estimation, Grasp Detection to Motion Planning: A Review. , 2019 .

[21]  Peter Stone,et al.  Policy gradient reinforcement learning for fast quadrupedal locomotion , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[22]  Henrik I. Christensen,et al.  Automatic grasp planning using shape primitives , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[23]  Stefan Schaal,et al.  Learning force control policies for compliant manipulation , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[24]  Vijay Kumar,et al.  Robotic grasping and contact: a review , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[25]  Tucker Hermans,et al.  Planning Multi-Fingered Grasps as Probabilistic Inference in a Learned Deep Network , 2018, ISRR.

[26]  Shiguo Lian,et al.  Vision-based Robotic Grasping from Object Localization, Pose Estimation, Grasp Detection to Motion Planning: A Review , 2019, ArXiv.