3D Vision for Object Grasp and Obstacle Avoidance of a Collaborative Robot

This paper presents a design and experimental study of 3D robotic vision for bin picking and obstacle avoidance. Through the 3D vision algorithm, the robotic picking system is able to analyze the imagery of cluttered objects, classify the objects and estimate the pose of identified objects for grasping. In order to facilitate the robot to work with a human nearby, obstacle avoidance during task execution is developed based on 3D vision. In this design, a RealSense SR300 RGB-D camera is utilized to acquire RGB images and depth images of clustered workpieces. A deep neural network (DNN) approach to object recognition is designed and combined with point cloud segmentation to enhance 3D object-pose estimation for grasping The robot avoids obstacles to assure safe operation during execution of the bin picking task. Practical experiments using a Techman TM5 6-DOF robot arm show that the proposed method effectively accomplishes obstacle avoidance in pick-and-place operations.

[1]  Li Fei-Fei,et al.  DenseCap: Fully Convolutional Localization Networks for Dense Captioning , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Ali Farhadi,et al.  YOLO9000: Better, Faster, Stronger , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[3]  Kai-Tai Song,et al.  Vision-based adaptive grasping of a humanoid robot arm , 2012, 2012 IEEE International Conference on Automation and Logistics.

[4]  Ali Farhadi,et al.  You Only Look Once: Unified, Real-Time Object Detection , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Antonio Bicchi,et al.  Integration of active and passive compliance control for safe human-robot coexistence , 2009, 2009 IEEE International Conference on Robotics and Automation.

[6]  Nassir Navab,et al.  Model globally, match locally: Efficient and robust 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[7]  Kai-Tai Song,et al.  CAD-based Pose Estimation Design for Random Bin Picking using a RGB-D Camera , 2017, J. Intell. Robotic Syst..

[8]  Han-Pang Huang,et al.  Motion planning of a dual-arm mobile robot in the configuration-time space , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Gary M. Bone,et al.  Real-time 3D Collision Avoidance Method for Safe Human and Robot Coexistence , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[10]  Charles W. Warren,et al.  Global path planning using artificial potential fields , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[11]  Ren C. Luo,et al.  Model-based 3D object recognition and fetching by a 7-DoF robot with online obstacle avoidance for factory automation , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[12]  Torsten Kröger,et al.  Opening the door to new sensor-based robot applications—The Reflexxes Motion Libraries , 2011, 2011 IEEE International Conference on Robotics and Automation.

[13]  Ming-Yu Liu,et al.  Voting-based pose estimation for robotic assembly using a 3D sensor , 2012, 2012 IEEE International Conference on Robotics and Automation.

[14]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[15]  Chin-Chia Wu,et al.  Error analysis and experiments of 3D reconstruction using a RGB-D sensor , 2014, 2014 IEEE International Conference on Automation Science and Engineering (CASE).

[16]  S. LaValle.  Rapidly-exploring random trees : a new tool for path planning , 1998 .

[17]  Michael Beetz,et al.  Real-time perception-guided motion planning for a personal robot , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[18]  Seth Hutchinson,et al.  A Framework for Real-time Path Planning in Changing Environments , 2002, Int. J. Robotics Res..

[19]  Oussama Khatib,et al.  A Depth Space Approach for Evaluating Distance to Objects , 2015, J. Intell. Robotic Syst..

[20]  Sven Behnke,et al.  NimbRo picking: Versatile part handling for warehouse automation , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[21]  Friedrich M. Wahl,et al.  Online Trajectory Generation: Basic Concepts for Instantaneous Reactions to Unforeseen Events , 2010, IEEE Transactions on Robotics.