Fast Object Learning and Dual-arm Coordination for Cluttered Stowing, Picking, and Packing

Robotic picking from cluttered bins is a demanding task, for which Amazon Robotics holds challenges. The 2017 Amazon Robotics Challenge (ARC) required stowing items into a storage system, picking specific items, and packing them into boxes. In this paper, we describe the entry of team NimbRo Picking. Our deep object perception pipeline can be quickly and efficiently adapted to new items using a custom turntable capture system and transfer learning. It produces high-quality item segments, on which grasp poses are found. A planning component coordinates manipulation actions between two robot arms, minimizing execution time. The system has been demonstrated successfully at ARC, where our team reached second places in both the picking task and the final stow-and-pick task. We also evaluate individual components.

[1]  Richard M. Karp,et al.  Reducibility Among Combinatorial Problems , 1972, 50 Years of Integer Programming.

[2]  Seth Hutchinson,et al.  Coordinating the motions of multiple robots with specified trajectories , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[3]  D. García-Castellanos,et al.  Poles of inaccessibility: A calculation algorithm for the remotest places on earth , 2007 .

[4]  Sören Kammel,et al.  Bimanual robotic cloth manipulation for laundry folding , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[5]  Akira Nakamura,et al.  Pick and place planning for dual-arm manipulators , 2012, 2012 IEEE International Conference on Robotics and Automation.

[6]  Danica Kragic,et al.  Dual arm manipulation - A survey , 2012, Robotics Auton. Syst..

[7]  Joel W. Burdick,et al.  Dual arm estimation for coordinated bimanual manipulation , 2013, 2013 IEEE International Conference on Robotics and Automation.

[8]  Horst Bischof,et al.  Image Guided Depth Upsampling Using Anisotropic Total Generalized Variation , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Pieter Abbeel,et al.  BigBIRD: A large-scale 3D database of object instances , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[10]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[11]  Jian Sun,et al.  Identity Mappings in Deep Residual Networks , 2016, ECCV.

[12]  Martijn Wisse,et al.  Team Delft's Robot Winner of the Amazon Picking Challenge 2016 , 2016, RoboCup.

[13]  Sven Behnke,et al.  NimbRo Rescue: Solving Disaster‐response Tasks with the Mobile Manipulation Robot Momaro , 2017, J. Field Robotics.

[14]  Ian D. Reid,et al.  RefineNet: Multi-path Refinement Networks for High-Resolution Semantic Segmentation , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[15]  Sven Behnke,et al.  NimbRo picking: Versatile part handling for warehouse automation , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Zhuowen Tu,et al.  Aggregated Residual Transformations for Deep Neural Networks , 2016, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[17]  Sven Behnke,et al.  RGB-D object detection and semantic segmentation for autonomous manipulation in clutter , 2018, Int. J. Robotics Res..

[18]  Sven Behnke,et al.  NimbRo Rescue: Solving Disaster-Response Tasks through Mobile Manipulation Robot Momaro , 2018, ArXiv.

[19]  Iasonas Kokkinos,et al.  DeepLab: Semantic Image Segmentation with Deep Convolutional Nets, Atrous Convolution, and Fully Connected CRFs , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Masaki Saito,et al.  End-to-End Learning of Object Grasp Poses in the Amazon Robotics Challenge , 2020 .