Autonomous Clothes Manipulation Using a Hierarchical Vision Architecture

This paper presents a novel robot vision architecture for perceiving generic 3-D clothes configurations. Our architecture is hierarchically structured, starting from low-level curvature features to mid-level geometric shapes and topology descriptions, and finally, high-level semantic surface descriptions. We demonstrate our robot vision architecture in a customized dual-arm industrial robot with our in-house developed stereo vision system, carrying out autonomous grasping and dual-arm flattening. The experimental results show the effectiveness of the proposed dual-arm flattening using the stereo vision system compared with the single-arm flattening using the widely cited Kinect-like sensor as the baseline. In addition, the proposed grasping approach achieves satisfactory performance when grasping various kind of garments, verifying the capability of the proposed visual perception architecture to be adapted to more than one clothing manipulation tasks.

[1]  Francesc Moreno-Noguer,et al.  FINDDD: A fast 3D descriptor to characterize textiles for robot manipulation , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Václav Hlavác,et al.  Polygonal Models for Clothing , 2014, TAROS.

[3]  J. Paul Siebert,et al.  Recognising the clothing categories from free-configuration using Gaussian-Process-based interactive perception , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[4]  J. Paul Siebert,et al.  Single-shot clothing category recognition in free-configurations with application to autonomous clothes sorting , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[5]  Danfei Xu,et al.  Folding deformable objects using predictive simulation and trajectory optimization , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[6]  Tae-Kyun Kim,et al.  Autonomous active recognition and unfolding of clothes using random decision forests and probabilistic planning , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Kenneth Y. Goldberg,et al.  Learning Deep Policies for Robot Bin Picking by Simulating Robust Grasping Sequences , 2017, CoRL.

[8]  Trevor Darrell,et al.  A geometric approach to robotic laundry folding , 2012, Int. J. Robotics Res..

[9]  Ian D. Walker,et al.  Classification of clothing using interactive perception , 2011, 2011 IEEE International Conference on Robotics and Automation.

[10]  Pieter Abbeel,et al.  Cloth grasp point detection based on multiple-view geometric cues with application to robotic towel folding , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Honglak Lee,et al.  Deep learning for detecting robotic grasps , 2013, Int. J. Robotics Res..

[12]  Ian D. Walker,et al.  A new approach to clothing classification using mid-level layers , 2013, 2013 IEEE International Conference on Robotics and Automation.

[13]  Vladimír Petrík,et al.  Garment perception and its folding using a dual-arm robot , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[14]  Dieter Fox,et al.  PoseCNN: A Convolutional Neural Network for 6D Object Pose Estimation in Cluttered Scenes , 2017, Robotics: Science and Systems.

[15]  Ian D. Walker,et al.  Model for unfolding laundry using interactive perception , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[16]  Xinyu Liu,et al.  Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point Clouds and Analytic Grasp Metrics , 2017, Robotics: Science and Systems.

[17]  Xinyu Liu,et al.  Dex-Net 3.0: Computing Robust Robot Suction Grasp Targets in Point Clouds using a New Analytic Model and Deep Learning , 2017, ArXiv.

[18]  Lawson L. S. Wong,et al.  Learning Grasp Strategies with Partial Shape Information , 2008, AAAI.

[19]  Shih-Fu Chang,et al.  Regrasping and unfolding of garments using predictive thin shell modeling , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[20]  J. Paul Siebert,et al.  Parallel Stereo Vision Algorithm , 2012, MARC@RWTH.

[21]  Francesc Moreno-Noguer,et al.  Using depth and appearance features for informed robot grasping of highly wrinkled clothes , 2012, 2012 IEEE International Conference on Robotics and Automation.

[22]  Yi Li,et al.  DeepIM: Deep Iterative Matching for 6D Pose Estimation , 2018, International Journal of Computer Vision.

[23]  Roger Y. Tsai,et al.  Real time versatile robotics hand/eye calibration using 3D machine vision , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[24]  Nobuyuki Kita,et al.  A method for handling a specific part of clothing by dual arms , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[25]  Ashutosh Saxena,et al.  Learning 3-D object orientation from images , 2009, 2009 IEEE International Conference on Robotics and Automation.

[26]  Roger Y. Tsai,et al.  A new technique for fully autonomous and efficient 3D robotics hand/eye calibration , 1988, IEEE Trans. Robotics Autom..

[27]  Trevor Darrell,et al.  Parametrized shape models for clothing , 2011, 2011 IEEE International Conference on Robotics and Automation.

[28]  James F. O'Brien,et al.  Bringing clothing into desired configurations with limited perception , 2011, 2011 IEEE International Conference on Robotics and Automation.

[29]  Danfei Xu,et al.  Multi-sensor surface analysis for robotic ironing , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[30]  Pieter Abbeel,et al.  Gravity-Based Robotic Cloth Folding , 2010, WAFR.

[31]  Simon Rogers,et al.  A Precise Method for Cloth Configuration Parsing Applied to Single-Arm Flattening , 2016 .

[32]  Alexander G. Belyaev,et al.  Detection of Surface Creases in Range Data , 2005, IMA Conference on the Mathematics of Surfaces.

[33]  Li Sun,et al.  Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Sergey Levine,et al.  Learning hand-eye coordination for robotic grasping with deep learning and large-scale data collection , 2016, Int. J. Robotics Res..

[35]  Vladimír Petrík,et al.  Folding Clothes Autonomously: A Complete Pipeline , 2016, IEEE Transactions on Robotics.

[36]  Nobuyuki Kita,et al.  Clothes state recognition using 3D observed data , 2009, 2009 IEEE International Conference on Robotics and Automation.

[37]  Robin R. Murphy,et al.  Lessons learned in integrating sensing into autonomous mobile robot architectures , 1997, J. Exp. Theor. Artif. Intell..

[38]  Peter K. Allen,et al.  Recognition of deformable object category and pose , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[39]  J. Paul Siebert,et al.  A Heuristic-Based Approach for Flattening Wrinkled Clothes , 2013, TAROS.

[40]  Andrea J. van Doorn,et al.  Surface shape and curvature scales , 1992, Image Vis. Comput..

[41]  Shih-Fu Chang,et al.  Real-time pose estimation of deformable objects using a volumetric approach , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[42]  Nikos A. Aspragathos,et al.  A geometric approach to robotic unfolding of garments , 2016, Robotics Auton. Syst..

[43]  Matteo Zoppi,et al.  On the Development of a Specialized Flexible Gripper for Garment Handling , 2013 .

[44]  Jan Paul Siebert,et al.  C3D™: a Novel Vision-Based 3-D Data Acquisition System , 1995 .

[45]  Tae-Kyun Kim,et al.  Active Random Forests: An Application to Autonomous Unfolding of Clothes , 2014, ECCV.