Continuous perception for deformable objects understanding

Abstract We present a robot vision approach to deformable object classification, with direct application to autonomous service robots. Our approach is based on the assumption that continuous perception provides robots with greater visual competence for deformable objects interpretation and classification. Our approach classifies the category of clothing items by continuously perceiving the dynamic interactions of the garment’s material and shape as it is being picked up. For this, we extract continuously visual features of a RGB-D video sequence and we fuse features by means of the Locality Constrained Group Sparse Representation (LGSR) algorithm. To evaluate the performance of our approach, we created a fully annotated database featuring 150 garment videos in random configurations. Experiments demonstrate that by continuously observing an object deform, our approach achieves a classification score of 66.7%, outperforming state-of-the-art approaches by a ∼ 27 . 3 % increase.

[1]  José García Rodríguez,et al.  A Review on Deep Learning Techniques Applied to Semantic Segmentation , 2017, ArXiv.

[2]  Jitendra Malik,et al.  Shape Context: A New Descriptor for Shape Matching and Object Recognition , 2000, NIPS.

[3]  J. Paul Siebert,et al.  Single-shot clothing category recognition in free-configurations with application to autonomous clothes sorting , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[4]  Ian D. Walker,et al.  A new approach to clothing classification using mid-level layers , 2013, 2013 IEEE International Conference on Robotics and Automation.

[5]  Dong Xu,et al.  Video Event Recognition Using Kernel Methods with Multilevel Temporal Alignment , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Kimitoshi Yamazaki,et al.  A method of state recognition of dressing clothes based on dynamic state matching , 2013, Proceedings of the 2013 IEEE/SICE International Symposium on System Integration.

[7]  Demetri Terzopoulos,et al.  Snakes: Active contour models , 2004, International Journal of Computer Vision.

[8]  Gary R. Bradski,et al.  Fast 3D recognition and pose using the Viewpoint Feature Histogram , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[9]  Sebastian Hoppe Nesgaard Jensen,et al.  An Adaptive Robotic System for Doing Pick and Place Operations with Deformable Objects , 2018, Journal of Intelligent & Robotic Systems.

[10]  Carme Torras,et al.  Robot-Aided Cloth Classification Using Depth Information and CNNs , 2016, AMDO.

[11]  Ioannis Mariolis,et al.  Matching Folded Garments to Unfolded Templates Using Robust Shape Analysis Techniques , 2013, CAIP.

[12]  Francesc Moreno-Noguer,et al.  Using depth and appearance features for informed robot grasping of highly wrinkled clothes , 2012, 2012 IEEE International Conference on Robotics and Automation.

[13]  Dong Xu,et al.  Human Gait Recognition Using Patch Distribution Feature and Locality-Constrained Group Sparse Representation , 2012, IEEE Transactions on Image Processing.

[14]  Rajat Raina,et al.  Efficient sparse coding algorithms , 2006, NIPS.

[15]  J. Paul Siebert,et al.  Recognising the clothing categories from free-configuration using Gaussian-Process-based interactive perception , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[16]  Danfei Xu,et al.  Multi-sensor surface analysis for robotic ironing , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[17]  Peter K. Allen,et al.  Recognition of deformable object category and pose , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[18]  Javier Ruiz-del-Solar,et al.  Continuous Perception for Clothing Understanding in Robotic Applications , 2019 .

[19]  Ian D. Walker,et al.  Model for unfolding laundry using interactive perception , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[20]  I. Walker,et al.  Classification of Clothing Using Midlevel Layers , 2013 .

[21]  Shih-Fu Chang,et al.  Real-time pose estimation of deformable objects using a volumetric approach , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[22]  Nobuyuki Kita,et al.  Recognizing clothing states using 3D data observed from multiple directions , 2013, 2013 13th IEEE-RAS International Conference on Humanoid Robots (Humanoids).

[23]  Li Sun,et al.  Accurate garment surface analysis using an active stereo robot head with application to dual-arm flattening , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[24]  Ioannis Mariolis,et al.  Pose and category recognition of highly deformable objects using deep learning , 2015, 2015 International Conference on Advanced Robotics (ICAR).

[25]  Neil D. Lawrence,et al.  Gaussian Process Latent Variable Models for Visualisation of High Dimensional Data , 2003, NIPS.

[26]  Mohan M. Trivedi,et al.  Segmentation of a high-resolution urban scene using texture operators , 1984, Comput. Vis. Graph. Image Process..

[27]  Daniel P. Huttenlocher,et al.  Efficient Graph-Based Image Segmentation , 2004, International Journal of Computer Vision.

[28]  Vladimír Petrík,et al.  Robotic Garment Folding: Precision Improvement and Workspace Enlargement , 2015, TAROS.

[29]  Nobuyuki Kita,et al.  A model-driven method of estimating the state of clothes for manipulating it , 2002, Sixth IEEE Workshop on Applications of Computer Vision, 2002. (WACV 2002). Proceedings..

[30]  Francesc Moreno-Noguer,et al.  Determining Where to Grasp Cloth Using Depth Information , 2011, CCIA.

[31]  Tae-Kyun Kim,et al.  Active Random Forests: An Application to Autonomous Unfolding of Clothes , 2014, ECCV.

[32]  Carme Torras,et al.  Active garment recognition and target grasping point detection using deep learning , 2018, Pattern Recognit..

[33]  Francesc Moreno-Noguer,et al.  Learning RGB-D descriptors of garment parts for informed robot grasping , 2014, Eng. Appl. Artif. Intell..

[34]  J. Paul Siebert,et al.  A Heuristic-Based Approach for Flattening Wrinkled Clothes , 2013, TAROS.