Textile Taxonomy and Classification Using Pulling and Twisting

Identification of textile properties is an important milestone toward advanced robotic manipulation tasks that consider interaction with clothing items such as assisted dressing, laundry folding, automated sewing, textile recycling and reusing. Despite the abundance of work considering this class of deformable objects, many open problems remain. These relate to the choice and modelling of the sensory feedback as well as the control and planning of the interaction and manipulation strategies. Most importantly, there is no structured approach for studying and assessing different approaches that may bridge the gap between the robotics community and textile production industry. To this end, we outline a textile taxonomy considering fiber types and production methods, commonly used in textile industry. We devise datasets according to the taxonomy, and study how robotic actions, such as pulling and twisting of the textile samples, can be used for the classification. We also provide important insights from the perspective of visualization and interpretability of the gathered data.

[1]  Ravinder Dahiya,et al.  Robotic Tactile Perception of Object Properties: A Review , 2017, ArXiv.

[2]  Frédo Durand,et al.  Visual vibrometry: Estimating material properties from small motions in video , 2015, CVPR.

[3]  C. Karen Liu,et al.  Data-driven haptic perception for robot-assisted dressing , 2016, 2016 25th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN).

[4]  Frédo Durand,et al.  Visual vibrometry: Estimating material properties from small motions in video , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[5]  Abhishek Das,et al.  Grad-CAM: Visual Explanations from Deep Networks via Gradient-Based Localization , 2016, 2017 IEEE International Conference on Computer Vision (ICCV).

[6]  Xiaogang Wang,et al.  DeepFashion: Powering Robust Clothes Recognition and Retrieval with Rich Annotations , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Stefan Wermter,et al.  Haptic material classification with a multi-channel neural network , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[8]  Volker Scholz,et al.  Cloth Motion from Optical Flow , 2004, VMV.

[9]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[10]  Danfei Xu,et al.  Folding deformable objects using predictive simulation and trajectory optimization , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[11]  Christian Theobalt,et al.  Multi-Garment Net: Learning to Dress 3D People From Images , 2019, 2019 IEEE/CVF International Conference on Computer Vision (ICCV).

[12]  Chaitanya Patel,et al.  TailorNet: Predicting Clothing in 3D as a Function of Human Pose, Shape and Garment Style , 2020, 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Tsukasa Ogasawara,et al.  Textile identification using fingertip motion and 3D force sensors in an open-source gripper , 2017, 2017 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[14]  S. Grishanov Structure and properties of textile materials , 2011 .

[15]  Vladimír Petrík,et al.  Feedback-based Fabric Strip Folding , 2019, 2019 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[16]  G. Cannata,et al.  A tactile-based fabric learning and classification architecture , 2016, 2016 IEEE International Conference on Information and Automation for Sustainability (ICIAfS).

[17]  Danica Kragic,et al.  Fashion Landmark Detection and Category Classification for Robotics , 2020, 2020 IEEE International Conference on Autonomous Robot Systems and Competitions (ICARSC).

[18]  Corinna Cortes,et al.  Support-Vector Networks , 1995, Machine Learning.

[19]  Anthony G. Cohn,et al.  ViTac: Feature Sharing Between Vision and Tactile Sensing for Cloth Texture Recognition , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[20]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[21]  R. Klatzky,et al.  Haptic classification of common objects: Knowledge-driven exploration , 1990, Cognitive Psychology.

[22]  Danica Kragic,et al.  Latent Space Roadmap for Visual Action Planning of Deformable and Rigid Object Manipulation , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[23]  Danica Kragic,et al.  Benchmarking Bimanual Cloth Manipulation , 2020, IEEE Robotics and Automation Letters.

[24]  Joseph M. Romano,et al.  Methods for robotic tool-mediated haptic surface recognition , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[25]  Danica Kragic,et al.  Interpretability in Contact-Rich Manipulation via Kinodynamic Images , 2021, 2021 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Yang Gao,et al.  Deep learning for tactile understanding from visual and haptic data , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[27]  Qiang Chen,et al.  Cross-Domain Image Retrieval with a Dual Attribute-Aware Ranking Network , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[28]  Eckehard Steinbach,et al.  Haptic Material Analysis and Classification Inspired by Human Exploratory Procedures , 2019, IEEE Transactions on Haptics.

[29]  Javier Silvestre-Blanes,et al.  Garment smoothness appearance evaluation through computer vision , 2012 .

[30]  Ruimao Zhang,et al.  DeepFashion2: A Versatile Benchmark for Detection, Pose Estimation, Segmentation and Re-Identification of Clothing Images , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  T. Martin McGinnity,et al.  Material recognition using tactile sensing , 2018, Expert Syst. Appl..

[32]  Jivko Sinapov,et al.  Vibrotactile Recognition and Categorization of Surfaces by a Humanoid Robot , 2011, IEEE Transactions on Robotics.

[33]  Zoe Doulgeri,et al.  A robotic system for handling textile and non rigid flat materials , 1995 .

[34]  Gerald E. Loeb,et al.  Bayesian Exploration for Intelligent Identification of Textures , 2012, Front. Neurorobot..

[35]  Edward H. Adelson,et al.  Active Clothing Material Perception Using Tactile Sensing and Deep Learning , 2017, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[36]  Belhassen-Chedli Bouzgarrou,et al.  Robotic manipulation and sensing of deformable objects in domestic and industrial applications: a survey , 2018, Int. J. Robotics Res..

[37]  Zackory M. Erickson,et al.  Multimodal Material Classification for Robots using Spectroscopy and High Resolution Texture Imaging , 2020, 2020 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[38]  Carme Torras,et al.  Perception of cloth in assistive robotic manipulation tasks , 2020, Natural Computing.

[39]  Greg Chance,et al.  A Quantitative Analysis of Dressing Dynamics for Robotic Dressing Assistance , 2017, Front. Robot. AI.

[40]  J. Paul Siebert,et al.  Recognising the clothing categories from free-configuration using Gaussian-Process-based interactive perception , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[41]  Makoto Hasegawa,et al.  A study on garment wrinkle detection through a 3D camera scanning with normal map and Hough transform , 2019, Other Conferences.