Haptic Zero-Shot Learning: Recognition of objects never touched before

Abstract Object recognition is essential to enable robots to interact with their environment. Robots should be capable, on one hand of recognizing previously experienced objects, and on the other, of using the experienced objects for learning novel objects, i.e. objects for which training data are not available. Recognition of such novel objects can be achieved with Zero-Shot Learning (ZSL). In this work, we show the potential of ZSL for haptic recognition. First, we design a zero-shot haptic recognition algorithm and, using the extensive PHAC-2 database (Chu et al., 2015) as well as our own, we adapt, analyze and optimize the ZSL for the challenges and constraints characteristic of haptic recognition. Finally, we apply the optimized algorithm for haptic recognition of daily-life objects using an anthropomorphic robot hand. Our algorithm enables the robot to recognize eight of the ten novel objects handed to it.

[1]  Aaron M. Dollar,et al.  Benchmarking grasping and manipulation: Properties of the Objects of Daily Living , 2010, 2010 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[2]  Sanja Fidler,et al.  Predicting Deep Zero-Shot Convolutional Neural Networks Using Textual Descriptions , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[3]  Heinz Wörn,et al.  Haptic object recognition for multi-fingered robot hands , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[4]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[5]  Shigeki Sugano,et al.  Tactile object recognition using deep learning and dropout , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[6]  Helge J. Ritter,et al.  A Probabilistic Approach to Tactile Shape Reconstruction , 2011, IEEE Transactions on Robotics.

[7]  Gordon Cheng,et al.  Re-using prior tactile experience by robotic hands to discriminate in-hand objects via texture properties , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[8]  Xiaodong Yu,et al.  Attribute-Based Transfer Learning for Object Categorization with Zero/One Training Example , 2010, ECCV.

[9]  M. Tarr,et al.  Visual Object Recognition , 1996, ISTCS.

[10]  Yang Gao,et al.  Deep learning for tactile understanding from visual and haptic data , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[11]  Hong Liu,et al.  Haptic Perception of Unknown Object by Robot Hand: Exploration Strategy and Recognition Approach , 2016, Int. J. Humanoid Robotics.

[12]  Kristen Grauman,et al.  Relative attributes , 2011, 2011 International Conference on Computer Vision.

[13]  Geoff Holmes,et al.  Classifier chains for multi-label classification , 2009, Machine Learning.

[14]  Aram Kawewong,et al.  Online incremental attribute-based zero-shot learning , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[15]  Gordon Cheng,et al.  Novel Tactile Descriptors and a Tactile Transfer Learning Technique for Active In-Hand Object Recognition via Texture Properties , 2016 .

[16]  Di Guo,et al.  Object Recognition Using Tactile Measurements: Kernel Sparse Coding Methods , 2016, IEEE Transactions on Instrumentation and Measurement.

[17]  Bernt Schiele,et al.  Zero-Shot Learning — The Good, the Bad and the Ugly , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[18]  Gordon Cheng,et al.  In-hand object recognition via texture properties with robotic hands, artificial skin, and novel tactile descriptors , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[19]  Gabriela Csurka,et al.  Distance-Based Image Classification: Generalizing to New Classes at Near-Zero Cost , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Babak Saleh,et al.  Write a Classifier: Zero-Shot Learning Using Purely Textual Descriptions , 2013, 2013 IEEE International Conference on Computer Vision.

[21]  Christoph H. Lampert,et al.  Attribute-Based Classification for Zero-Shot Visual Object Categorization , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Aaron M. Dollar,et al.  Single-Grasp Object Classification and Feature Extraction with Simple Robot Hands and Tactile Sensors , 2016, IEEE Transactions on Haptics.

[23]  Yiannis Demiris,et al.  Incrementally Learning Objects by Touch: Online Discriminative and Generative Models for Tactile-Based Recognition , 2014, IEEE Transactions on Haptics.

[24]  Kristen Grauman,et al.  Zero-shot recognition with unreliable attributes , 2014, NIPS.

[25]  Kenneth S. Roberts,et al.  Haptic object recognition using a multi-fingered dextrous hand , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[26]  Giorgio Metta,et al.  Controlled tactile exploration and haptic object recognition , 2017, 2017 18th International Conference on Advanced Robotics (ICAR).

[27]  Siddhartha S. Srinivasa,et al.  Object Modeling and Recognition from Sparse, Noisy Data via Voxel Depth Carving , 2014, ISER.

[28]  Bernt Schiele,et al.  What helps where – and why? Semantic relatedness for knowledge transfer , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[29]  Christian Balkenius,et al.  Haptic Perception with Self-Organizing ANNs and an Anthropomorphic Robot Hand , 2010, J. Robotics.

[30]  Danica Kragic,et al.  ST-HMP: Unsupervised Spatio-Temporal feature learning for tactile data , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[31]  Philip H. S. Torr,et al.  An embarrassingly simple approach to zero-shot learning , 2015, ICML.

[32]  Shih-Fu Chang,et al.  Designing Category-Level Attributes for Discriminative Visual Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  R. Klatzky,et al.  Hand movements: A window into haptic object recognition , 1987, Cognitive Psychology.

[34]  Takamitsu Matsubara,et al.  Object manifold learning with action features for active tactile object recognition , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Aaron M. Dollar,et al.  Unplanned, model-free, single grasp object classification with underactuated hands and force sensors , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[36]  Charles C. Kemp,et al.  A list of household objects for robotic retrieval prioritized by people with ALS , 2008, 2009 IEEE International Conference on Rehabilitation Robotics.

[37]  Tomonori Yamamoto,et al.  Use of tactile feedback to control exploratory movements to characterize object compliance , 2012, Front. Neurorobot..

[38]  Qiang Yu,et al.  Random Forest Classifier for Zero-Shot Learning Based on Relative Attribute , 2018, IEEE Transactions on Neural Networks and Learning Systems.

[39]  Trevor Darrell,et al.  Using robotic exploratory procedures to learn the meaning of haptic adjectives , 2013, 2013 IEEE International Conference on Robotics and Automation.

[40]  Christian Balkenius,et al.  Recognizing texture and hardness by touch , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[41]  Frank Kirchner,et al.  Haptic Object Recognition in Underwater and Deep-sea Environments , 2015, J. Field Robotics.

[42]  Wolfram Burgard,et al.  Object identification with tactile sensors using bag-of-features , 2009, 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[43]  Fei-Fei Li,et al.  Attribute Learning in Large-Scale Datasets , 2010, ECCV Workshops.

[44]  Fuchun Sun,et al.  Tactile sequence classification using joint kernel sparse coding , 2015, 2015 International Joint Conference on Neural Networks (IJCNN).

[45]  Danfei Xu,et al.  Tactile identification of objects using Bayesian exploration , 2013, 2013 IEEE International Conference on Robotics and Automation.

[46]  Daniel N. Osherson,et al.  Joshua Stern, Ormond Wilkie, Michael Stob, Edward E. Smith: Default Probability , 1991, Cogn. Sci..

[47]  Ali Farhadi,et al.  Describing objects by their attributes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[48]  Heinz Wörn,et al.  Haptic object recognition using passive joints and haptic key features , 2010, 2010 IEEE International Conference on Robotics and Automation.

[49]  Pejman Iravani,et al.  Bayesian tactile object recognition: Learning and recognising objects using a new inexpensive tactile sensor , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[50]  Heinz Wörn,et al.  Haptic object recognition using statistical point cloud features , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[51]  Yuanfei Zhang,et al.  Object shape recognition approach for sparse point clouds from tactile exploration , 2013, 2013 IEEE International Conference on Robotics and Biomimetics (ROBIO).

[52]  Trevor Darrell,et al.  Robotic learning of haptic adjectives through physical interaction , 2015, Robotics Auton. Syst..

[53]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[54]  Francesco Zanichelli,et al.  Haptic object recognition with a dextrous hand based on volumetric shape representations , 1994, Proceedings of 1994 IEEE International Conference on MFI '94. Multisensor Fusion and Integration for Intelligent Systems.

[55]  Tamim Asfour,et al.  Haptic exploration for 3D shape reconstruction using five-finger hands , 2007, 2007 7th IEEE-RAS International Conference on Humanoid Robots.

[56]  Nathan F. Lepora,et al.  Active haptic shape recognition by intrinsic motivation with a robot hand , 2015, 2015 IEEE World Haptics Conference (WHC).

[57]  Jan Peters,et al.  Evaluation of tactile feature extraction for interactive object recognition , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[58]  Bernt Schiele,et al.  Evaluating knowledge transfer and zero-shot learning in a large-scale setting , 2011, CVPR 2011.

[59]  Christoph H. Lampert,et al.  Learning to detect unseen object classes by between-class attribute transfer , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[60]  Cees Snoek,et al.  COSTA: Co-Occurrence Statistics for Zero-Shot Classification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  Daniel,et al.  Default Probability , 2004 .

[62]  Geoffrey E. Hinton,et al.  Zero-shot Learning with Semantic Output Codes , 2009, NIPS.