Enhancing Perception with Tactile Object Recognition in Adaptive Grippers for Human–Robot Interaction

The use of tactile perception can help first response robotic teams in disaster scenarios, where visibility conditions are often reduced due to the presence of dust, mud, or smoke, distinguishing human limbs from other objects with similar shapes. Here, the integration of the tactile sensor in adaptive grippers is evaluated, measuring the performance of an object recognition task based on deep convolutional neural networks (DCNNs) using a flexible sensor mounted in adaptive grippers. A total of 15 classes with 50 tactile images each were trained, including human body parts and common environment objects, in semi-rigid and flexible adaptive grippers based on the fin ray effect. The classifier was compared against the rigid configuration and a support vector machine classifier (SVM). Finally, a two-level output network has been proposed to provide both object-type recognition and human/non-human classification. Sensors in adaptive grippers have a higher number of non-null tactels (up to 37% more), with a lower mean of pressure values (up to 72% less) than when using a rigid sensor, with a softer grip, which is needed in physical human–robot interaction (pHRI). A semi-rigid implementation with 95.13% object recognition rate was chosen, even though the human/non-human classification had better results (98.78%) with a rigid sensor.

[1]  Mehmet Remzi Dogar,et al.  Haptic identification of objects using a modular soft robotic gripper , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[2]  Berthold Bäuml,et al.  Robust material classification with a tactile skin using deep learning , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[3]  Ravinder Dahiya,et al.  Robotic Tactile Perception of Object Properties: A Review , 2017, ArXiv.

[4]  Amit Konar,et al.  Object-shape recognition from tactile images using a feed-forward neural network , 2012, The 2012 International Joint Conference on Neural Networks (IJCNN).

[5]  Yan-Bin Jia,et al.  Grasping deformable planar objects: Squeeze, stick/slip analysis, and energy-based optimalities , 2014, Int. J. Robotics Res..

[6]  Li Fei-Fei,et al.  ImageNet: A large-scale hierarchical image database , 2009, CVPR.

[7]  Stefan Wermter,et al.  Haptic material classification with a multi-channel neural network , 2017, 2017 International Joint Conference on Neural Networks (IJCNN).

[8]  Robert J. Wood,et al.  Soft Robotic Grippers for Biological Sampling on Deep Reefs , 2016, Soft robotics.

[9]  Giulio Sandini,et al.  Tactile Sensing—From Humans to Humanoids , 2010, IEEE Transactions on Robotics.

[10]  Nawid Jamali,et al.  Majority Voting: Material Classification by Tactile Sensing Using Surface Texture , 2011, IEEE Transactions on Robotics.

[11]  Kaspar Althoefer,et al.  A computationally fast algorithm for local contact shape and pose classification using a tactile array sensor , 2012, 2012 IEEE International Conference on Robotics and Automation.

[12]  Tony J. Dodd,et al.  Feeling the Shape: Active Exploration Behaviors for Object Recognition With a Robotic Hand , 2018, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Robert J. Wood,et al.  Soft robotic glove for combined assistance and at-home rehabilitation , 2015, Robotics Auton. Syst..

[14]  Lionel Birglen,et al.  A compliant self-adaptive gripper with proprioceptive haptic feedback , 2014, Auton. Robots.

[15]  Kaspar Althoefer,et al.  Tactile Object Recognition with Semi-Supervised Learning , 2015, ICIRA.

[16]  Heinz Wörn,et al.  Haptic object recognition for multi-fingered robot hands , 2012, 2012 IEEE Haptics Symposium (HAPTICS).

[17]  Ah Chung Tsoi,et al.  Face recognition: a convolutional neural-network approach , 1997, IEEE Trans. Neural Networks.

[18]  Nigel H. Lovell,et al.  A review of tactile sensing technologies with applications in biomedical engineering , 2012 .

[19]  Juan M. Gandarias,et al.  Human and object recognition with a high-resolution tactile sensor , 2017, 2017 IEEE SENSORS.

[20]  Rostislav Khlebnikov,et al.  Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) , 2016 .

[21]  Di Guo,et al.  Object Recognition Using Tactile Measurements: Kernel Sparse Coding Methods , 2016, IEEE Transactions on Instrumentation and Measurement.

[22]  Catherine E. Lewis,et al.  Tactile Feedback Induces Reduced Grasping Force in Robot-Assisted Surgery , 2009, IEEE Transactions on Haptics.

[23]  Simona Crea,et al.  A Flexible Sensor Technology for the Distributed Measurement of Interaction Pressure , 2013, Sensors.

[24]  Alin Albu-Schäffer,et al.  Requirements for Safe Robots: Measurements, Analysis and New Insights , 2009, Int. J. Robotics Res..

[25]  Guanghua Xu,et al.  A tactile sensing and feedback system for tumor localization , 2016, 2016 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI).

[26]  Andrea Lockerd Thomaz,et al.  Touched by a robot: An investigation of subjective responses to robot-initiated touch , 2011, 2011 6th ACM/IEEE International Conference on Human-Robot Interaction (HRI).

[27]  S B Heymsfield,et al.  Anthropometric measurement of muscle mass: revised equations for calculating bone-free arm muscle area. , 1982, The American journal of clinical nutrition.

[28]  Alessandro Albini,et al.  Human hand recognition from robotic skin measurements in human-robot physical interactions , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[29]  Wolfram Burgard,et al.  Tactile Sensing for Mobile Manipulation , 2011, IEEE Transactions on Robotics.

[30]  William S. Harwin,et al.  Evaluation of Sensor Configurations for Robotic Surgical Instruments , 2015, Sensors.

[31]  Edward H. Adelson,et al.  GelSight: High-Resolution Robot Tactile Sensors for Estimating Geometry and Force , 2017, Sensors.

[32]  Danica Kragic,et al.  Learning of grasp adaptation through experience and tactile sensing , 2014, 2014 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[33]  Yao Sun,et al.  A novel pneumatic soft sensor for measuring contact force and curvature of a soft gripper , 2017 .

[34]  K. Cureton,et al.  Sex difference in muscular strength in equally-trained men and women. , 1987, Ergonomics.

[35]  William C. Messner,et al.  Fin Ray® Effect Inspired Soft Robotic Gripper: From the RoboSoft Grand Challenge toward Optimization , 2016, Front. Robot. AI.

[36]  Pietro Falco,et al.  Cross-modal visuo-tactile object recognition using robotic active exploration , 2017, 2017 IEEE International Conference on Robotics and Automation (ICRA).

[37]  A. G. Pipe,et al.  A variable compliance, soft gripper , 2014, Auton. Robots.

[38]  Shigeki Sugano,et al.  Tactile object recognition using deep learning and dropout , 2014, 2014 IEEE-RAS International Conference on Humanoid Robots.

[39]  Fernando Torres Medina,et al.  3D Visual Data-Driven Spatiotemporal Deformations for Non-Rigid Object Grasping Using Robot Hands , 2016, Sensors.

[40]  Nicola Vitiello,et al.  Sensing Pressure Distribution on a Lower-Limb Exoskeleton Physical Human-Machine Interface , 2010, Sensors.

[41]  Heinz Wörn,et al.  Capacitive Tactile Proximity Sensing: From Signal Processing to Applications in Manipulation and Safe Human-Robot Interaction , 2015 .

[42]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[43]  Mitsuru Higashimori,et al.  Convolutional Neural Network based Estimation of Gel-like Food Texture by a Robotic Sensing System , 2017, Robotics.

[44]  Yang Gao,et al.  Deep learning for tactile understanding from visual and haptic data , 2015, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[45]  Kaspar Althoefer,et al.  Novel Tactile-SIFT Descriptor for Object Shape Recognition , 2015, IEEE Sensors Journal.

[46]  Loredana Zollo,et al.  Slippage Detection with Piezoresistive Tactile Sensors , 2017, Sensors.

[47]  Alfonso J. García-Cerezo,et al.  Tactile Sensing and Machine Learning for Human and Object Recognition in Disaster Scenarios , 2017, ROBOT.

[48]  Luis Moreno,et al.  Tactile-Based In-Hand Object Pose Estimation , 2017, ROBOT.

[49]  Kaspar Althoefer,et al.  Iterative Closest Labeled Point for tactile object shape recognition , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[50]  Francesca Cordella,et al.  Evaluation of Pressure Capacitive Sensors for Application in Grasping and Manipulation Analysis , 2017, Sensors.

[51]  D. Rus,et al.  Design, fabrication and control of soft robots , 2015, Nature.

[52]  Jorge Pomares,et al.  Control Framework for Dexterous Manipulation Using Dynamic Visual Servoing and Tactile Sensors' Feedback , 2014, Sensors.

[53]  Ciro Natale,et al.  Design and evaluation of tactile sensors for the estimation of grasped wire shape , 2017, 2017 IEEE International Conference on Advanced Intelligent Mechatronics (AIM).

[54]  Eric Kauderer-Abrams,et al.  Quantifying Translation-Invariance in Convolutional Neural Networks , 2017, ArXiv.

[55]  Lu Fang,et al.  Deep Learning for Surface Material Classification Using Haptic and Visual Information , 2015, IEEE Transactions on Multimedia.

[56]  Paul Lukowicz,et al.  Textile Pressure Mapping Sensor for Emotional Touch Detection in Human-Robot Interaction , 2017, Sensors.

[57]  Allison M. Okamura,et al.  Methods to Segment Hard Inclusions in Soft Tissue During Autonomous Robotic Palpation , 2015, IEEE Transactions on Robotics.

[58]  Danica Kragic,et al.  ST-HMP: Unsupervised Spatio-Temporal feature learning for tactile data , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).