Haptic Glove and Platform with Gestural Control For Neuromorphic Tactile Sensory Feedback In Medical Telepresence †

Advancements in the study of the human sense of touch are fueling the field of haptics. This is paving the way for augmenting sensory perception during object palpation in tele-surgery and reproducing the sensed information through tactile feedback. Here, we present a novel tele-palpation apparatus that enables the user to detect nodules with various distinct stiffness buried in an ad-hoc polymeric phantom. The contact force measured by the platform was encoded using a neuromorphic model and reproduced on the index fingertip of a remote user through a haptic glove embedding a piezoelectric disk. We assessed the effectiveness of this feedback in allowing nodule identification under two experimental conditions of real-time telepresence: In Line of Sight (ILS), where the platform was placed in the visible range of a user; and the more demanding Not In Line of Sight (NILS), with the platform and the user being 50 km apart. We found that the entailed percentage of identification was higher for stiffer inclusions with respect to the softer ones (average of 74% within the duration of the task), in both telepresence conditions evaluated. These promising results call for further exploration of tactile augmentation technology for telepresence in medical interventions.

[1]  Edwige Pissaloux,et al.  Tactile Displays in Human-Machine Interaction: Four Case Studies , 2008, Int. J. Virtual Real..

[2]  Arianna Menciassi,et al.  Neuromorphic Vibrotactile Stimulation of Fingertips for Encoding Object Stiffness in Telepresence Sensory Substitution and Augmentation Applications , 2018, Sensors.

[3]  Gordon Cheng,et al.  Tactile-based object center of mass exploration and discrimination , 2017, 2017 IEEE-RAS 17th International Conference on Humanoid Robotics (Humanoids).

[4]  Eduardo Palermo,et al.  Neuromorphic haptic glove and platform with gestural control for tactile sensory feedback in medical telepresence applications , 2018, 2018 IEEE International Symposium on Medical Measurements and Applications (MeMeA).

[5]  B. Green,et al.  Thermal stimulation of taste , 2000, Nature.

[6]  Eugene M. Izhikevich,et al.  Simple model of spiking neurons , 2003, IEEE Trans. Neural Networks.

[7]  Peter Korondi,et al.  Vibro-tactile feedback for VR systems , 2009, 2009 2nd Conference on Human System Interactions.

[8]  Jeff Miller,et al.  On the analysis of psychometric functions: The Spearman-Kärber method , 2001, Perception & psychophysics.

[9]  Alberto Mazzoni,et al.  Neuromorphic Artificial Touch for Categorization of Naturalistic Textures , 2017, IEEE Transactions on Neural Networks and Learning Systems.

[10]  Jaydev P. Desai,et al.  Real-Time Haptic Feedback in Laparoscopic Tools for Use in Gastro-Intestinal Surgery , 2002, MICCAI.

[11]  Jan Peirs,et al.  A micro optical force sensor for force feedback during minimally invasive robotic surgery , 2003 .

[12]  Chris Melhuish,et al.  Toward Bio-Inspired Tactile Sensing Capsule Endoscopy for Detection of Submucosal Tumors , 2017, IEEE Sensors Journal.

[13]  A. Hodgkin,et al.  A quantitative description of membrane current and its application to conduction and excitation in nerve , 1952, The Journal of physiology.

[14]  W.J. Tompkins,et al.  Electrotactile and vibrotactile displays for sensory substitution systems , 1991, IEEE Transactions on Biomedical Engineering.

[15]  Alberto Mazzoni,et al.  Encapsulation of Piezoelectric Transducers for Sensory Augmentation and Substitution with Wearable Haptic Devices , 2017, Micromachines.

[16]  R. S. Johansson,et al.  Roles of glabrous skin receptors and sensorimotor memory in automatic control of precision grip when lifting rougher or more slippery objects , 2004, Experimental Brain Research.

[17]  Benjamin White,et al.  Perceptual Findings with the Vision-Substitution System , 1970 .

[18]  E. Messing,et al.  Quantitative characterization of viscoelastic properties of human prostate correlated with histology. , 2008, Ultrasound in medicine & biology.

[19]  Timothy N Judkins,et al.  Augmented reality and haptic interfaces for robot‐assisted surgery , 2012, The international journal of medical robotics + computer assisted surgery : MRCAS.

[20]  D. Plewes,et al.  Elastic moduli of normal and pathological human breast tissues: an inversion-technique-based investigation of 169 samples , 2007, Physics in medicine and biology.

[21]  P. Rossini,et al.  Intraneural stimulation elicits discrimination of textural features by artificial fingertip in intact and amputee humans , 2016, eLife.

[22]  James C. Bliss,et al.  Optical-to-Tactile Image Conversion for the Blind , 1970 .

[23]  Mark R. Cutkosky,et al.  A robust, low-cost and low-noise artificial skin for human-friendly robots , 2010, 2010 IEEE International Conference on Robotics and Automation.

[24]  R. T. Verrillo.  Psychophysics of vibrotactile stimulation. , 1985, The Journal of the Acoustical Society of America.

[25]  Giulio Sandini,et al.  Tactile Sensing—From Humans to Humanoids , 2010, IEEE Transactions on Robotics.

[26]  Gordon Cheng,et al.  Active Prior Tactile Knowledge Transfer for Learning Tactual Properties of New Objects , 2018, Sensors.

[27]  L M Patnaik,et al.  Haptics: State of the Art Survey , 2012 .

[28]  Gordon Cheng,et al.  Robust Tactile Descriptors for Discriminating Objects From Textural Properties via Artificial Robotic Skin , 2018, IEEE Transactions on Robotics.

[29]  Alessandro Albini,et al.  Human hand recognition from robotic skin measurements in human-robot physical interactions , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[30]  Claudio Pacchierotti,et al.  Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery , 2016, IEEE Transactions on Biomedical Engineering.

[31]  M. Schijven,et al.  The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review , 2009, Surgical Endoscopy.

[32]  A.U. Alahakone,et al.  Vibrotactile feedback systems: Current trends in rehabilitation, sports and information display , 2009, 2009 IEEE/ASME International Conference on Advanced Intelligent Mechatronics.

[33]  M. Sile O'Modhrain,et al.  Cutaneous Grooves: Composing for the Sense of Touch , 2002 .

[34]  Ellen Yi-Luen Do,et al.  Digital Lollipop , 2016, ACM Trans. Multim. Comput. Commun. Appl..

[35]  Jürgen Schmidhuber,et al.  Learning tactile skills through curious exploration , 2012, Front. Neurorobot..

[36]  Carter C. Collins,et al.  Tactile Television - Mechanical and Electrical Image Projection , 1970 .

[37]  Alphus D. Wilson,et al.  Advances in Electronic-Nose Technologies Developed for Biomedical Applications , 2011, Sensors.

[38]  Koh Hosoda,et al.  Detection and prevention of slip using sensors with different properties embedded in elastic artificial skin on the basis of previous experience , 2011, 2011 15th International Conference on Advanced Robotics (ICAR).

[39]  Ryohei Nakatsu,et al.  Tongue Mounted Interface for Digitally Actuating the Sense of Taste , 2012, 2012 16th International Symposium on Wearable Computers.

[40]  Nigel H. Lovell,et al.  A review of tactile sensing technologies with applications in biomedical engineering , 2012 .

[41]  Nikolaos G. Tsagarakis,et al.  Mechano thermo and proprioceptor feedback for integrated haptic feedback , 1997, Proceedings of International Conference on Robotics and Automation.

[42]  Eugene M. Izhikevich,et al.  Which model to use for cortical spiking neurons? , 2004, IEEE Transactions on Neural Networks.

[43]  A. Wing,et al.  Active touch sensing , 2011, Philosophical Transactions of the Royal Society B: Biological Sciences.

[44]  Gaurav S. Sukhatme,et al.  Force estimation and slip detection/classification for grip control using a biomimetic tactile sensor , 2015, 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids).

[45]  Christoph H. Lampert,et al.  Learning Dynamic Tactile Sensing With Robust Vision-Based Training , 2011, IEEE Transactions on Robotics.

[46]  Seungmoon Choi,et al.  Vibrotactile Display: Perception, Technology, and Applications , 2013, Proceedings of the IEEE.

[47]  Gordon Cheng,et al.  Tactile-based active object discrimination and target object search in an unknown workspace , 2018, Autonomous Robots.

[48]  BENJAMIN WHITE,et al.  Vision Substitution by Tactile Image Projection , 1969, Nature.

[49]  P K Kuhl,et al.  Investigating the MESA (multipoint electrotactile speech aid): the transmission of connected discourse. , 1979, The Journal of the Acoustical Society of America.

[50]  R. Johansson,et al.  Properties of cutaneous mechanoreceptors in the human hand related to touch sensation. , 1984, Human neurobiology.

[51]  J. Sibert,et al.  Vibrotactile Feedback for Enhanced Control of Urban Search and Rescue Robots , 2007 .

[52]  Gordon Cheng,et al.  New materials and advances in making electronic skin for interactive robots , 2015, Adv. Robotics.

[53]  Mahdi Tavakoli,et al.  A force reflective master-slave system for minimally invasive surgery , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[54]  Mark Lee,et al.  Review Article Tactile sensing for mechatronics—a state of the art survey , 1999 .

[55]  Silvestro Micera,et al.  Artificial spatiotemporal touch inputs reveal complementary decoding in neocortical neurons , 2017, Scientific Reports.

[56]  R. Johansson,et al.  Tactile sensibility in the human hand: relative and absolute densities of four types of mechanoreceptive units in glabrous skin. , 1979, The Journal of physiology.

[57]  Hiroo Iwata,et al.  Food simulator: a haptic interface for biting , 2004, IEEE Virtual Reality 2004.

[58]  Lorna M. Brown,et al.  Tactons: Structured Tactile Messages for Non-Visual Information Display , 2004, AUIC.