Cutaneous haptic feedback to ensure the stability of robotic teleoperation systems

Cutaneous haptic feedback can be used to enhance the performance of robotic teleoperation systems while guaranteeing their safety. Delivering ungrounded cutaneous cues to the human operator conveys in fact information about the forces exerted at the slave side and does not affect the stability of the control loop. In this work we analyze the feasibility, effectiveness, and implications of providing solely cutaneous feedback in robotic teleoperation. We carried out two peg-in-hole experiments, both in a virtual environment and in a real (teleoperated) environment. Two novel 3-degree-of-freedom fingertip cutaneous displays deliver a suitable amount of cutaneous feedback at the thumb and index fingers. Results assessed the feasibility and effectiveness of the proposed approach. Cutaneous feedback was outperformed by full haptic feedback provided by grounded haptic interfaces, but it outperformed conditions providing no force feedback at all. Moreover, cutaneous feedback always kept the system stable, even in the presence of destabilizing factors such as communication delays and hard contacts.

[1]  Thomas B. Sheridan,et al.  Telerobotics, Automation, and Human Supervisory Control , 2003 .

[2]  Warren S. Grundfest,et al.  A Multielement Tactile Feedback System for Robot-Assisted Minimally Invasive Surgery , 2009, IEEE Transactions on Haptics.

[3]  Dale A. Lawrence Stability and transparency in bilateral teleoperation , 1993, IEEE Trans. Robotics Autom..

[4]  Blake Hannaford,et al.  Task-level testing of the JPL-OMV smart end effector , 1987 .

[5]  Allison M. Okamura,et al.  Sensory Substitution and Augmentation Using 3-Degree-of-Freedom Skin Deformation Feedback , 2015, IEEE Transactions on Haptics.

[6]  Christopher R. Wagner,et al.  The role of force feedback in surgery: analysis of blunt dissection , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[7]  Vincent Hayward,et al.  Haptic interfaces and devices , 2004 .

[8]  K. J. Kuchenbecker,et al.  Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery , 2015, Surgical Endoscopy.

[9]  Claudio Pacchierotti,et al.  Improving Transparency in Teleoperation by Means of Cutaneous Tactile Force Feedback , 2014, TAP.

[10]  Michael J. Massimino,et al.  Improved force perception through sensory substitution , 1995 .

[11]  Kaspar Althoefer,et al.  Haptics for Multi-fingered Palpation , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[12]  Thomas B. Sheridan Teleoperation, Telerobotics, and Telepresence: A Progress Report , 1992 .

[13]  William R. Provancher,et al.  Fingerpad Skin Stretch Increases the Perception of Virtual Friction , 2009, IEEE Transactions on Haptics.

[14]  R. Johansson,et al.  Encoding of Direction of Fingertip Forces by Human Tactile Afferents , 2001, The Journal of Neuroscience.

[15]  Jean-Jacques E. Slotine,et al.  Telemanipulation with Time Delays , 2004, Int. J. Robotics Res..

[16]  Robert J. Webster,et al.  A novel two-dimensional tactile slip display: design, kinematics and perceptual experiments , 2005, TAP.

[17]  Antonio Frisoli,et al.  Design of a SMA actuated 2-DoF tactile device for displaying tangential skin displacement , 2011, 2011 IEEE World Haptics Conference.

[18]  T B Sheridan,et al.  Teleoperator Performance with Varying Force and Visual Feedback , 1994, Human factors.

[19]  Claudio Pacchierotti,et al.  Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery , 2016, IEEE Transactions on Biomedical Engineering.

[20]  B. Edin,et al.  Skin strain patterns provide kinaesthetic information to the human central nervous system. , 1995, The Journal of physiology.

[21]  Kenneth O. Johnson,et al.  The roles and functions of cutaneous mechanoreceptors , 2001, Current Opinion in Neurobiology.

[22]  Tsukasa Ogasawara,et al.  Weight and friction display device by controlling the slip condition of a fingertip , 2011, 2011 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  D. Prattichizzo,et al.  Cutaneous device for teleoperated needle insertion , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[24]  Allison M. Okamura,et al.  Sensory augmentation of stiffness using fingerpad skin stretch , 2013, 2013 World Haptics Conference (WHC).

[25]  Jenny Dankelman,et al.  The Effect of Augmented Feedback on Grasp Force in Laparoscopic Grasp Control , 2010, IEEE Transactions on Haptics.

[26]  John Kenneth Salisbury,et al.  A constraint-based god-object method for haptic display , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[27]  Mark W. Spong,et al.  Bilateral teleoperation: An historical survey , 2006, Autom..

[28]  Jong-Phil Kim,et al.  Robustly Stable Haptic Interaction Control using an Energy-bounding Algorithm , 2010, Int. J. Robotics Res..

[29]  Thomas B. Sheridan,et al.  Telerobotics , 1989, Autom..

[30]  Shinichi Hirai,et al.  Development of a soft-fingertip and its modeling based on force distribution , 2003, 2003 IEEE International Conference on Robotics and Automation (Cat. No.03CH37422).

[31]  Robert D. Howe,et al.  Tactile Display of Vibratory Information in Teleoperation and Virtual Environments , 1995, Presence: Teleoperators & Virtual Environments.

[32]  Diana Bohm,et al.  L2 Gain And Passivity Techniques In Nonlinear Control , 2016 .

[33]  D. Yuh,et al.  Effect of sensory substitution on suture-manipulation forces for robotic surgical systems. , 2005, The Journal of thoracic and cardiovascular surgery.

[34]  Sandra Hirche,et al.  Bilateral Control Architectures for Telerobotics , 2007, Advances in Telerobotics.

[35]  Claudio Pacchierotti,et al.  Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics , 2011, IEEE Transactions on Haptics.

[36]  Catherine E. Lewis,et al.  Tactile Feedback Induces Reduced Grasping Force in Robot-Assisted Surgery , 2009, IEEE Transactions on Haptics.

[37]  Septimiu E. Salcudean,et al.  Control for teleoperation and haptic interfaces , 1998 .

[38]  Claudio Pacchierotti,et al.  Towards Wearability in Fingertip Haptics: A 3-DoF Wearable Device for Cutaneous Force Feedback , 2013, IEEE Transactions on Haptics.

[39]  Antonio Frisoli,et al.  A Fingertip Haptic Display for Improving Curvature Discrimination , 2008, PRESENCE: Teleoperators and Virtual Environments.

[40]  Katherine J. Kuchenbecker,et al.  Tool Contact Acceleration Feedback for Telerobotic Surgery , 2011, IEEE Transactions on Haptics.

[41]  Claudio Pacchierotti,et al.  Sensory Subtraction in Robot-Assisted Surgery: Fingertip Skin Deformation Feedback to Ensure Safety and Improve Transparency in Bimanual Haptic Interaction , 2014, IEEE Transactions on Biomedical Engineering.

[42]  Claudio Pacchierotti,et al.  Two Finger Grasping Simulation with Cutaneous and Kinesthetic Force Feedback , 2012, EuroHaptics.

[43]  Dongjun Lee,et al.  Passive-Set-Position-Modulation Framework for Interactive Robotic Systems , 2010, IEEE Transactions on Robotics.

[44]  K. Salisbury,et al.  Haptic Rendering of Surfaces Defined by Implicit Functions , 1997, Dynamic Systems and Control.

[45]  Blake Hannaford,et al.  Stable teleoperation with time-domain passivity control , 2002, IEEE Transactions on Robotics and Automation.

[46]  Stefano Stramigioli,et al.  Bilateral Telemanipulation With Time Delays: A Two-Layer Approach Combining Passivity and Transparency , 2011, IEEE Transactions on Robotics.

[47]  Vincent Hayward,et al.  Mechanical behavior of the fingertip in the range of frequencies and displacements relevant to touch. , 2012, Journal of biomechanics.

[48]  Claudio Pacchierotti,et al.  Using a Fingertip Tactile Device to Substitute Kinesthetic Feedback in Haptic Interaction , 2010, EuroHaptics.

[49]  Theodoros N. Arvanitis,et al.  Objective surgical performance evaluation based on haptic feedback. , 2002, Studies in health technology and informatics.

[50]  Caroline G. L. Cao,et al.  Vibrotactile force feedback system for minimally invasive surgical procedures , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[51]  Thomas B. Sheridan,et al.  Sensory Substitution for Force Feedback in Teleoperation , 1992, Presence: Teleoperators & Virtual Environments.