Cutaneous Feedback of Fingertip Deformation and Vibration for Palpation in Robotic Surgery

Despite its expected clinical benefits, current teleoperated surgical robots do not provide the surgeon with haptic feedback largely because grounded forces can destabilize the system's closed-loop controller. This paper presents an alternative approach that enables the surgeon to feel fingertip contact deformations and vibrations while guaranteeing the teleoperator's stability. We implemented our cutaneous feedback solution on an Intuitive Surgical da Vinci Standard robot by mounting a SynTouch BioTac tactile sensor to the distal end of a surgical instrument and a custom cutaneous display to the corresponding master controller. As the user probes the remote environment, the contact deformations, dc pressure, and ac pressure (vibrations) sensed by the BioTac are directly mapped to input commands for the cutaneous device's motors using a model-free algorithm based on look-up tables. The cutaneous display continually moves, tilts, and vibrates a flat plate at the operator's fingertip to optimally reproduce the tactile sensations experienced by the BioTac. We tested the proposed approach by having eighteen subjects use the augmented da Vinci robot to palpate a heart model with no haptic feedback, only deformation feedback, and deformation plus vibration feedback. Fingertip deformation feedback significantly improved palpation performance by reducing the task completion time, the pressure exerted on the heart model, and the subject's absolute error in detecting the orientation of the embedded plastic stick. Vibration feedback significantly improved palpation performance only for the seven subjects who dragged the BioTac across the model, rather than pressing straight into it.

[1]  R. Klatzky,et al.  Hand movements: A window into haptic object recognition , 1987, Cognitive Psychology.

[2]  Thomas B. Sheridan,et al.  Telerobotics , 1989, Autom..

[3]  Blake Hannaford,et al.  Performance evaluation of a six-axis generalized force-reflecting teleoperator , 1991, IEEE Trans. Syst. Man Cybern..

[4]  T B Sheridan,et al.  Teleoperator Performance with Varying Force and Visual Feedback , 1994, Human factors.

[5]  Robert D. Howe,et al.  Tactile Display of Vibratory Information in Teleoperation and Virtual Environments , 1995, Presence: Teleoperators & Virtual Environments.

[6]  Septimiu E. Salcudean,et al.  Performance measurement in scaled teleoperation for microsurgery , 1997, CVRMed.

[7]  Sunil Arya,et al.  ANN: library for approximate nearest neighbor searching , 1998 .

[8]  Kenneth O. Johnson,et al.  The roles and functions of cutaneous mechanoreceptors , 2001, Current Opinion in Neurobiology.

[9]  Arif Kazi,et al.  Operator Performance in Surgical Telemanipulation , 2001, Presence: Teleoperators & Virtual Environments.

[10]  R. Johansson,et al.  Encoding of Direction of Fingertip Forces by Human Tactile Afferents , 2001, The Journal of Neuroscience.

[11]  Jaydev P. Desai,et al.  A Novel Approach to Robotic Cardiac Surgery Using Haptics and Vision , 2002 .

[12]  Christopher R. Wagner,et al.  The role of force feedback in surgery: analysis of blunt dissection , 2002, Proceedings 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems. HAPTICS 2002.

[13]  Theodoros N. Arvanitis,et al.  Objective surgical performance evaluation based on haptic feedback. , 2002, Studies in health technology and informatics.

[14]  Thomas B. Sheridan,et al.  Telerobotics, Automation, and Human Supervisory Control , 2003 .

[15]  Fabrice Labeau,et al.  Discrete Time Signal Processing , 2004 .

[16]  Allison M. Okamura,et al.  Methods for haptic feedback in teleoperated robot-assisted surgery , 2004 .

[17]  L. Wauben,et al.  Application of ergonomic guidelines during minimally invasive surgery: a questionnaire survey of 284 surgeons , 2006, Surgical Endoscopy And Other Interventional Techniques.

[18]  Mark R. Cutkosky,et al.  Contact Location Display for Haptic Perception of Curvature and Object Motion , 2005, Int. J. Robotics Res..

[19]  Mark R. Cutkosky,et al.  Feedback Strategies for Telemanipulation with Shared Control of Object Handling Forces , 2005, Presence: Teleoperators & Virtual Environments.

[20]  D. Yuh,et al.  Effect of sensory substitution on suture-manipulation forces for robotic surgical systems. , 2005, The Journal of thoracic and cardiovascular surgery.

[21]  Caroline G. L. Cao,et al.  Vibrotactile force feedback system for minimally invasive surgical procedures , 2006, 2006 IEEE International Conference on Systems, Man and Cybernetics.

[22]  John Kenneth Salisbury,et al.  Stability of Haptic Rendering: Discretization, Quantization, Time Delay, and Coulomb Effects , 2006, IEEE Transactions on Robotics.

[23]  R. Berguer,et al.  Postural ergonomics during robotic and laparoscopic gastric bypass surgery: a pilot project , 2007, Journal of robotic surgery.

[24]  Jaydev P. Desai,et al.  Evaluating the Effect of Force Feedback in Cell Injection , 2007, IEEE Transactions on Automation Science and Engineering.

[25]  M. Schijven,et al.  The value of haptic feedback in conventional and robot-assisted minimal invasive surgery and virtual reality training: a current review , 2009, Surgical Endoscopy.

[26]  J. Dankelman,et al.  Haptics in minimally invasive surgery – a review , 2008, Minimally invasive therapy & allied technologies : MITAT : official journal of the Society for Minimally Invasive Therapy.

[27]  Warren S. Grundfest,et al.  A Multielement Tactile Feedback System for Robot-Assisted Minimally Invasive Surgery , 2009, IEEE Transactions on Haptics.

[28]  A. Okamura Haptic feedback in robot-assisted minimally invasive surgery , 2009, Current opinion in urology.

[29]  Allison M. Okamura,et al.  Tissue property estimation and graphical display for teleoperated robot-assisted surgery , 2009, 2009 IEEE International Conference on Robotics and Automation.

[30]  Catherine E. Lewis,et al.  Tactile Feedback Induces Reduced Grasping Force in Robot-Assisted Surgery , 2009, IEEE Transactions on Haptics.

[31]  Claudio Pacchierotti,et al.  Using a Fingertip Tactile Device to Substitute Kinesthetic Feedback in Haptic Interaction , 2010, EuroHaptics.

[32]  Gerald E. Loeb,et al.  Haptic feature extraction from a biomimetic tactile sensor: Force, contact location and curvature , 2011, 2011 IEEE International Conference on Robotics and Biomimetics.

[33]  Katherine J. Kuchenbecker,et al.  Tool Contact Acceleration Feedback for Telerobotic Surgery , 2011, IEEE Transactions on Haptics.

[34]  David I. Lee,et al.  Future perspectives in robotic surgery , 2011, BJU international.

[35]  Claudio Pacchierotti,et al.  Cutaneous Force Feedback as a Sensory Subtraction Technique in Haptics , 2011, IEEE Transactions on Haptics.

[36]  Claudio Pacchierotti,et al.  Two Finger Grasping Simulation with Cutaneous and Kinesthetic Force Feedback , 2012, EuroHaptics.

[37]  Claudio Pacchierotti,et al.  Towards Wearability in Fingertip Haptics: A 3-DoF Wearable Device for Cutaneous Force Feedback , 2013, IEEE Transactions on Haptics.

[38]  Kaspar Althoefer,et al.  Haptics for Multi-fingered Palpation , 2013, 2013 IEEE International Conference on Systems, Man, and Cybernetics.

[39]  Claudio Pacchierotti,et al.  A Data-Driven Approach to Remote Tactile Interaction: From a BioTac Sensor to any Fingertip Cutaneous Device , 2014, EuroHaptics.

[40]  Gerald E. Loeb,et al.  Using the BioTac as a tumor localization tool , 2014, 2014 IEEE Haptics Symposium (HAPTICS).

[41]  K. J. Kuchenbecker,et al.  Surgeons and non-surgeons prefer haptic feedback of instrument vibrations during robotic surgery , 2015, Surgical Endoscopy.

[42]  Claudio Pacchierotti,et al.  Sensory Subtraction in Robot-Assisted Surgery: Fingertip Skin Deformation Feedback to Ensure Safety and Improve Transparency in Bimanual Haptic Interaction , 2014, IEEE Transactions on Biomedical Engineering.

[43]  Kaspar Althoefer,et al.  Multi-fingered haptic palpation utilizing granular jamming stiffness feedback actuators , 2014 .

[44]  Claudio Pacchierotti,et al.  Improving Transparency in Teleoperation by Means of Cutaneous Tactile Force Feedback , 2014, TAP.

[45]  Allison M. Okamura,et al.  Controllable Surface Haptics via Particle Jamming and Pneumatics , 2015, IEEE Transactions on Haptics.

[46]  Claudio Pacchierotti,et al.  Displaying Sensed Tactile Cues with a Fingertip Haptic Device , 2015, IEEE Transactions on Haptics.