Providing haptic feedback in robot-assisted minimally invasive surgery: A direct optical force-sensing solution for haptic rendering of deformable bodies

Abstract This paper presents an enhanced haptic-enabled master-slave teleoperation system which can be used to provide force feedback to surgeons in minimally invasive surgery (MIS). One of the research goals was to develop a combined-control architecture framework that included both direct force reflection (DFR) and position-error-based (PEB) control strategies. To achieve this goal, it was essential to measure accurately the direct contact forces between deformable bodies and a robotic tool tip. To measure the forces at a surgical tool tip and enhance the performance of the teleoperation system, an optical force sensor was designed, prototyped, and added to a robot manipulator. The enhanced teleoperation architecture was formulated by developing mathematical models for the optical force sensor, the extended slave robot manipulator, and the combined-control strategy. Human factor studies were also conducted to (a) examine experimentally the performance of the enhanced teleoperation system with the optical force sensor, and (b) study human haptic perception during the identification of remote object deformability. The first experiment was carried out to discriminate deformability of objects when human subjects were in direct contact with deformable objects by means of a laparoscopic tool. The control parameters were then tuned based on the results of this experiment using a gain-scheduling method. The second experiment was conducted to study the effectiveness of the force feedback provided through the enhanced teleoperation system. The results show that the force feedback increased the ability of subjects to correctly identify materials of different deformable types. In addition, the virtual force feedback provided by the teleoperation system comes close to the real force feedback experienced in direct MIS. The experimental results provide design guidelines for choosing and validating the control architecture and the optical force sensor.

[1]  M. Moallem,et al.  Methods and mechanisms for contact feedback in a robot-assisted minimally invasive environment , 2006, Surgical Endoscopy And Other Interventional Techniques.

[2]  S. Rush,et al.  Resistivity of Body Tissues at Low Frequencies , 1963, Circulation research.

[3]  Robert D. Howe,et al.  A Robust Uniaxial Force Sensor for Minimally Invasive Surgery , 2010, IEEE Transactions on Biomedical Engineering.

[4]  Hongbin Liu,et al.  Miniature 3-Axis Distal Force Sensor for Minimally Invasive Surgical Palpation , 2012, IEEE/ASME Transactions on Mechatronics.

[5]  P. Polygerinos,et al.  Triaxial Catheter-Tip Force Sensor for MRI-Guided Cardiac Procedures , 2013, IEEE/ASME Transactions on Mechatronics.

[6]  Ashitava Ghosal,et al.  A force–torque sensor based on a Stewart Platform in a near-singular configuration , 2004 .

[7]  R. Gassert,et al.  MRI/fMRI-compatible robotic system with force feedback for interaction with human motion , 2006, IEEE/ASME Transactions on Mechatronics.

[8]  Jan Peirs,et al.  A micro optical force sensor for force feedback during minimally invasive robotic surgery , 2003 .

[9]  B. Hannaford,et al.  Task decomposition of laparoscopic surgery for objective evaluation of surgical residents' learning curve using hidden Markov model. , 2002, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[10]  D. Yuh,et al.  Effects of visual force feedback on robot-assisted surgical task performance. , 2008, The Journal of thoracic and cardiovascular surgery.

[11]  Kaspar Althoefer,et al.  MRI-Compatible Intensity-Modulated Force Sensor for Cardiac Catheterization Procedures , 2011, IEEE Transactions on Biomedical Engineering.

[12]  Mathias Stefan Müller,et al.  Fiber Bragg Grating-Based Force-Torque Sensor with Six Degrees of Freedom , 2009 .

[13]  Panadda Marayong,et al.  The effect of visual and haptic feedback on computer-assisted needle insertion , 2004, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[14]  Alessandro Massaro,et al.  INNOVATIVE OPTICAL TACTILE SENSOR FOR ROBOTIC SYSTEM BY GOLD NANOCOMPOSITE MATERIAL , 2011 .

[15]  Gregory S. Fischer,et al.  A 3-axis optical force/torque sensor for prostate needle placement in Magnetic resonance imaging environments , 2009, 2009 IEEE International Conference on Technologies for Practical Robot Applications.

[16]  Kaspar Althoefer,et al.  Mathematical Modeling of Intensity-Modulated Bent-Tip Optical Fiber Displacement Sensors , 2010, IEEE Transactions on Instrumentation and Measurement.

[17]  Mandayam A Srinivasan,et al.  Virtual-reality-based laparoscopic surgical training: The role of simulation fidelity in haptic feedback , 2004, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[18]  Christos Trantakis,et al.  Fundamentals of Force Feedback and Application to a Surgery Simulator , 2003, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[19]  Jaydev P. Desai,et al.  Real-Time Haptic Feedback in Laparoscopic Tools for Use in Gastro-Intestinal Surgery , 2002, MICCAI.

[20]  Ana Luisa Trejos,et al.  Force sensing and its application in minimally invasive surgery and therapy: A survey , 2010 .

[21]  Dale A. Lawrence Stability and transparency in bilateral teleoperation , 1993, IEEE Trans. Robotics Autom..

[22]  Kaspar Althoefer,et al.  Modeling of Light Intensity-Modulated Fiber-Optic Displacement Sensors , 2011, IEEE Transactions on Instrumentation and Measurement.

[23]  L.D. Seneviratne,et al.  State-of-the-Art in Force and Tactile Sensing for Minimally Invasive Surgery , 2008, IEEE Sensors Journal.

[24]  Bernhard Kübler,et al.  Prototype of Instrument for Minimally Invasive Surgery with 6-Axis Force Sensing Capability , 2005, Proceedings of the 2005 IEEE International Conference on Robotics and Automation.

[25]  Blake Hannaford,et al.  A design framework for teleoperators with kinesthetic feedback , 1989, IEEE Trans. Robotics Autom..

[26]  Alin Albu-Schäffer,et al.  DLR MiroSurge: a versatile system for research in endoscopic telesurgery , 2010, International Journal of Computer Assisted Radiology and Surgery.

[27]  D. Burschka,et al.  Dynamic Augmented Reality for Sensory Substitution in Robot-Assisted Surgical Systems , 2006, 2006 International Conference of the IEEE Engineering in Medicine and Biology Society.

[28]  Alejandro Jarillo Silva,et al.  PHANToM OMNI Haptic Device: Kinematic and Manipulability , 2009, 2009 Electronics, Robotics and Automotive Mechanics Conference (CERMA).

[29]  Tobias Ortmaier,et al.  A Virtual-Reality-Based Haptic Surgical Training System , 2003, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[30]  Ming Zhu,et al.  Transparent Bilateral Teleoperation under Position and Rate Control , 2000, Int. J. Robotics Res..