Interactive force-sensing feedback system for remote robotic laparoscopic surgery

This paper presents the details of a combined hardware/software system, which has been developed to provide haptic feedback for teleoperated laparoscopic surgical robots. Surgical instruments incorporating quantum tunnelling composite (QTC) force measuring sensors have been developed and mounted on a pair of Mitsubishi PA-10 industrial robots. Feedback forces are rendered on pseudo-surgical instruments based on a pair of PHANTOM Omni devices, which are also used to remotely manipulate the robotic arms. Measurements of the behaviour of the QTC sensors during a simulated teleoperated procedure are given. In addition, a method is proposed that can compensate for their non-linear characteristics in order to provide a ‘realistic feel’ to the surgeon through the haptic feedback channel. The paper concludes by explaining how the force feedback channel is combined with a visual feedback channel to enable a surgeon to perform a two-handed surgical procedure better on a remote patient by more accurately controlling a pair of robot arms via a computer network.

[1]  Siep Weiland,et al.  Haptic feedback designs in teleoperation systems for minimal invasive surgery , 2004, 2004 IEEE International Conference on Systems, Man and Cybernetics (IEEE Cat. No.04CH37583).

[2]  A. Castellanos,et al.  Force Feedback Plays a Significant Role in Minimally Invasive Surgery: Results and Analysis , 2005, Annals of surgery.

[3]  M. Hashizume,et al.  Early experiences of endoscopic procedures in general surgery assisted by a computer-enhanced surgical system , 2002, Surgical Endoscopy And Other Interventional Techniques.

[4]  A. Okamura.  Haptic feedback in robot-assisted minimally invasive surgery , 2009, Current opinion in urology.

[5]  Robert W. Lindeman,et al.  Handling of Virtual Contact in Immersive Virtual Environments: Beyond Visuals , 2002, Virtual Reality.

[6]  Christopher R. Wagner,et al.  Force Feedback Benefit Depends on Experience in Multiple Degree of Freedom Robotic Surgery Task , 2007, IEEE Transactions on Robotics.

[7]  Russell H. Taylor,et al.  A Modular 2-DOF Force-Sensing Instrument For Laparoscopic Surgery , 2003, MICCAI.

[8]  Jaesoon Choi,et al.  An Implementation of Sensor-Based Force Feedback in a Compact Laparoscopic Surgery Robot , 2009, ASAIO journal.

[9]  Karen McMenemy,et al.  Adding tactile realism to a virtual reality laparoscopic surgical simulator with a cost-effective human interface device , 2006, Electronic Imaging.

[10]  Cynthia Breazeal,et al.  Design of a therapeutic robotic companion for relational, affective touch , 2005, ROMAN 2005. IEEE International Workshop on Robot and Human Interactive Communication, 2005..

[11]  Stuart Ferguson,et al.  Measuring the Forces Applied to a Virtual Reality Laparoscopic Surgical Simulator with Quantum Tunnelling Composite Sensors , 2008, BIODEVICES.

[12]  Ana Luisa Trejos,et al.  A Sensorized Instrument for Skills Assessment and Training in Minimally Invasive Surgery , 2009 .

[13]  Miriam Reiner,et al.  Seeing Through Touch: The Role of Haptic Information in Visualization , 2008 .

[14]  Dominiek Reynaerts,et al.  Design of an optical force sensor for force feedback during minimally invasive robotic surgery , 2003 .

[15]  Dean Callaghan,et al.  A Force Measurement Evaluation Tool for Telerobotic Cutting Applications: Development of an Effective Characterization Platform , 2007 .

[16]  David Bloor,et al.  Metal–polymer composite with nanostructured filler particles and amplified physical properties , 2006 .

[17]  Adam Whiton,et al.  A Wearable for Physical Abuse Detection , 2007, 2007 11th IEEE International Symposium on Wearable Computers.

[18]  Kay M. Stanney,et al.  Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations , 2004, IEEE Computer Graphics and Applications.

[19]  D. Yuh,et al.  Effects of visual force feedback on robot-assisted surgical task performance. , 2008, The Journal of thoracic and cardiovascular surgery.

[20]  Gerd Hirzinger,et al.  Prototypic Force Feedback Instrument for Minimally Invasive Robotic Surgery , 2008 .

[21]  Thomas Harold Massie,et al.  Design of a three degree of freedom force-reflecting haptic interface , 1993 .

[22]  Shuxiang Guo,et al.  A Tele-operation system based on haptic feedback , 2006, 2006 IEEE International Conference on Information Acquisition.

[23]  Alin Albu-Schäffer,et al.  DLR MiroSurge: a versatile system for research in endoscopic telesurgery , 2010, International Journal of Computer Assisted Radiology and Surgery.

[24]  Rick Kazman,et al.  Vibrotactile feedback in delicate virtual reality operations , 1997, MULTIMEDIA '96.

[25]  Robert O. Ambrose,et al.  Tactile gloves for autonomous grasping with the NASA/DARPA Robonaut , 2004, IEEE International Conference on Robotics and Automation, 2004. Proceedings. ICRA '04. 2004.

[26]  Rajnikant V. Patel,et al.  Evaluation of force feedback requirements for minimally invasive lung tumour localization , 2007, 2007 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[27]  Stuart Ferguson,et al.  Interactive force sensing feedback system for remote robotic laparoscopic surgery , 2009, 2009 IEEE International Workshop on Haptic Audio visual Environments and Games.

[28]  R.A. Russell,et al.  A Robotic Opto-tactile Sensor for Assessing Object Surface Texture , 2006, 2006 IEEE Conference on Robotics, Automation and Mechatronics.

[29]  Pilot study of a device for measuring instrument forces during endoscopic sinus surgery , 2004, Acta oto-laryngologica.

[30]  Mahdi Tavakoli,et al.  A haptic interface for computer-integrated endoscopic surgery and training , 2006, Virtual Reality.

[31]  N. Zemiti,et al.  Mechatronic Design of a New Robot for Force Control in Minimally Invasive Surgery , 2007, IEEE/ASME Transactions on Mechatronics.

[32]  J. R. Hewit,et al.  Tactile sensing technology for minimal access surgery––a review , 2003 .

[33]  M.D. Hancher,et al.  A modular robotic system with applications to space exploration , 2006, 2nd IEEE International Conference on Space Mission Challenges for Information Technology (SMC-IT'06).

[34]  A. Csillag,et al.  Atlas of the sensory organs : functional and clinical anatomy , 2005 .