Telesurgery and surgical simulation: design, modeling, and evaluation of haptic interfaces to real and virtual surgical environments

Telesurgery and Surgical Simulation: Design, Modeling, and Evaluation of Haptic Interfaces to Real and Virtual Surgical Environments by Murat Cenk C avu so glu Doctor of Philosophy in Engineering | Electrical Engineering and Computer Sciences University of California at Berkeley Professor S. Shankar Sastry, Chair In this thesis, telesurgery and surgical simulation are treated as parallel research problems of haptic interfacing to real and virtual surgical environments, respectively. The analytical tools of systems and control theory and robotics are used to address several research problems in these areas. The work on telesurgery is centered around the UC Berkeley/UC San Francisco Laparoscopic Telesurgical Workstation, focusing on the design and analysis of the system, with details of the design speci cations, solution of the forward and inverse kinematics, and control issues. This is followed by a discussion on the general issues on the testing of the system. A novel approach using open surgical suturing motion data to evaluate the kinematics of a robotic telesurgical manipulator without prototyping a physical system is proposed within this context. High delity teleoperation controller design for the telesurgical system is studied, and a theoretical and experimental framework is established for design and evaluation of teleoperation controllers for telemanipulation of deformable objects. In this study, the teleoperator control design is speci cally for manipulation of deformable objects and uses a task based optimization scheme which explicitly takes into account human perceptual capabilities for the task at hand, telesurgery and sti ness discrimination. Integrated with this control design approach, two quantitative methods to analytically compare sensory schemes for teleoperators are proposed and a new experimental methodology to evaluate

[1]  John C. Platt,et al.  Elastically deformable models , 1987, SIGGRAPH.

[2]  Mark W. Spong,et al.  Bilateral control of teleoperators with time delay , 1988, Proceedings of the 1988 IEEE International Conference on Systems, Man, and Cybernetics.

[3]  John F. Canny,et al.  Haptic interaction with global deformations , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[4]  Blake Hannaford,et al.  Force-reflection and shared compliant control in operating telemanipulators with time delay , 1992, IEEE Trans. Robotics Autom..

[5]  W. Eric L. Grimson,et al.  Simulating arthroscopic knee surgery using volumetric object representations, real-time volume rendering and haptic feedback , 1997, CVRMed.

[6]  Vincent Hayward,et al.  Multirate haptic simulation achieved by coupling finite element meshes through Norton equivalents , 1998, Proceedings. 1998 IEEE International Conference on Robotics and Automation (Cat. No.98CH36146).

[7]  Petros G. Voulgaris,et al.  On optimal ℓ∞ to ℓ∞ filtering , 1995, Autom..

[8]  Jeffrey M. Wendlandt,et al.  Milli Robotics for Endoscopy , 1994 .

[9]  Frederick P. Brooks,et al.  Feeling and seeing: issues in force display , 1990, I3D '90.

[10]  Tsuneo Yoshikawa,et al.  Bilateral control of master-slave manipulators for ideal kinesthetic coupling-formulation and experiment , 1994, IEEE Trans. Robotics Autom..

[11]  J C Bowersox,et al.  Remote operative urology using a surgical telemanipulator system: preliminary observations. , 1998, Urology.

[12]  Robert D. Howe,et al.  Tactile imaging: a method for documenting breast masses , 1999, Proceedings of the First Joint BMES/EMBS Conference. 1999 IEEE Engineering in Medicine and Biology 21st Annual Conference and the 1999 Annual Fall Meeting of the Biomedical Engineering Society (Cat. N.

[13]  Bruce A. Francis,et al.  Bilateral controller for teleoperators with time delay via μ-synthesis , 1995, IEEE Trans. Robotics Autom..

[14]  Jean-Claude Latombe,et al.  Treatment Planning for Image-Guided Robotic Radiosurgery , 1995, CVRMed.

[15]  Blake Hannaford,et al.  Performance evaluation of a six-axis generalized force-reflecting teleoperator , 1991, IEEE Trans. Syst. Man Cybern..

[16]  A Darzi,et al.  Early experience and validation work with Procedicus VA--the Prosolvia virtual reality shoulder arthroscopy trainer. , 1999, Studies in health technology and informatics.

[17]  Homayoon Kazerooni,et al.  A controller design framework for telerobotic systems , 1993, IEEE Trans. Control. Syst. Technol..

[18]  S. E. Salcudean,et al.  On the Emulation of Stiff Walls and Static Friction with a Magnetically Levitated Input/Output Devic , 1997 .

[19]  M. Srinivasan,et al.  Tactual discrimination of softness. , 1995, Journal of neurophysiology.

[20]  Jean-Jacques E. Slotine,et al.  Stable Adaptive Teleoperation , 1990, 1990 American Control Conference.

[21]  Blake Hannaford,et al.  Performance testing of passive communication and control in teleoperation with time delay , 1993, [1993] Proceedings IEEE International Conference on Robotics and Automation.

[22]  Keith K. Green,et al.  Computer-simulated eye surgery. A novel teaching method for residents and practitioners. , 1995, Ophthalmology.

[23]  G Fischer,et al.  Virtual reality arthroscopy training simulator. , 1995, Computers in biology and medicine.

[24]  Lynette A. Jones,et al.  ANALYSIS OF THE HUMAN OPERATOR CONTROLLING A TELEOPERATED MICROSURGICAL ROBOT , 1995 .

[25]  Brian Mirtich,et al.  Impulse-based dynamic simulation of rigid body systems , 1996 .

[26]  Richard A. Robb,et al.  Simulation of spinal nerve blocks for training anesthesiology residents , 1998, Photonics West - Biomedical Optics.

[27]  Acuerdo Para Entrevistanarrador.  Oh! , 1908 .

[28]  F. A. Seiler,et al.  Numerical Recipes in C: The Art of Scientific Computing , 1989 .

[29]  Giuseppe Saccomandi,et al.  Topics in finite elasticity , 2001 .

[30]  Won Soo Kim,et al.  Operator Performance with Alternative Manual Control Modes in Teleoperation , 1992, Presence: Teleoperators & Virtual Environments.

[31]  J L Tasto,et al.  PreOp endoscopic simulator: a PC-based immersive training system for bronchoscopy. , 1999, Studies in health technology and informatics.

[32]  Frank Tendick,et al.  Frequency Dependence of Compliance Contrast Detection , 2000, Dynamic Systems and Control: Volume 2.

[33]  S. Shankar Sastry,et al.  A laparoscopic telesurgical workstation , 1999, IEEE Trans. Robotics Autom..

[34]  Russell H. Taylor,et al.  A telerobotic assistant for laparoscopic surgery , 1995 .

[35]  John W. Hill,et al.  Telepresence surgery demonstration system , 1994, Proceedings of the 1994 IEEE International Conference on Robotics and Automation.

[36]  Dimitris N. Metaxas.  Physics-Based Deformable Models: Applications to Computer Vision, Graphics, and Medical Imaging , 1996 .

[37]  Christopher R. Wagner,et al.  A compliant tactile display for teletaction , 2000, Proceedings 2000 ICRA. Millennium Conference. IEEE International Conference on Robotics and Automation. Symposia Proceedings (Cat. No.00CH37065).

[38]  J. Peifer,et al.  Applied virtual reality for simulation of endoscopic retrograde cholangio-pancreatography (ERCP). , 1996, Studies in health technology and informatics.

[39]  Patrick Dubois,et al.  Training simulator for retinal laser photocoagulation: a new approach for surgeons' apprenticeships , 1998, Photonics West - Biomedical Optics.

[40]  L. Kaufman,et al.  Handbook of perception and human performance , 1986 .

[41]  Christian Laugier,et al.  Parameter identification for dynamic simulation , 1997, Proceedings of International Conference on Robotics and Automation.

[42]  Blake Hannaford,et al.  A design framework for teleoperators with kinesthetic feedback , 1989, IEEE Trans. Robotics Autom..

[43]  Ronald S. Fearing,et al.  Tactile feedback for teleoperation , 1993, Other Conferences.

[44]  Dinesh K. Pai,et al.  ArtDefo: accurate real time deformable objects , 1999, SIGGRAPH.

[45]  Blake Hannaford,et al.  Stable haptic interaction with virtual environments , 1999, IEEE Trans. Robotics Autom..

[46]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[47]  A D McCulloch,et al.  A three-dimensional finite element method for large elastic deformations of ventricular myocardium: I--Cylindrical and spherical polar coordinates. , 1996, Journal of biomechanical engineering.

[48]  Herve Delingette,et al.  Real-Time Elastic Deformations of Soft Tissues for Surgery Simulation , 1999, IEEE Trans. Vis. Comput. Graph..

[49]  K. Bathe.  Finite Element Procedures , 1995 .

[50]  John Kenneth Salisbury,et al.  A constraint-based god-object method for haptic display , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[51]  Peter Kazanzides,et al.  An image-directed robotic system for precise orthopaedic surgery , 1994, IEEE Trans. Robotics Autom..

[52]  Nathan Ida,et al.  Introduction to the Finite Element Method , 1997 .

[53]  R. W. Daniel,et al.  Fundamental Limits of Performance for Force Reflecting Teleoperation , 1998, Int. J. Robotics Res..

[54]  G. Sung,et al.  Robotic-assisted laparoscopic pyeloplasty: a pilot study. , 1999, Urology.

[55]  Philippe Cinquin,et al.  Image guided operating robot: a clinical application in stereotactic neurosurgery , 1992, Proceedings 1992 IEEE International Conference on Robotics and Automation.

[56]  Blake Hannaford,et al.  Stability and performance tradeoffs in bi-lateral telemanipulation , 1989, Proceedings, 1989 International Conference on Robotics and Automation.

[57]  Ronald S. Fearing,et al.  A surface micromachined microtactile sensor array , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[58]  Frank L. Lewis,et al.  Applied Optimal Control and Estimation , 1992 .

[59]  Alberto Rovetta,et al.  Remote control in telerobotic surgery , 1996, IEEE Trans. Syst. Man Cybern. Part A.

[60]  John Kenneth Salisbury,et al.  The Black Falcon: a teleoperated surgical instrument for minimally invasive surgery , 1998, Proceedings. 1998 IEEE/RSJ International Conference on Intelligent Robots and Systems. Innovations in Theory, Practice and Applications (Cat. No.98CH36190).

[61]  Parris S. Wellman,et al.  TEMPORAL BANDWIDTH REQUIREMENTS FOR TACTILE SHAPE DISPLAYS , 1997 .

[62]  C. Detter,et al.  Early experience with robotic technology for coronary artery surgery. , 1999, The Annals of thoracic surgery.

[63]  F. Tendick.  MAXIMIZING THE SENSATION OF COMPLIANCE IN TELEOPERATIVE SURGERY , 1999 .

[64]  Craig R. Carignan,et al.  Simulator/planner for CT-directed needle biopsy of the spine , 1998, Photonics West - Biomedical Optics.

[65]  M. Talamini,et al.  Human vs robotic organ retraction during laparoscopic Nissen fundoplication , 1999, Surgical Endoscopy.

[66]  M. Bro-Nielsen.  Fast finite elements for surgery simulation. , 1997, Studies in health technology and informatics.

[67]  J. Strikwerda.  Finite Difference Schemes and Partial Differential Equations , 1989 .

[68]  S. Shankar Sastry,et al.  Design and control of a simplified Stewart platform for endoscopy , 1994, Proceedings of 1994 33rd IEEE Conference on Decision and Control.

[69]  Christian Laugier,et al.  Towards a realistic echographic simulator with force feedback , 1999, Proceedings 1999 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human and Environment Friendly Robots with High Intelligence and Emotional Quotients (Cat. No.99CH36289).

[70]  J. Himpens,et al.  [Nissen fundoplication done by remotely controlled robotic technique]. , 1999, Annales de chirurgie.

[71]  A. García-Ruiz,et al.  Manual vs robotically assisted laparoscopic surgery in the performance of basic manipulation and suturing tasks. , 1998, Archives of surgery.

[72]  Frank L. Lewis,et al.  Kinematic analysis of a Stewart platform manipulator , 1993, IEEE Trans. Ind. Electron..

[73]  Robert D. Howe,et al.  A tactile shape sensing and display system for teleoperated manipulation , 1995, Proceedings of 1995 IEEE International Conference on Robotics and Automation.

[74]  Mark W. Spong,et al.  Asymptotic Stability for Force Reflecting Teleoperators with Time Delay , 1992 .

[75]  Philip D. Loewen,et al.  Robust controller design for teleoperation systems , 1995, 1995 IEEE International Conference on Systems, Man and Cybernetics. Intelligent Systems for the 21st Century.

[76]  Ronald S. Fearing,et al.  Some basic issues in teletaction , 1997, Proceedings of International Conference on Robotics and Automation.

[77]  Koji Ikuta,et al.  Virtual Endoscope System with Force Sensation , 1998, MICCAI.

[78]  Fumihito Arai,et al.  Multimedia tele-surgery using high speed optical fiber network and its application to intravascular neurosurgery - system configuration and computer networked robotic implementation , 1996, Proceedings of IEEE International Conference on Robotics and Automation.

[79]  M. Bro-Nielsen,et al.  Finite element modeling in surgery simulation , 1998, Proc. IEEE.

[80]  Paul S. Schenker,et al.  A New Robot for High Dexterity Microsurgery , 1995, CVRMed.

[81]  J. Edward Colgate,et al.  Issues in the haptic display of tool use , 1995, Proceedings 1995 IEEE/RSJ International Conference on Intelligent Robots and Systems. Human Robot Interaction and Cooperative Robots.

[82]  S. Shankar Sastry,et al.  Surgical applications of milli-robots , 1995, J. Field Robotics.

[83]  H Delingette,et al.  Efficient linear elastic models of soft tissues for real-time surgery simulation. , 1999, Studies in health technology and informatics.

[84]  Lawrence W. Stark,et al.  Sensing and Manipulation Problems in Endoscopic Surgery: Experiment, Analysis, and Observation , 1993, Presence: Teleoperators & Virtual Environments.

[85]  J. Edward Colgate,et al.  Robust impedance shaping telemanipulation , 1993, IEEE Trans. Robotics Autom..