Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing

This paper presents a robotic vision system that automatically retrieves and positions surgical instruments during robotized laparoscopic surgical operations. The instrument is mounted on the end-effector of a surgical robot which is controlled by visual servoing. The goal of the automated task is to safely bring the instrument at a desired three-dimensional location from an unknown or hidden position. Light-emitting diodes are attached on the tip of the instrument, and a specific instrument holder fitted with optical fibers is used to project laser dots on the surface of the organs. These optical markers are detected in the endoscopic image and allow localizing the instrument with respect to the scene. The instrument is recovered and centered in the image plane by means of a visual servoing algorithm using feature errors in the image. With this system, the surgeon can specify a desired relative position between the instrument and the pointed organ. The relationship between the velocity screw of the surgical instrument and the velocity of the markers in the image is estimated online and, for safety reasons, a multistages servoing scheme is proposed. Our approach has been successfully validated in a real surgical environment by performing experiments on living tissues in the surgical training room of the Institut de Recherche sur les Cancers de l'Appareil Digestif (IRCAD), Strasbourg, France.

[1]  Min-Seok Kim,et al.  복강경 수술용 도구의 실시간 영상 추적 및 복강경 조종기의 지능형 제어 방법 = Real-time visual servoing for laparoscopic surgery , 2003 .

[2]  François Chaumette,et al.  2½D visual servoing , 1999, IEEE Trans. Robotics Autom..

[3]  Minoru Asada,et al.  Versatile visual servoing without knowledge of true Jacobian , 1994, Proceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS'94).

[4]  Yoshihiko Nakamura,et al.  Laser-pointing endoscope system for intra-operative 3D geometric registration , 2001, Proceedings 2001 ICRA. IEEE International Conference on Robotics and Automation (Cat. No.01CH37164).

[5]  Y F Wang,et al.  A new framework for vision-enabled and robotically assisted minimally invasive surgery. , 1998, Computerized medical imaging and graphics : the official journal of the Computerized Medical Imaging Society.

[6]  G. Hirzinger,et al.  Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation , 1997, IEEE Engineering in Medicine and Biology Magazine.

[7]  Peter I. Corke,et al.  A tutorial on visual servo control , 1996, IEEE Trans. Robotics Autom..

[8]  Patrick Rives,et al.  A new approach to visual servoing in robotics , 1992, IEEE Trans. Robotics Autom..

[9]  Yoshihiko Nakamura,et al.  Laser-Pointing Endoscope System for Intraoperative 3D Geometric Registration , 2003 .

[10]  Luc Soler,et al.  Development of Semi-autonomous Control Modes in Laparoscopic Surgery Using Automatic Visual Servoing , 2001, MICCAI.

[11]  E. Malis,et al.  2D 1/2 Visual Servoing , 1998 .

[12]  Shimon Ullman,et al.  Recognizing solid objects by alignment with an image , 1990, International Journal of Computer Vision.

[13]  Luc Soler,et al.  Autonomous retrieval and positioning of surgical instruments in robotized laparoscopic surgery using visual servoing and laser pointers , 2002, Proceedings 2002 IEEE International Conference on Robotics and Automation (Cat. No.02CH37292).

[14]  Miguel Ribo.  State of the Art Report on Optical Tracking , 2001 .

[15]  Luc Soler,et al.  Towards Semi-autonomy in Laparoscopic Surgery through Vision and Force Feedback Control , 2000, ISER.

[16]  Harvey Lipkin,et al.  A dynamic quasi-Newton method for uncalibrated visual servoing , 1999, Proceedings 1999 IEEE International Conference on Robotics and Automation (Cat. No.99CH36288C).

[17]  Olac Fuentes,et al.  Experimental evaluation of uncalibrated visual servoing for precision manipulation , 1997, Proceedings of International Conference on Robotics and Automation.