Image-Based Trajectory Tracking Control of 4-DoF Laparoscopic Instruments Using a Rotation Distinguishing Marker

In this letter, we propose a new method to fully control complete 4-image-DoF manipulation of laparoscopic instruments [with remote center of motion (RCM) mechanism] based on the geometric features of a designed marker in a 2-D image. Our marker encodes the configuration of the instruments by computing geometric features among the projected image points from segmented areas in hue-saturation-value (HSV) space. We can then construct an image geometric feature vector to locally characterize the configuration of a laparoscopic instrument. Furthermore, we design an image-based kinematic controller to asymptotically track a planned trajectory using the constructed feature vector as the feedback. We evaluate our integration of rotation distinguishing marker and kinematic controller by several experiments in terms of illumination-invariance, rotation angle accuracy, and controller performance.

[1]  Luc Soler,et al.  Autonomous 3-D positioning of surgical instruments in robotized laparoscopic surgery using visual servoing , 2003, IEEE Trans. Robotics Autom..

[2]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[3]  A. Lanfranco,et al.  Robotic Surgery: A Current Perspective , 2004, Annals of surgery.

[4]  Darius Burschka,et al.  DaVinci Canvas: A Telerobotic Surgical System with Integrated, Robot-Assisted, Laparoscopic Ultrasound Capability , 2005, MICCAI.

[5]  R. Veldkamp,et al.  Laparoscopic surgery versus open surgery for colon cancer : short-term outcomes of a randomised trial , 2022 .

[6]  Pierre-Brice Wieber,et al.  Visual servoing when visual information is missing: Experimental comparison of visual feature prediction schemes , 2015, 2015 IEEE International Conference on Robotics and Automation (ICRA).

[7]  Guangjun Zhang,et al.  Calibrating camera radial distortion with cross-ratio invariability , 2003 .

[8]  Nima Najmaei,et al.  Visual servoing in medical robotics: a survey. Part I: endoscopic and direct vision imaging – techniques and applications , 2014, The international journal of medical robotics + computer assisted surgery : MRCAS.

[9]  G. Hirzinger,et al.  Real-time visual servoing for laparoscopic surgery. Controlling robot motion with color image segmentation , 1997, IEEE Engineering in Medicine and Biology Magazine.

[10]  Peter Kazanzides,et al.  Computer-integrated revision total hip replacement surgery: concept and preliminary results , 1999, Medical Image Anal..

[11]  Chunwoo Kim,et al.  Endocavity Ultrasound Probe Manipulators , 2013, IEEE/ASME Transactions on Mechatronics.

[12]  M. Braga,et al.  Training period in laparoscopic colorectal surgery , 2001, Surgical Endoscopy And Other Interventional Techniques.

[13]  Min-Seok Kim,et al.  Visual Tracking Algorithm for Laparoscopic Robot Surgery , 2005, FSKD.

[14]  Hongliang Ren,et al.  Passive Markers for Tracking Surgical Instruments in Real-Time 3-D Ultrasound Imaging , 2012, IEEE Transactions on Medical Imaging.

[15]  Luc Soler,et al.  Circular needle and needle-holder localization for computer-aided suturing in laparoscopic surgery , 2005, SPIE Medical Imaging.

[16]  Philippe Zanne,et al.  A Model-free Vision-based Robot Control for Minimally Invasive Surgery using ESM Tracking and Pixels Color Selection , 2007, Proceedings 2007 IEEE International Conference on Robotics and Automation.

[17]  Zerui Wang,et al.  Adaptive image-based positioning of RCM mechanisms using angle and distance features , 2015, 2015 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[18]  Russell H. Taylor,et al.  Control of the coupled motion of a 6 DoF robotic arm and a continuum manipulator for the treatment of pelvis osteolysis , 2014, 2014 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[19]  J C Demoulin,et al.  Laparoscopic Nissen fundoplication: detailed analysis of 132 patients. , 1993, Surgical laparoscopy & endoscopy.

[20]  Philippe Cinquin,et al.  ViKY Robotic Scope Holder: Initial Clinical Experience and Preliminary Results Using Instrument Tracking , 2010, IEEE/ASME Transactions on Mechatronics.

[21]  Jean Ponce,et al.  Computer Vision: A Modern Approach , 2002 .

[22]  Philippe Zanne,et al.  Visual Servoing-Based Endoscopic Path Following for Robot-Assisted Laparoscopic Surgery , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[23]  Russell H. Taylor,et al.  A telerobotic assistant for laparoscopic surgery , 1995 .

[24]  Farzam Farahmand,et al.  Simulation and control of a multi-DOF laparoscopic tele-surgery system in virtual reality , 2011, The 2nd International Conference on Control, Instrumentation and Automation.

[25]  Russell H. Taylor,et al.  Medical robotics in computer-integrated surgery , 2003, IEEE Trans. Robotics Autom..