Hand-eye calibration for robotic assisted minimally invasive surgery without a calibration object

In a robot mounted camera arrangement, hand-eye calibration estimates the rigid relationship between the robot and camera coordinate frames. Most hand-eye calibration techniques use a calibration object to estimate the relative transformation of the camera in several views of the calibration object and link these to the forward kinematics of the robot to compute the hand-eye transformation. Such approaches achieve good accuracy for general use but for applications such as robotic assisted minimally invasive surgery, acquiring a calibration sequence multiple times during a procedure is not practical. In this paper, we present a new approach to tackle the problem by using the robotic surgical instruments as the calibration object with well known geometry from CAD models used for manufacturing. Our approach removes the requirement of a custom sterile calibration object to be used in the operating room and it simplifies the process of acquiring calibration data when the laparoscope is constrained to move around a remote centre of motion. This is the first demonstration of the feasibility to perform hand-eye calibration using components of the robotic system itself and we show promising validation results on synthetic data as well as data acquired with the da Vinci Research Kit.

[1]  Heinrich Niemann,et al.  Robust Hand-Eye Calibration of an Endoscopic Surgery Robot Using Dual Quaternions , 2003, DAGM-Symposium.

[2]  Philippe Cinquin,et al.  Automatic Detection of Instruments in Laparoscopic Images: A First Step Towards High-level Command of Robotic Endoscopic Holders , 2007, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[3]  Heinrich Niemann,et al.  Data Selection for Hand-eye Calibration: A Vector Quantization Approach , 2008, Int. J. Robotics Res..

[4]  Austin Reiter,et al.  Feature Classification for Tracking Articulated Surgical Tools , 2012, MICCAI.

[5]  Radu Horaud,et al.  Robot Hand-Eye Calibration Using Structure-from-Motion , 2001, Int. J. Robotics Res..

[6]  Sébastien Ourselin,et al.  Similarity Registration Problems for 2D/3D Ultrasound Calibration , 2016, ECCV.

[7]  Sébastien Ourselin,et al.  Image Based Surgical Instrument Pose Estimation with Multi-class Labelling and Optical Flow , 2015, MICCAI.

[8]  Brian L. Davies,et al.  Active Constraints/Virtual Fixtures: A Survey , 2014, IEEE Transactions on Robotics.

[9]  Yoon Keun Kwak,et al.  Calibration of geometric and non-geometric errors of an industrial robot , 2001, Robotica.

[10]  Abed Malti,et al.  Robust hand-eye calibration for computer aided medical endoscopy , 2010, 2010 IEEE International Conference on Robotics and Automation.

[11]  Hanqi Zhuang,et al.  Hand/eye calibration for electronic assembly robots , 1997, Proceedings of International Conference on Robotics and Automation.

[12]  Fadi Dornaika,et al.  Simultaneous robot-world and hand-eye calibration , 1998, IEEE Trans. Robotics Autom..

[13]  Heinrich Niemann,et al.  Calibration-Free Hand-Eye Calibration: A Structure-from-Motion Approach , 2005, DAGM-Symposium.

[14]  Richard M. Murray,et al.  A Mathematical Introduction to Robotic Manipulation , 1994 .

[15]  Jack C. K. Chou,et al.  Finding the Position and Orientation of a Sensor on a Robot Manipulator Using Quaternions , 1991, Int. J. Robotics Res..

[16]  Sébastien Ourselin,et al.  2D-3D Pose Tracking of Rigid Instruments in Minimally Invasive Surgery , 2014, IPCAI.

[17]  Jack C. K. Chou,et al.  QUATERNIONS APPROACH TO SOLVE THE KINEMATIC EQUATION OF ROTATION, A,A, = OF A SENSOR-MOUNTED ROBOTIC MANIPULATOR , 1988 .

[18]  P. Schönemann,et al.  A generalized solution of the orthogonal procrustes problem , 1966 .

[19]  Danail Stoyanov,et al.  Surgical Vision , 2011, Annals of Biomedical Engineering.

[20]  Simon J. D. Prince,et al.  Computer Vision: Index , 2012 .

[21]  Simon J. D. Prince,et al.  Computer Vision: Models, Learning, and Inference , 2012 .

[22]  Yuan F. Zheng,et al.  Robotic eye-in-hand calibration by calibrating optical axis and target pattern , 1995, J. Intell. Robotic Syst..

[23]  Gregory D. Hager,et al.  Articulated object tracking by rendering consistent appearance parts , 2009, 2009 IEEE International Conference on Robotics and Automation.

[24]  Yiu Cheung Shiu,et al.  A noise-tolerant algorithm for robotic hand-eye calibration with or without sensor orientation measurement , 1993, IEEE Trans. Syst. Man Cybern..

[25]  Peter Kazanzides,et al.  An open-source research kit for the da Vinci® Surgical System , 2014, 2014 IEEE International Conference on Robotics and Automation (ICRA).

[26]  Danail Stoyanov,et al.  Tissue Surface Reconstruction Aided by Local Normal Information Using a Self-calibrated Endoscopic Structured Light System , 2015, MICCAI.

[27]  Danail Stoyanov,et al.  Robust Real-Time Visual Odometry for Stereo Endoscopy Using Dense Quadrifocal Tracking , 2014, IPCAI.

[28]  Michal Havlena,et al.  Structure-from-motion based hand-eye calibration using L∞ minimization , 2011, CVPR 2011.

[29]  L. Kavan,et al.  Dual Quaternions for Rigid Transformation Blending , 2006 .

[30]  M Hashizume,et al.  Preoperative planning system for surgical robotics setup with kinematics and haptics , 2005, The international journal of medical robotics + computer assisted surgery : MRCAS.

[31]  Guang-Zhong Yang,et al.  Laparoscope Self-calibration for Robotic Assisted Minimally Invasive Surgery , 2005, MICCAI.

[32]  J. Marescaux,et al.  Telerobotic Laparoscopic Cholecystectomy: Initial Clinical Experience With 25 Patients , 2001, Annals of surgery.

[33]  Septimiu E. Salcudean,et al.  Bimanual teleoperation with heart motion compensation on the da Vinci® Research Kit: Implementation and preliminary experiments , 2016, 2016 IEEE International Conference on Robotics and Automation (ICRA).

[34]  Gregory S. Chirikjian,et al.  Simultaneous Hand-Eye and Robot-World Calibration by Solving the $AX=YB$ Problem Without Correspondence , 2016, IEEE Robotics and Automation Letters.

[35]  Sébastien Ourselin,et al.  Toward Detection and Localization of Instruments in Minimally Invasive Surgery , 2013, IEEE Transactions on Biomedical Engineering.

[36]  Kostas Daniilidis,et al.  Hand-Eye Calibration Using Dual Quaternions , 1999, Int. J. Robotics Res..

[37]  Sébastien Ourselin,et al.  Towards Detection and Localisation of Instruments in Minimally Invasive Surgery , 2012 .

[38]  Jose Luis Blanco,et al.  A tutorial on SE(3) transformation parameterizations and on-manifold optimization , 2012 .

[39]  Roger Y. Tsai,et al.  Real time versatile robotics hand/eye calibration using 3D machine vision , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[40]  Sébastien Ourselin,et al.  Spatial calibration of a 2D/3D ultrasound using a tracked needle , 2016, International Journal of Computer Assisted Radiology and Surgery.

[41]  A W Partin,et al.  Comparison of robotic versus human laparoscopic camera control . , 1995, The Journal of urology.

[42]  Ève Coste-Manière,et al.  Flexible Calibration of Actuated Stereoscopic Endoscope for Overlay in Robot Assisted Surgery , 2002, MICCAI.

[43]  Gerd Hirzinger,et al.  Optimal Hand-Eye Calibration , 2006, 2006 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[44]  Roger Y. Tsai,et al.  A new technique for fully autonomous and efficient 3D robotics hand/eye calibration , 1988, IEEE Trans. Robotics Autom..

[45]  M. Kamel,et al.  Quaternions approach to solve the kinematic equation of rotation, A/sub a/A/sub x/=A/sub x/A/sub b/, of a sensor-mounted robotic manipulator , 1988, Proceedings. 1988 IEEE International Conference on Robotics and Automation.

[46]  Zuzana Kukelova,et al.  Hand-Eye Calibration without Hand Orientation Measurement Using Minimal Solution , 2012, ACCV.