Data-Driven Visual Tracking in Retinal Microsurgery

In the context of retinal microsurgery, visual tracking of instruments is a key component of robotics assistance. The difficulty of the task and major reason why most existing strategies fail on in-vivo image sequences lies in the fact that complex and severe changes in instrument appearance are challenging to model. This paper introduces a novel approach, that is both data-driven and complementary to existing tracking techniques. In particular, we show how to learn and integrate an accurate detector with a simple gradient-based tracker within a robust pipeline which runs at framerate. In addition, we present a fully annotated dataset of retinal instruments in in-vivo surgeries, which we use to quantitatively validate our approach. We also demonstrate an application of our method in a laparascopy image sequence.

[1]  Gregory D. Hager,et al.  Articulated object tracking by rendering consistent appearance parts , 2009, 2009 IEEE International Conference on Robotics and Automation.

[2]  Pascal Fua,et al.  A Real-Time Deformable Detector , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Mark Jenkinson,et al.  Non-local Shape Descriptor: A New Similarity Metric for Deformable Multi-modal Registration , 2011, MICCAI.

[4]  Darius Burschka,et al.  Navigating inner space: 3-D assistance for minimally invasive surgery , 2005, Robotics Auton. Syst..

[5]  Russell H. Taylor,et al.  Single Fiber Optical Coherence Tomography Microsurgical Instruments for Computer and Robot-Assisted Retinal Surgery , 2009, MICCAI.

[6]  Philippe Cinquin,et al.  Automatic Detection of Instruments in Laparoscopic Images: A First Step Towards High-level Command of Robotic Endoscopic Holders , 2007, The First IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics, 2006. BioRob 2006..

[7]  Russell H. Taylor,et al.  Visual Tracking of Surgical Tools for Proximity Detection in Retinal Surgery , 2011, IPCAI.

[8]  Christopher J. Taylor,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2009 , 2009, Lecture Notes in Computer Science.

[9]  Mark R. Pickering,et al.  A new multi-modal similarity measure for fast gradient-based 2D-3D image registration , 2009, 2009 Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[10]  Russell H. Taylor,et al.  New steady-hand Eye Robot with micro-force sensing for vitreoretinal surgery , 2010, 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[11]  Selim Benhimane,et al.  Homography-based 2D Visual Tracking and Servoing , 2007, Int. J. Robotics Res..

[12]  Russell H. Taylor,et al.  Unified Detection and Tracking in Retinal Microsurgery , 2011, MICCAI.

[13]  Russell H. Taylor,et al.  Information Processing in Computer-Assisted Interventions - Second International Conference, IPCAI 2011, Berlin, Germany, June 22, 2011. Proceedings , 2011, IPCAI.