Real-time retinal vessel mapping and localization for intraocular surgery

Computer-aided intraocular surgery requires precise, real-time knowledge of the vasculature during retinal procedures such as laser photocoagulation or vessel cannulation. Because vitreoretinal surgeons manipulate retinal structures on the back of the eye through ports in the sclera, voluntary and involuntary tool motion rotates the eye in the socket and causes movement to the microscope view of the retina. The dynamic nature of the surgical workspace during intraocular surgery makes mapping, tracking, and localizing vasculature in real time a challenge. We present an approach that both maps and localizes retinal vessels by temporally fusing and registering individual-frame vessel detections. On video of porcine and human retina, we demonstrate real-time performance, rapid convergence, and robustness to variable illumination and tool occlusion.

[1]  Louis A. Lobes,et al.  Vision-Based Control of a Handheld Surgical Micromanipulator With Virtual Fixtures , 2013, IEEE Transactions on Robotics.

[2]  Victor M. Brea,et al.  FPGA-accelerated retinal vessel-tree extraction , 2009, 2009 International Conference on Field Programmable Logic and Applications.

[3]  Jianxin Shen,et al.  Automatic fundus images mosaic based on SIFT feature , 2010, 2010 3rd International Congress on Image and Signal Processing.

[4]  Max A. Viergever,et al.  Ridge-based vessel segmentation in color images of the retina , 2004, IEEE Transactions on Medical Imaging.

[5]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[6]  Sungwook Yang,et al.  Towards vision-based control of a handheld micromanipulator for retinal cannulation in an eyeball phantom , 2012, 2012 4th IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics (BioRob).

[7]  Roberto Marcondes Cesar Junior,et al.  Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification , 2005, IEEE Transactions on Medical Imaging.

[8]  JiangXiaoyi,et al.  Adaptive Local Thresholding by Verification-Based Multithreshold Probing with Application to Vessel Detection in Retinal Images , 2003 .

[9]  Russell H. Taylor,et al.  New steady-hand Eye Robot with micro-force sensing for vitreoretinal surgery , 2010, 2010 3rd IEEE RAS & EMBS International Conference on Biomedical Robotics and Biomechatronics.

[10]  Xiaoyi Jiang,et al.  Adaptive Local Thresholding by Verification-Based Multithreshold Probing with Application to Vessel Detection in Retinal Images , 2003, IEEE Trans. Pattern Anal. Mach. Intell..

[11]  Emanuele Trucco,et al.  FABC: Retinal Vessel Segmentation Using AdaBoost , 2010, IEEE Transactions on Information Technology in Biomedicine.

[12]  C Aras,et al.  Surgery for full-thickness macular holes. , 2001, Ophthalmology.

[13]  H. Brooks,et al.  Macular hole surgery with and without internal limiting membrane peeling. , 2000, Ophthalmology.

[14]  Jaspreet Kaur,et al.  An Efficient Blood Vessel Detection Algorithm For Retinal Images Using Local Entropy Thresholding , 2012 .

[15]  T. Sano,et al.  [Diabetic retinopathy]. , 2001, Nihon rinsho. Japanese journal of clinical medicine.

[16]  Chia-Ling Tsai,et al.  The dual-bootstrap iterative closest point algorithm with application to retinal image registration , 2003, IEEE Transactions on Medical Imaging.

[17]  J. Weiss,et al.  Injection of tissue plasminogen activator into a branch retinal vein in eyes with central retinal vein occlusion. , 2001, Ophthalmology.

[18]  Wolfram Burgard,et al.  An efficient fastSLAM algorithm for generating maps of large-scale cyclic environments from raw laser range measurements , 2003, Proceedings 2003 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS 2003) (Cat. No.03CH37453).

[19]  Tsu-Chin Tsao,et al.  A Prototype Surgical Manipulator for Robotic Intraocular Micro Surgery , 2009, MMVR.

[20]  Sebastian Thrun,et al.  FastSLAM: a factored solution to the simultaneous localization and mapping problem , 2002, AAAI/IAAI.

[21]  Elisa Ricci,et al.  Cellular Neural Networks With Virtual Template Expansion for Retinal Vessel Segmentation , 2007, IEEE Transactions on Circuits and Systems II: Express Briefs.

[22]  Hong Shen,et al.  Rapid automated tracing and feature extraction from retinal fundus images using direct exploratory algorithms , 1999, IEEE Transactions on Information Technology in Biomedicine.

[23]  Christopher Hunt,et al.  Notes on the OpenSURF Library , 2009 .

[24]  L. Bynoe,et al.  RETINAL ENDOVASCULAR SURGERY FOR CENTRAL RETINAL VEIN OCCLUSION: Initial Experience of Four Surgeons , 2004, Retina.

[25]  J-P Hubschman,et al.  ‘The Microhand’: a new concept of micro-forceps for ocular robotic surgery , 2010, Eye.

[26]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[27]  H Das,et al.  Evaluation of a telerobotic system to assist surgeons in microsurgery. , 1999, Computer aided surgery : official journal of the International Society for Computer Aided Surgery.

[28]  Guoliang Fan,et al.  Hybrid retinal image registration , 2006, IEEE Transactions on Information Technology in Biomedicine.

[29]  Gregory D. Hager,et al.  Vision-assisted control for manipulation using virtual fixtures , 2001, IEEE Transactions on Robotics.

[30]  Jie Tian,et al.  A Partial Intensity Invariant Feature Descriptor for Multimodal Retinal Image Registration , 2010, IEEE Transactions on Biomedical Engineering.

[31]  P. Bankhead,et al.  Fast Retinal Vessel Detection and Measurement Using Wavelets and Edge Location Refinement , 2012, PloS one.

[32]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[33]  Martina Broehan,et al.  Real-Time Multimodal Retinal Image Registration for a Computer-Assisted Laser Photocoagulation System , 2011, IEEE Transactions on Biomedical Engineering.

[34]  Charles V. Stewart,et al.  Retinal Vessel Centerline Extraction Using Multiscale Matched Filters, Confidence and Edge Measures , 2006, IEEE Transactions on Medical Imaging.

[35]  Berthold K. P. Horn,et al.  Closed-form solution of absolute orientation using unit quaternions , 1987 .

[36]  Akio Morita,et al.  Robot-assisted vitreoretinal surgery: development of a prototype and feasibility studies in an animal model. , 2009, Ophthalmology.

[37]  Peter Kazanzides,et al.  Portability and applicability of virtual fixtures across medical and manufacturing tasks , 2006, Proceedings 2006 IEEE International Conference on Robotics and Automation, 2006. ICRA 2006..

[38]  M. Goldbaum,et al.  Detection of blood vessels in retinal images using two-dimensional matched filters. , 1989, IEEE transactions on medical imaging.

[39]  R. Frank,et al.  Retinal laser photocoagulation: Benefits and risks , 1980, Vision Research.

[40]  Wei Wei,et al.  Performance Evaluation for Multi-arm Manipulation of Hollow Suspended Organs , 2009, IEEE Transactions on Robotics.

[41]  Cameron N. Riviere,et al.  Physiological tremor amplitude during retinal microsurgery , 2002, Proceedings of the IEEE 28th Annual Northeast Bioengineering Conference (IEEE Cat. No.02CH37342).

[42]  A Chignell,et al.  Manual of Retinal Surgery , 1991 .

[43]  R. T. Smith,et al.  A novel registration method for retinal images based on local features , 2008, 2008 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society.

[44]  Martin Culjat,et al.  FEASIBILITY STUDY OF INTRAOCULAR ROBOTIC SURGERY WITH THE da VINCI SURGICAL SYSTEM , 2008, Retina.

[45]  Gregory D. Hager,et al.  Retinal vessel cannulation with an image-guided handheld robot , 2010, 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology.

[46]  Salvador Pané,et al.  Toward targeted retinal drug delivery with wireless magnetic microrobots , 2008, 2008 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[47]  B. Roysam,et al.  Image processing algorithms for retinal montage synthesis, mapping, and real-time location determination , 1998, IEEE Transactions on Biomedical Engineering.

[48]  J Castro Navarro,et al.  [Macular hole surgery with and without internal limiting membrane peeling]. , 2003, Archivos de la Sociedad Espanola de Oftalmologia.

[49]  Louis B. Rosenberg,et al.  Virtual fixtures: Perceptual tools for telerobotic manipulation , 1993, Proceedings of IEEE Virtual Reality Annual International Symposium.

[50]  Cameron N. Riviere,et al.  Micron: An Actively Stabilized Handheld Tool for Microsurgery , 2012, IEEE Transactions on Robotics.

[51]  Manuel G. Penedo,et al.  Fast retinal vessel tree extraction: A pixel parallel approach , 2008, Int. J. Circuit Theory Appl..

[52]  Philippe C. Cattin,et al.  Retina Mosaicing Using Local Features , 2006, MICCAI.

[53]  Charles V. Stewart,et al.  A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina , 2002, IEEE Trans. Pattern Anal. Mach. Intell..