Superimposition of eye fundus images for longitudinal analysis from large public health databases

In this paper, a method is presented for superimposition (i.e. registration) of eye fundus images from persons with diabetes screened over many years for diabetic retinopathy. The method is fully automatic and robust to camera changes and colour variations across the images both in space and time. All the stages of the process are designed for longitudinal analysis of cohort public health databases where retinal examinations are made at approximately yearly intervals. The method relies on a model correcting two radial distortions and an affine transformation between pairs of images which is robustly fitted on salient points. Each stage involves linear estimators followed by non-linear optimisation. The model of image warping is also invertible for fast computation. The method has been validated (1) on a simulated montage and (2) on public health databases with 69 patients with high quality images (271 pairs acquired mostly with different types of camera and 268 pairs acquired mostly with the same type of camera) with success rates of 92% and 98%, and five patients (20 pairs) with low quality images with a success rate of 100%. Compared to two state-of-the-art methods, ours gives better results.

[1]  Sotirios A. Tsaftaris,et al.  Medical Image Computing and Computer Assisted Intervention , 2017 .

[2]  J. J. Moré,et al.  Levenberg--Marquardt algorithm: implementation and theory , 1977 .

[3]  Andrew W. Fitzgibbon,et al.  Simultaneous linear estimation of multiple view geometry and lens distortion , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[4]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[5]  Philippe C. Cattin,et al.  Objective and expert-independent validation of retinal image registration algorithms by a projective imaging distortion model , 2010, Medical Image Anal..

[6]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[7]  Nobuhiko Hata,et al.  Medical Image Computing and Computer-Assisted Intervention – MICCAI 2014 , 2014, Lecture Notes in Computer Science.

[8]  Junhee Park,et al.  Lens distortion correction using ideal image coordinates , 2009, IEEE Transactions on Consumer Electronics.

[9]  Max A. Viergever,et al.  A survey of medical image registration - under review , 2016, Medical Image Anal..

[10]  Robyn A. Owens,et al.  Registration of stereo and temporal images of the retina , 1999, IEEE Transactions on Medical Imaging.

[11]  Max A. Viergever,et al.  A survey of medical image registration , 1998, Medical Image Anal..

[12]  Chia-Ling Tsai,et al.  The dual-bootstrap iterative closest point algorithm with application to retinal image registration , 2003, IEEE Transactions on Medical Imaging.

[13]  D. Owens,et al.  Contrast enhancement of eye fundus images , 2015 .

[14]  Gwénolé Quellec,et al.  Exudate detection in color retinal images for mass screening of diabetic retinopathy , 2014, Medical Image Anal..

[15]  Koen A. Vermeer,et al.  A Hierarchical Coarse-to-Fine Approach for Fundus Image Registration , 2014, WBIR.

[16]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[17]  Stephen J. Aldington,et al.  A Practical Manual of Diabetic Retinopathy Management , 2009 .

[18]  Ingrid U Scott,et al.  Single-field fundus photography for diabetic retinopathy screening: a report by the American Academy of Ophthalmology. , 2004, Ophthalmology.

[19]  J. Olson,et al.  Automated grading for diabetic retinopathy: a large-scale audit using arbitration by clinical experts , 2010, British Journal of Ophthalmology.

[20]  R V North,et al.  Incidence of diabetic retinopathy in people with type 2 diabetes mellitus attending the Diabetic Retinopathy Screening Service for Wales: retrospective analysis , 2012, BMJ : British Medical Journal.

[21]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[22]  J.-C. Klein,et al.  A registration algorithm of eye fundus images using a Bayesian Hough transform , 1999 .

[23]  Jie Tian,et al.  A Partial Intensity Invariant Feature Descriptor for Multimodal Retinal Image Registration , 2010, IEEE Transactions on Biomedical Engineering.

[24]  Antonis A. Argyros,et al.  Retinal image registration through simultaneous camera pose and eye shape estimation , 2016, 2016 38th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC).

[25]  Jean Charles Gilbert,et al.  Numerical Optimization: Theoretical and Practical Aspects , 2003 .

[26]  Jorge J. Moré,et al.  Recent Developments in Algorithms and Software for Trust Region Methods , 1982, ISMP.

[27]  Philippe C. Cattin,et al.  Retina Mosaicing Using Local Features , 2006, MICCAI.

[28]  Konstantina S. Nikita,et al.  Automatic retinal image registration scheme using global optimization techniques , 1999, IEEE Transactions on Information Technology in Biomedicine.

[29]  Charles V. Stewart,et al.  A Feature-Based, Robust, Hierarchical Algorithm for Registering Pairs of Images of the Curved Human Retina , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[30]  Gérard G. Medioni,et al.  Retinal image registration from 2D to 3D , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[32]  Joseph M. Reinhardt,et al.  Feature-based pairwise retinal image registration by radial distortion correction , 2007, SPIE Medical Imaging.

[33]  William M. Wells,et al.  Medical Image Computing and Computer-Assisted Intervention — MICCAI’98 , 1998, Lecture Notes in Computer Science.

[34]  V.R.S Mani,et al.  Survey of Medical Image Registration , 2013 .

[35]  Jorge J. Moré,et al.  The Levenberg-Marquardt algo-rithm: Implementation and theory , 1977 .

[36]  Frédéric Zana,et al.  A multimodal registration algorithm of eye fundus images using vessels detection and Hough transform , 1999, IEEE Transactions on Medical Imaging.

[37]  Thomas Walter,et al.  Automatic Analysis of Color Fundus Photographs and Its Application to the Diagnosis of Diabetic Retinopathy , 2005 .

[38]  A. V. Cideciyan,et al.  Registration of ocular fundus images: an algorithm using cross-correlation of triple invariant image descriptors , 1995 .

[39]  Gwénolé Quellec,et al.  Automatic detection of referral patients due to retinal pathologies through data mining , 2016, Medical Image Anal..

[40]  Guoliang Fan,et al.  Hybrid retinal image registration , 2006, IEEE Transactions on Information Technology in Biomedicine.

[41]  Alicia Dickenstein,et al.  Solving Polynomial Equations , 2005 .

[42]  B. Sturmfels.  SOLVING SYSTEMS OF POLYNOMIAL EQUATIONS , 2002 .

[43]  Jamshid Shanbehzadeh,et al.  An efficient approach for robust multimodal retinal image registration based on UR-SIFT features and PIIFD descriptors , 2013, EURASIP J. Image Video Process..

[44]  Wonpil Yu.  An embedded camera lens distortion correction method for mobile computing applications , 2003, IEEE Trans. Consumer Electron..

[45]  Pascale Massin,et al.  Automatic detection of microaneurysms in color fundus images , 2007, Medical Image Anal..

[46]  J. Renaud.  Numerical Optimization, Theoretical and Practical Aspects— , 2006, IEEE Transactions on Automatic Control.

[47]  Yuan Yan Tang,et al.  Elastic registration for retinal images based on reconstructed vascular trees , 2006, IEEE Transactions on Biomedical Engineering.

[48]  Thomas Walter.  Application de la morphologie mathématique au diagnostic de la rétinopathie diabétique à partir d' images couleur , 2003 .

[49]  Zhenyu He,et al.  A Global-to-Local Matching Strategy for Registering Retinal Fundus Images , 2005, IbPRIA.

[50]  R Greenwood,et al.  Grading and disease management in national screening for diabetic retinopathy in England and Wales , 2003, Diabetic medicine : a journal of the British Diabetic Association.

[51]  George Wolberg,et al.  Digital image warping , 1990 .

[52]  Conor Heneghan,et al.  Registration of digital retinal images using landmark correspondence by expectation maximization , 2004, Image Vis. Comput..

[53]  Karel J. Zuiderveld,et al.  Contrast Limited Adaptive Histogram Equalization , 1994, Graphics Gems.

[54]  Chia-Ling Tsai,et al.  Registration of Challenging Image Pairs: Initialization, Estimation, and Decision , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[55]  Joan Serrat,et al.  Retinal image registration using creases as anatomical landmarks , 2000, Proceedings 15th International Conference on Pattern Recognition. ICPR-2000.

[56]  G. Quellec,et al.  Automated analysis of retinal images for detection of referable diabetic retinopathy. , 2013, JAMA ophthalmology.

[57]  Martin Grötschel,et al.  Mathematical Programming The State of the Art, XIth International Symposium on Mathematical Programming, Bonn, Germany, August 23-27, 1982 , 1983, ISMP.

[58]  Matthijs C. Dorst.  Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[59]  Guy Cazuguel,et al.  TeleOphta: Machine learning and image processing methods for teleophthalmology , 2013 .

[60]  Joseph M. Reinhardt,et al.  Retinal image mosaicing using the radial distortion correction model , 2008, SPIE Medical Imaging.

[61]  João Manuel R S Tavares,et al.  Medical image registration: a review , 2014, Computer methods in biomechanics and biomedical engineering.