Multimodal Registration of Multiple Retinal Images Based on Line Structures

We propose a framework to perform multimodal registration of multiple images. In retinal imaging, this alignment enables the physician to correlate the features across modalities, which can help formulate a diagnosis. The images appear very different and there are few reliable modality-invariant features. We base our registration on the salient line structures extracted with a tensor-voting approach and aligned to minimize the Chamfer distance. For every pair of images, we match the line junctions and extremities to get a candidate transformation that is further refined with an Iterative Closest Point approach. We use a global chained registration framework to recover from failed registration and we account for non-planarities with a Thin-Plate Splines deformation. Our approach can handle large variations across modalities and is evaluated on real-world retinal images with 5 modalities per eye. We achieve an average error of 52 μm on our dataset.

[1]  Xing Zhang,et al.  Retinal Fundus Image Registration via Vascular Structure Graph Matching , 2010, Int. J. Biomed. Imaging.

[2]  Jinzhong Yang,et al.  The thin plate spline robust point matching (TPS-RPM) algorithm: A revisit , 2011, Pattern Recognit. Lett..

[3]  Max A. Viergever,et al.  Mutual-information-based registration of medical images: a survey , 2003, IEEE Transactions on Medical Imaging.

[4]  Honglie Shen,et al.  Registration of SD-OCT en-face images with color fundus photographs based on local patch matching , 2014 .

[5]  Gérard G. Medioni,et al.  Mutual information computation and maximization using GPU , 2008, 2008 IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops.

[6]  Chia-Ling Tsai,et al.  Registration of Challenging Image Pairs: Initialization, Estimation, and Decision , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[7]  Rajagopalan Srinivasan,et al.  A Fast Mutual Information Method for Multi-modal Registration , 1999, IPMI.

[8]  Chia-Ling Tsai,et al.  The dual-bootstrap iterative closest point algorithm with application to retinal image registration , 2003, IEEE Transactions on Medical Imaging.

[9]  Xiaodong Wu,et al.  Automated multilayer segmentation and characterization in 3D spectral-domain optical coherence tomography images , 2013, Photonics West - Biomedical Optics.

[10]  Andrew Blake,et al.  Multiscale Categorical Object Recognition Using Contour Fragments , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[11]  J. Kruskal.  On the shortest spanning subtree of a graph and the traveling salesman problem , 1956 .

[12]  Tae Eun Choe,et al.  Registration of multimodal fluorescein images sequence of the retina , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[13]  Robert C. Bolles,et al.  Parametric Correspondence and Chamfer Matching: Two New Techniques for Image Matching , 1977, IJCAI.

[14]  Nanning Zheng,et al.  Scaling iterative closest point algorithm for registration of m-D point sets , 2010, J. Vis. Commun. Image Represent..

[15]  Jayanthi Sivaswamy,et al.  Robust matching of multi-modal retinal images using radon transform based local descriptor , 2010, IHI.

[16]  Bahram Parvin,et al.  Iterative Tensor Voting for Perceptual Grouping of Ill-Defined Curvilinear Structures , 2011, IEEE Transactions on Medical Imaging.

[17]  Giovanni Gregori,et al.  Registration of OCT fundus images with color fundus photographs based on blood vessel ridges , 2010, Optics express.

[18]  Jamshid Shanbehzadeh,et al.  An efficient approach for robust multimodal retinal image registration based on UR-SIFT features and PIIFD descriptors , 2013, EURASIP J. Image Video Process..

[19]  Gérard G. Medioni,et al.  Retinal image registration from 2D to 3D , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Gérard G. Medioni,et al.  Object modelling by registration of multiple range images , 1992, Image Vis. Comput..

[21]  Gérard G. Medioni,et al.  Inferring global perceptual contours from local features , 1993, Proceedings of IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[23]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[24]  Gérard G. Medioni,et al.  A Closed-Form Solution to Tensor Voting: Theory and Applications , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Jie Tian,et al.  A Partial Intensity Invariant Feature Descriptor for Multimodal Retinal Image Registration , 2010, IEEE Transactions on Biomedical Engineering.