Novel Registration and Fusion Algorithm for Multimodal Railway Images with Different Field of Views

Objects intruding high-speed railway clearance do great threat to running trains. In order to improve accuracy of railway intrusion detection, an automatic multimodal registration and fusion algorithm for infrared and visible images with different field of views is presented. The ratio of the nearest to next nearest distance, geometric, similar triangle, and RANSAC constraints are used to refine the matching SURF feature points successively. Correct matching points are accumulated with multiframe to overcome the insufficient matching points in single image pair. After being registered, an improved Contourlet transform fusion algorithm combined with total variation and local region energy is proposed. Inverse Contourlet transform to low frequency subband coefficient fused with total variation model and high frequency subband coefficients fused with local region energy is used to reconstruct the fused image. The comparison to other 4 popular fusion methods shows that our algorithm has the best comprehensive performance for multimodal railway image fusion.

[1]  Jun Chen,et al.  Infrared and visible image fusion using total variation model , 2016, Neurocomputing.

[2]  Huchuan Lu,et al.  Robust Object Tracking via Sparse Collaborative Appearance Model , 2014, IEEE Transactions on Image Processing.

[3]  Jerry L. Prince,et al.  Multimodal Registration via Mutual Information Incorporating Geometric and Spatial Context , 2015, IEEE Transactions on Image Processing.

[4]  Maoguo Gong,et al.  A Novel Coarse-to-Fine Scheme for Automatic Image Registration Based on SIFT and Mutual Information , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Ravindra Dhuli,et al.  Two-scale image fusion of visible and infrared images using saliency detection , 2016 .

[6]  Yun Zhang,et al.  A critical review of image registration methods , 2010 .

[7]  Jiayi Ma,et al.  Infrared and visible image fusion via gradient transfer and total variation minimization , 2016, Inf. Fusion.

[8]  Adrien Bartoli,et al.  KAZE Features , 2012, ECCV.

[9]  Zhizhong Fu,et al.  Infrared and visible images fusion based on RPCA and NSCT , 2016 .

[10]  Yang Song,et al.  Fusion of infrared and visible images based on nonsubsampled contourlet transform and sparse K-SVD dictionary learning , 2017 .

[11]  Fanjie Meng,et al.  Image fusion based on object region detection and Non-Subsampled Contourlet Transform , 2017, Comput. Electr. Eng..

[12]  Guillaume-Alexandre Bilodeau,et al.  Visible and infrared image registration using trajectories and composite foreground images , 2011, Image Vis. Comput..

[13]  Wei Huang,et al.  An improved fusion algorithm for infrared and visible images based on multi-scale transform , 2016 .

[14]  Xin Yang,et al.  Local Difference Binary for Ultrafast and Distinctive Feature Description , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[15]  Mingliang Xu,et al.  High quality multi-spectral and panchromatic image fusion technologies based on Curvelet transform , 2015, Neurocomputing.

[16]  Ji Zhao,et al.  Non-rigid visible and infrared face registration via regularized Gaussian fields criterion , 2015, Pattern Recognit..

[17]  Shuigeng Zhou,et al.  A Novel Image Registration Algorithm for Remote Sensing Under Affine Transformation , 2014, IEEE Transactions on Geoscience and Remote Sensing.

[18]  Xinkai Wu,et al.  Car Detection from Low-Altitude UAV Imagery with the Faster R-CNN , 2017 .

[19]  Lei Yan,et al.  Image Fusion Algorithm Based on Contourlet Transform and PCNN for Detecting Obstacles in Forests , 2015 .

[20]  Qi Li,et al.  Fusion of visible and infrared images using saliency analysis and detail preserving based image decomposition , 2013 .

[21]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[22]  Yafei Zhang,et al.  Infrared and visible image fusion scheme based on NSCT and low-level visual features , 2016 .

[23]  Nasir M. Rajpoot,et al.  Registration of thermal and visible light images of diseased plants using silhouette extraction in the wavelet domain , 2015, Pattern Recognit..

[24]  Hua Zong,et al.  Infrared and visible image fusion based on visual saliency map and weighted least square optimization , 2017 .

[25]  Lei Han,et al.  An object tracking method based on guided filter for night fusion image , 2016 .

[26]  Magdy A. Bayoumi,et al.  MIRF: A Multimodal Image Registration and Fusion Module Based on DT-CWT , 2013, J. Signal Process. Syst..

[27]  Jungong Han,et al.  Visible and infrared image registration in man-made environments employing hybrid visual features , 2013, Pattern Recognit. Lett..

[28]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..

[29]  Yihua Tan,et al.  Unsupervised Multilayer Feature Learning for Satellite Image Scene Classification , 2016, IEEE Geoscience and Remote Sensing Letters.