A Novel Multi-Sensor Image Matching Algorithm Based on Adaptive Multiscale Structure Orientation

Automatic and reliable multi-sensor image matching is a very challenging task due to the significant nonlinear radiometric differences between multi-sensor images. In this paper, a novel dense descriptor based on adaptive multiscale structure orientation is proposed for capturing the geometrical structure information of an image. The dense descriptor of the proposed matching algorithm is not only illumination and contrast invariant but also robust against the image noise. Further, an improved similarity measurement is introduced for adapting the orientation reversal caused by the intensity inversion between multi-sensor images. Based on the robust dense descriptor and the improved similarity measurement, we developed a novel and practical template matching algorithm to match multi-sensor images reliably. We evaluate the proposed matching algorithm by comparing it with other state-of-the-art algorithms. The experimental results show the proposed algorithm has a significant advantage on matching accuracy.

[1]  Jacqueline Le Moigne,et al.  Multiresolution registration of remote sensing imagery by optimization of mutual information using a stochastic gradient , 2003, IEEE Trans. Image Process..

[2]  Peter Kovesi,et al.  Image Features from Phase Congruency , 1995 .

[3]  E. Haber,et al.  Intensity Gradient Based Registration and Fusion of Multi-modal Images , 2007, Methods of Information in Medicine.

[4]  Sabih H. Gerez,et al.  Systematic Methods for the Computation of the Directional Fields and Singular Points of Fingerprints , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[5]  Stephen Lin,et al.  Deep Self-correlation Descriptor for Dense Cross-Modal Correspondence , 2016, ECCV.

[6]  Michael Brady,et al.  Phase mutual information as a similarity measure for registration , 2005, Medical Image Anal..

[7]  Cordelia Schmid,et al.  DeepMatching: Hierarchical Deformable Dense Matching , 2015, International Journal of Computer Vision.

[8]  Roberto Brunelli,et al.  Template Matching Techniques in Computer Vision: Theory and Practice , 2009 .

[9]  Amir Averbuch,et al.  Implicit similarity: a new approach to multi-sensor image registration , 2003, 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2003. Proceedings..

[10]  Lorenzo Bruzzone,et al.  A Theoretical Framework for Change Detection Based on a Compound Multiclass Statistical Model of the Difference Image , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[11]  Alexander Sibiryakov,et al.  Fast and high-performance template matching method , 2011, CVPR 2011.

[12]  Raquel Urtasun,et al.  Efficient Deep Learning for Stereo Matching , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[13]  Yacov Hel-Or,et al.  Fast template matching in non-linear tone-mapped images , 2011, 2011 International Conference on Computer Vision.

[14]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[15]  Philippe Martinet,et al.  Vision-based navigation of unmanned aerial vehicles , 2010 .

[16]  Yong Zhao,et al.  Map2DFusion: Real-time incremental UAV image mosaicing based on monocular SLAM , 2016, 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[17]  Nikos Komodakis,et al.  Simultaneous registration and change detection in multitemporal, very high resolution remote sensing data , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW).

[18]  Qingwu Hu,et al.  Optimal Illumination and Color Consistency for Optical Remote-Sensing Image Mosaicking , 2017, IEEE Geoscience and Remote Sensing Letters.

[19]  Minh N. Do,et al.  DASC: Robust Dense Descriptor for Multi-Modal and Multi-Spectral Correspondence Estimation , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[20]  Jianya Gong,et al.  A Novel Adjustment Model for Mosaicking Low-Overlap Sweeping Images , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Lorenzo Bruzzone,et al.  Earthquake Damage Assessment of Buildings Using VHR Optical and SAR Imagery , 2010, IEEE Transactions on Geoscience and Remote Sensing.

[22]  H. Keshavarz,et al.  Developing a new approach for registering LWIR and MWIR images using local transformation function , 2015, Signal Image Video Process..

[23]  Yue Wu,et al.  Multimodal Remote Sensing Image Registration Based on Image Transfer and Local Features , 2019, IEEE Geoscience and Remote Sensing Letters.

[24]  Vladimir V. Lukin,et al.  Multimodal Remote Sensing Image Registration With Accuracy Estimation at Local and Global Scales , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[25]  Pramod K. Varshney,et al.  Mutual information-based image registration for remote sensing data , 2003 .

[26]  Li Shen,et al.  Robust Optical-to-SAR Image Matching Based on Shape Properties , 2017, IEEE Geoscience and Remote Sensing Letters.

[27]  Yuanxin Ye,et al.  A local descriptor based registration method for multispectral remote sensing images with non-linear intensity differences , 2014 .

[28]  Yang Yang,et al.  Multi-Temporal Remote Sensing Image Registration Using Deep Convolutional Features , 2018, IEEE Access.

[29]  Wanying Xu,et al.  Robust Multisensor Image Matching Using Bayesian Estimated Mutual Information , 2013 .

[30]  Eli Shechtman,et al.  Matching Local Self-Similarities across Images and Videos , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[32]  P. Milanfar,et al.  Multiscale principal components analysis for image local orientation estimation , 2002, Conference Record of the Thirty-Sixth Asilomar Conference on Signals, Systems and Computers, 2002..

[33]  M A Karim,et al.  Optical symbolic substitution: edge detection using Prewitt, Sobel, and Roberts operators. , 1989, Applied optics.

[34]  Andrea Vedaldi,et al.  Fully-trainable deep matching , 2016, BMVC.

[35]  Francesca Bovolo,et al.  A Fast and Robust Matching Framework for Multimodal Remote Sensing Image Registration , 2018, ArXiv.

[36]  Shai Avidan,et al.  FasT-Match: Fast Affine Template Matching , 2013, CVPR.

[37]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[38]  Jeff Orchard.  Globally Optimal Multimodal Rigid Registration: An Analytic Solution using Edge Information , 2007, 2007 IEEE International Conference on Image Processing.

[39]  Lorenzo Bruzzone,et al.  Robust Registration of Multimodal Remote Sensing Images Based on Structural Similarity , 2017, IEEE Transactions on Geoscience and Remote Sensing.

[40]  Jordi Inglada,et al.  On the possibility of automatic multisensor image registration , 2004, IEEE Transactions on Geoscience and Remote Sensing.

[41]  Zheng Liu,et al.  Phase congruence measurement for image similarity assessment , 2007, Pattern Recognit. Lett..

[42]  Paul A. Viola,et al.  Alignment by Maximization of Mutual Information , 1997, International Journal of Computer Vision.

[43]  Amin Sedaghat,et al.  Distinctive Order Based Self-Similarity descriptor for multi-sensor remote sensing image matching , 2015 .

[44]  Alex Kendall,et al.  End-to-End Learning of Geometry and Context for Deep Stereo Regression , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[45]  Stavri G. Nikolov,et al.  Image fusion: Advances in the state of the art , 2007, Inf. Fusion.

[46]  Jerry L. Prince,et al.  Multimodal Registration via Mutual Information Incorporating Geometric and Spatial Context , 2015, IEEE Transactions on Image Processing.

[47]  Y. Ye,et al.  HOPC: A NOVEL SIMILARITY METRIC BASED ON GEOMETRIC STRUCTURAL PROPERTIES FOR MULTI-MODAL REMOTE SENSING IMAGE MATCHING , 2016 .