Feature Matching for Remote Sensing Image Registration via Manifold Regularization

Feature matching is critical in analyzing remote sensing images, aiming to find the optimal mapping between correspondences. Regularization technology is essential to ensure the well-posedness of feature matching. However, current regularization-based methods scarcely consider the geometry structure of the image, which is beneficial for estimating the mapping, especially when the image pairs have a large view or scale change and local distortion. In this article, we introduce manifold regularization to overcome this limit and formulate feature matching as a unified semisupervised latent variable mixture model for both rigid and nonrigid transformations. Especially, we apply a Bayesian model with latent variables indicating whether matches in the putative correspondences are outliers or inliers. Moreover, we employ all the feature points, only part of which have correct matches, to express the intrinsic structure, which is preserved by manifold regularization. Finally, we combine manifold regularization with three different transformation models (e.g., rigid, affine, and thin-plate spline) to estimate the corresponding mappings. Experimental results on four remote sensing image datasets demonstrate that our method can significantly outperform the state of the art.

[1]  Liangpei Zhang,et al.  Robust Registration by Rank Minimization for Multiangle Hyper/Multispectral Remotely Sensed Imagery , 2014, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[2]  Guoyou Wang,et al.  Robust Scale-Invariant Feature Matching for Remote Sensing Image Registration , 2009, IEEE Geoscience and Remote Sensing Letters.

[3]  Quan Z. Sheng,et al.  Nonrigid Point Set Registration With Robust Transformation Learning Under Manifold Regularization , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[4]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[5]  Sim Heng Ong,et al.  Remote Sensing Image Registration Using Multiple Image Features , 2017, Remote. Sens..

[6]  Wei Liu,et al.  SIRF: Simultaneous Satellite Image Registration and Fusion in a Unified Framework , 2015, IEEE Transactions on Image Processing.

[7]  James S. Duncan,et al.  Rigid point feature registration using mutual information , 1999, Medical Image Anal..

[8]  D Marr,et al.  Theory of edge detection , 1979, Proceedings of the Royal Society of London. Series B. Biological Sciences.

[9]  Yansheng Li,et al.  Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration , 2017, Inf. Sci..

[10]  Riqing Chen,et al.  Non-rigid point set registration via global and local constraints , 2018, Multimedia Tools and Applications.

[11]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[12]  T. Eck,et al.  Accuracy assessments of aerosol optical properties retrieved from Aerosol Robotic Network (AERONET) Sun and sky radiance measurements , 2000 .

[13]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[14]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[15]  Manchun Li,et al.  Automated High-Resolution Satellite Image Registration Using Supraglacial Rivers on the Greenland Ice Sheet , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[16]  Jan Flusser,et al.  A moment-based approach to registration of images with affine geometric distortion , 1994, IEEE Trans. Geosci. Remote. Sens..

[17]  Md Jan Nordin,et al.  Invariant Feature Matching for Image Registration Application Based on New Dissimilarity of Spatial Features , 2016, PloS one.

[18]  A. Ardeshir Goshtasby Image Registration: Principles, Tools and Methods , 2012 .

[19]  Hans P. Moravec Rover Visual Obstacle Avoidance , 1981, IJCAI.

[20]  Zhanyi Hu,et al.  Rejecting Mismatches by Correspondence Function , 2010, International Journal of Computer Vision.

[21]  Yan Wu,et al.  SAR Image Registration Using Multiscale Image Patch Features With Sparse Representation , 2017, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[22]  K. Miller Least Squares Methods for Ill-Posed Problems with a Prescribed Bound , 1970 .

[23]  Seyed Mostafa Mousavi Kahaki,et al.  Accurate registration of in vivo time-lapse images , 2019, Medical Imaging: Image Processing.

[24]  Yuan C. Hsieh,et al.  Performance Evaluation of Scene Registration and Stereo Matching for Artographic Feature Extraction , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[25]  Ji Zhao,et al.  Nonrigid Image Deformation Using Moving Regularized Least Squares , 2013, IEEE Signal Processing Letters.

[26]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[27]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[28]  Wenyu Liu,et al.  A Unified Framework for Multioriented Text Detection and Recognition , 2014, IEEE Transactions on Image Processing.

[29]  Tao Lu,et al.  Image Deformation With Vector-Field Interpolation Based on MRLS-TPS , 2018, IEEE Access.

[30]  Zhenghong Yu,et al.  Feature guided non-rigid image/surface deformation via moving least squares with manifold regularization , 2017, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[31]  Michael D. King,et al.  A flexible inversion algorithm for retrieval of aerosol optical properties from Sun and sky radiance measurements , 2000 .

[32]  Junjun Jiang,et al.  LMR: Learning a Two-Class Classifier for Mismatch Removal , 2019, IEEE Transactions on Image Processing.

[33]  Gang Wang,et al.  Removing mismatches for retinal image registration via multi-attribute-driven regularized mixture model , 2016, Inf. Sci..

[34]  Yonghuai Liu,et al.  Automatic Range Image Registration in the Markov Chain , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Junjun Jiang,et al.  Image Matching from Handcrafted to Deep Features: A Survey , 2020, International Journal of Computer Vision.

[36]  Christopher G. Harris,et al.  A Combined Corner and Edge Detector , 1988, Alvey Vision Conference.

[37]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[38]  B. N. Chatterji,et al.  An FFT-based technique for translation, rotation, and scale-invariant image registration , 1996, IEEE Trans. Image Process..

[39]  Jacqueline Le Moigne,et al.  An automated parallel image registration technique based on the correlation of wavelet features , 2013, IEEE Trans. Geosci. Remote. Sens..

[40]  Junjun Jiang,et al.  Locality Preserving Matching , 2017, IJCAI.

[41]  Jiri Matas,et al.  Locally Optimized RANSAC , 2003, DAGM-Symposium.

[42]  Jiayi Ma,et al.  Cross-Weather Image Alignment via Latent Generative Model With Intensity Consistency , 2020, IEEE Transactions on Image Processing.

[43]  Alan L. Yuille,et al.  Semi-Supervised Sparse Representation Based Classification for Face Recognition With Insufficient Labeled Samples , 2016, IEEE Transactions on Image Processing.

[44]  Edwin R. Hancock,et al.  Multiple line-template matching with the EM algorithm , 1997, Pattern Recognit. Lett..

[45]  Scott Schaefer,et al.  Image deformation using moving least squares , 2006, ACM Trans. Graph..

[46]  Nanning Zheng,et al.  Exploiting local linear geometric structure for identifying correct matches , 2014, Comput. Vis. Image Underst..

[47]  Shuicheng Yan,et al.  Common visual pattern discovery via spatially coherent correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[48]  Mikhail Belkin,et al.  Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples , 2006, J. Mach. Learn. Res..

[49]  Haslina Arshad,et al.  Geometric feature descriptor and dissimilarity-based registration of remotely sensed imagery , 2018, PloS one.

[50]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[51]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[52]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.