Robust Feature Matching for Remote Sensing Image Registration via Linear Adaptive Filtering

As a fundamental and critical task in feature-based remote sensing image registration, feature matching refers to establishing reliable point correspondences from two images of the same scene. In this article, we propose a simple yet efficient method termed linear adaptive filtering (LAF) for both rigid and nonrigid feature matching of remote sensing images and apply it to the image registration task. Our algorithm starts with establishing putative feature correspondences based on local descriptors and then focuses on removing outliers using geometrical consistency priori together with filtering and denoising theory. Specifically, we first grid the correspondence space into several nonoverlapping cells and calculate a typical motion vector for each one. Subsequently, we remove false matches by checking the consistency between each putative match and the typical motion vector in the corresponding cell, which is achieved by a Gaussian kernel convolution operation. By refining the typical motion vector in an iterative manner, we further introduce a progressive strategy based on the coarse-to-fine theory to promote the matching accuracy gradually. In addition, an adaptive parameter setting strategy and posterior probability estimation based on the expectation–maximization algorithm enhance the robustness of our method to different data. Most importantly, our method is quite efficient where the gridding strategy enables it to achieve linear time complexity. Consequently, some sparse point-based tasks may inspire from our method when they are achieved by deep learning techniques. Extensive feature matching and image registration experiments on several remote sensing data sets demonstrate the superiority of our approach over the state of the art.

[1]  Hongyuan Zha,et al.  Multi-Graph Matching via Affinity Optimization with Graduated Consistency Regularization , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[2]  Junchi Yan,et al.  Adaptive Discrete Hypergraph Matching , 2018, IEEE Transactions on Cybernetics.

[3]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[4]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[5]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[6]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[7]  Yang Yang,et al.  Image registration using two-layer cascade reciprocal pipeline and context-aware dissimilarity measure , 2020, Neurocomputing.

[8]  Lei Wang,et al.  Progressive Mode-Seeking on Graphs for Sparse Feature Matching , 2014, ECCV.

[9]  Qingwu Hu,et al.  RIFT: Multi-modal Image Matching Based on Radiation-invariant Feature Transform , 2018, ArXiv.

[10]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[11]  Bhu Dev Sharma,et al.  Remote Sensing Image Registration Techniques: A Survey , 2010, ICISP.

[12]  Siamak Khorram,et al.  A feature-based image registration algorithm using improved chain-code representation combined with invariant moments , 1999, IEEE Trans. Geosci. Remote. Sens..

[13]  Qing Ma,et al.  Feature Matching Based on Top K Rank Similarity , 2018, 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[14]  Yang Yang,et al.  Multi-Temporal Remote Sensing Image Registration Using Deep Convolutional Features , 2018, IEEE Access.

[15]  Vincent Lepetit,et al.  LIFT: Learned Invariant Feature Transform , 2016, ECCV.

[16]  Shuang Wang,et al.  A deep learning framework for remote sensing image registration , 2018, ISPRS Journal of Photogrammetry and Remote Sensing.

[17]  Paul Suetens,et al.  Nonrigid Image Registration Using Conditional Mutual Information , 2010, IEEE Transactions on Medical Imaging.

[18]  Yu Zhou,et al.  Mismatch removal via coherent spatial mapping , 2012, 2012 19th IEEE International Conference on Image Processing.

[19]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[20]  Kidiyo Kpalma,et al.  An automatic image registration for applications in remote sensing , 2005, IEEE Transactions on Geoscience and Remote Sensing.

[21]  Shih-Fu Chang,et al.  Learning Spread-Out Local Feature Descriptors , 2017, 2017 IEEE International Conference on Computer Vision (ICCV).

[22]  B. N. Chatterji,et al.  An FFT-based technique for translation, rotation, and scale-invariant image registration , 1996, IEEE Trans. Image Process..

[23]  Zhanyi Hu,et al.  Line matching leveraged by point correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[24]  Jacqueline Le Moigne,et al.  Mutual information as a similarity measure for remote sensing image registration , 2001, SPIE Defense + Commercial Sensing.

[25]  Shuicheng Yan,et al.  Common visual pattern discovery via spatially coherent correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[26]  Qi Tian,et al.  SIFT Meets CNN: A Decade Survey of Instance Retrieval , 2016, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  G. Wahba Spline models for observational data , 1990 .

[28]  Junjun Jiang,et al.  Robust Feature Matching Using Spatial Clustering With Heavy Outliers , 2020, IEEE Transactions on Image Processing.

[29]  Guy Marchal,et al.  Multimodality image registration by maximization of mutual information , 1997, IEEE Transactions on Medical Imaging.

[30]  Minh N. Do,et al.  CODE: Coherence Based Decision Boundaries for Feature Correspondence , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  João Manuel R S Tavares,et al.  Medical image registration: a review , 2014, Computer methods in biomechanics and biomedical engineering.

[32]  Jiayi Ma,et al.  Infrared and visible image fusion methods and applications: A survey , 2018, Inf. Fusion.

[33]  Junjun Jiang,et al.  Multiscale Locality and Rank Preservation for Robust Feature Matching of Remote Sensing Images , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[34]  John F. Canny,et al.  A Computational Approach to Edge Detection , 1986, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Zhuowen Tu,et al.  Robust Estimation of Nonrigid Transformation for Point Set Registration , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[36]  Vincent Lepetit,et al.  Learning to Find Good Correspondences , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[37]  Sim Heng Ong,et al.  Remote Sensing Image Registration Using Multiple Image Features , 2017, Remote. Sens..

[38]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[39]  Bao-Long Guo,et al.  Pseudo-log-polar Fourier transform for image registration , 2006, IEEE Signal Processing Letters.

[40]  T. Higuchi,et al.  High-Accuracy Subpixel Image Registration Based on Phase-Only Correlation , 2003, IEICE Trans. Fundam. Electron. Commun. Comput. Sci..

[41]  Shang-Hong Lai,et al.  Efficient NCC-Based Image Matching in Walsh-Hadamard Domain , 2008, ECCV.

[42]  Mert R. Sabuncu,et al.  An Unsupervised Learning Model for Deformable Medical Image Registration , 2018, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[43]  Hassan Foroosh,et al.  Extension of phase correlation to subpixel registration , 2002, IEEE Trans. Image Process..

[44]  Yasuyuki Matsushita,et al.  GMS: Grid-Based Motion Statistics for Fast, Ultra-robust Feature Correspondence , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[45]  Weidong Min,et al.  Remote Sensing Image Registration Using Convolutional Neural Network Features , 2018, IEEE Geoscience and Remote Sensing Letters.

[46]  Z. Jane Wang,et al.  A CNN Regression Approach for Real-Time 2D/3D Registration , 2016, IEEE Transactions on Medical Imaging.

[47]  Jan Flusser,et al.  Image registration methods: a survey , 2003, Image Vis. Comput..

[48]  Quan Z. Sheng,et al.  Nonrigid Point Set Registration With Robust Transformation Learning Under Manifold Regularization , 2019, IEEE Transactions on Neural Networks and Learning Systems.

[49]  Jun Wang,et al.  Consistency-Driven Alternating Optimization for Multigraph Matching: A Unified Approach , 2015, IEEE Transactions on Image Processing.

[50]  Nikos Komodakis,et al.  A Deep Metric for Multimodal Registration , 2016, MICCAI.

[51]  Jubai An,et al.  A Simple and Robust Feature Point Matching Algorithm Based on Restricted Spatial Order Constraints for Aerial Image Registration , 2012, IEEE Transactions on Geoscience and Remote Sensing.

[52]  Julie Delon,et al.  SAR-SIFT: A SIFT-Like Algorithm for SAR Images , 2015, IEEE Trans. Geosci. Remote. Sens..

[53]  Qingwu Hu,et al.  Robust feature matching via support-line voting and affine-invariant ratios , 2017 .

[54]  Zhanyi Hu,et al.  Rejecting Mismatches by Correspondence Function , 2010, International Journal of Computer Vision.

[55]  Nikos Paragios,et al.  Deformable Medical Image Registration: A Survey , 2013, IEEE Transactions on Medical Imaging.

[56]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[57]  David Fofi,et al.  A review of recent range image registration methods with accuracy evaluation , 2007, Image Vis. Comput..

[58]  Zhang Li,et al.  Image Registration Based on Autocorrelation of Local Structure , 2016, IEEE Transactions on Medical Imaging.

[59]  Amin Sedaghat,et al.  Remote Sensing Image Matching Based on Adaptive Binning SIFT Descriptor , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[60]  Jiayi Ma,et al.  Progressive Filtering for Feature Matching , 2019, ICASSP 2019 - 2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[61]  Badrinath Roysam,et al.  Image change detection algorithms: a systematic survey , 2005, IEEE Transactions on Image Processing.

[62]  Gong-Jian Wen,et al.  A High-Performance Feature-Matching Method for Image Registration by Combining Spatial and Similarity Information , 2008, IEEE Transactions on Geoscience and Remote Sensing.

[63]  Junjun Jiang,et al.  Locality Preserving Matching , 2017, IJCAI.

[64]  Torsten Sattler,et al.  Comparative Evaluation of Hand-Crafted and Learned Local Features , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[65]  Yue Wu,et al.  A Novel Two-Step Registration Method for Remote Sensing Images Based on Deep and Local Features , 2019, IEEE Transactions on Geoscience and Remote Sensing.

[66]  Radford M. Neal Pattern Recognition and Machine Learning , 2007, Technometrics.

[67]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[68]  Maria Petrou,et al.  Image registration using the Walsh transform , 2006, IEEE Transactions on Image Processing.

[69]  Gary R. Bradski,et al.  ORB: An efficient alternative to SIFT or SURF , 2011, 2011 International Conference on Computer Vision.

[70]  Bram van Ginneken,et al.  A survey on deep learning in medical image analysis , 2017, Medical Image Anal..

[71]  Pramod K. Varshney,et al.  Performance of mutual information similarity measure for registration of multitemporal remote sensing images , 2003, IEEE Trans. Geosci. Remote. Sens..

[72]  Junjun Jiang,et al.  LMR: Learning a Two-Class Classifier for Mismatch Removal , 2019, IEEE Transactions on Image Processing.

[73]  Josien P. W. Pluim,et al.  Evaluation of Optimization Methods for Nonrigid Medical Image Registration Using Mutual Information and B-Splines , 2007, IEEE Transactions on Image Processing.

[74]  Marc Niethammer,et al.  Quicksilver: Fast predictive image registration – A deep learning approach , 2017, NeuroImage.

[75]  Jacqueline Le Moigne,et al.  An automated parallel image registration technique based on the correlation of wavelet features , 2013, IEEE Trans. Geosci. Remote. Sens..

[76]  Jan-Michael Frahm,et al.  Structure-from-Motion Revisited , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[77]  Andrea Vedaldi,et al.  Learning Covariant Feature Detectors , 2016, ECCV Workshops.

[78]  Markus Vincze,et al.  Guided Matching Based on Statistical Optical Flow for Fast and Robust Correspondence Analysis , 2016, ECCV.

[79]  Rahul Sukthankar,et al.  MatchNet: Unifying feature and metric learning for patch-based matching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[80]  Ming-Hsuan Yang,et al.  Object Tracking Benchmark , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[81]  Nicholas J. Tate,et al.  A critical synthesis of remotely sensed optical image change detection techniques , 2015 .

[82]  Michel Defrise,et al.  Symmetric Phase-Only Matched Filtering of Fourier-Mellin Transforms for Image Registration and Recognition , 1994, IEEE Trans. Pattern Anal. Mach. Intell..

[83]  Michael T. Orchard,et al.  A fast direct Fourier-based algorithm for subpixel registration of images , 2001, IEEE Trans. Geosci. Remote. Sens..

[84]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[85]  Lisa M. Brown,et al.  A survey of image registration techniques , 1992, CSUR.

[86]  Tao Liu,et al.  Deep learning-based fusion of Landsat-8 and Sentinel-2 images for a harmonized surface reflectance product , 2019 .

[87]  Junjun Jiang,et al.  Guided Locality Preserving Feature Matching for Remote Sensing Image Registration , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[88]  Hongyuan Zha,et al.  A Short Survey of Recent Advances in Graph Matching , 2016, ICMR.

[89]  Luc Van Gool,et al.  Speeded-Up Robust Features (SURF) , 2008, Comput. Vis. Image Underst..