Locality Preserving Matching

Seeking reliable correspondences between two feature sets is a fundamental and important task in computer vision. This paper attempts to remove mismatches from given putative image feature correspondences. To achieve the goal, an efficient approach, termed as locality preserving matching (LPM), is designed, the principle of which is to maintain the local neighborhood structures of those potential true matches. We formulate the problem into a mathematical model, and derive a closed-form solution with linearithmic time and linear space complexities. Our method can accomplish the mismatch removal from thousands of putative correspondences in only a few milliseconds. To demonstrate the generality of our strategy for handling image matching problems, extensive experiments on various real image pairs for general feature matching, as well as for point set registration, visual homing and near-duplicate image retrieval are conducted. Compared with other state-of-the-art alternatives, our LPM achieves better or favorably competitive performance in accuracy while intensively cutting time cost by more than two orders of magnitude.

[1]  Kamil Adamczewski,et al.  Discrete Tabu Search for Graph Matching , 2015, 2015 IEEE International Conference on Computer Vision (ICCV).

[2]  Ralph R. Martin,et al.  Regularization Based Iterative Point Match Weighting for Accurate Rigid Transformation Estimation , 2015, IEEE Transactions on Visualization and Computer Graphics.

[3]  Peter J. Huber,et al.  Robust Statistics , 2005, Wiley Series in Probability and Statistics.

[4]  Sim Heng Ong,et al.  A robust global and local mixture distance based non-rigid point set registration , 2015, Pattern Recognit..

[5]  Baba C. Vemuri,et al.  Robust Point Set Registration Using Gaussian Mixture Models , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[6]  Junjun Jiang,et al.  Robust Feature Matching for Remote Sensing Image Registration via Locally Linear Transforming , 2015, IEEE Transactions on Geoscience and Remote Sensing.

[7]  Gang Wang,et al.  Removing mismatches for retinal image registration via multi-attribute-driven regularized mixture model , 2016, Inf. Sci..

[8]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[9]  Andrew Zisserman,et al.  MLESAC: A New Robust Estimator with Application to Estimating Image Geometry , 2000, Comput. Vis. Image Underst..

[10]  Sim Heng Ong,et al.  Remote Sensing Image Registration Using Multiple Image Features , 2017, Remote. Sens..

[11]  Yong Wang,et al.  Image retrieval based on image-to-class similarity , 2016, Pattern Recognit. Lett..

[12]  Minsu Cho,et al.  Mode-seeking on graphs via random walks , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[13]  Gang Wang,et al.  Context-Aware Gaussian Fields for Non-rigid Point Set Registration , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Junjun Jiang,et al.  Non-Rigid Point Set Registration with Robust Transformation Estimation under Manifold Regularization , 2017, AAAI.

[15]  Jiri Matas,et al.  Matching with PROSAC - progressive sample consensus , 2005, 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'05).

[16]  Vladimir Kolmogorov,et al.  Feature Correspondence Via Graph Matching: Models and Global Optimization , 2008, ECCV.

[17]  Zhanyi Hu,et al.  Rejecting Mismatches by Correspondence Function , 2010, International Journal of Computer Vision.

[18]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[19]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[20]  Minsu Cho,et al.  Progressive graph matching: Making a move of graphs via probabilistic voting , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[21]  David S. Doermann,et al.  Robust point matching for nonrigid shapes by preserving local neighborhood structures , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[22]  Jon Louis Bentley,et al.  Multidimensional binary search trees used for associative searching , 1975, CACM.

[23]  Lei Wang,et al.  Progressive Mode-Seeking on Graphs for Sparse Feature Matching , 2014, ECCV.

[24]  Kenneth Steiglitz,et al.  Combinatorial Optimization: Algorithms and Complexity , 1981 .

[25]  Lei Peng,et al.  Learning coherent vector fields for robust point matching under manifold regularization , 2016, Neurocomputing.

[26]  Zhuowen Tu,et al.  Learning Context-Sensitive Shape Similarity by Graph Transduction , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Junjun Jiang,et al.  Guided Locality Preserving Feature Matching for Remote Sensing Image Registration , 2018, IEEE Transactions on Geoscience and Remote Sensing.

[28]  Zhuowen Tu,et al.  Robust Estimation of Nonrigid Transformation for Point Set Registration , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Anand Rangarajan,et al.  A new point matching algorithm for non-rigid registration , 2003, Comput. Vis. Image Underst..

[30]  Radu Horaud,et al.  Rigid and Articulated Point Registration with Expectation Conditional Maximization , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[31]  Bing-Yu Chen,et al.  Matching Images With Multiple Descriptors: An Unsupervised Approach for Locally Adaptive Descriptor Selection , 2015, IEEE Transactions on Image Processing.

[32]  Matthijs C. Dorst Distinctive Image Features from Scale-Invariant Keypoints , 2011 .

[33]  Zheng Wang,et al.  SRLSP: A Face Image Super-Resolution Algorithm Using Smooth Regression With Local Structure Prior , 2017, IEEE Transactions on Multimedia.

[34]  Shuicheng Yan,et al.  Common visual pattern discovery via spatially coherent correspondences , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[35]  Markus Vincze,et al.  Guided Matching Based on Statistical Optical Flow for Fast and Robust Correspondence Analysis , 2016, ECCV.

[36]  Junjun Jiang,et al.  Locality Preserving Matching , 2017, IJCAI.

[37]  Anders Bjorholm Dahl,et al.  Large-Scale Data for Multiple-View Stereopsis , 2016, International Journal of Computer Vision.

[38]  Ronen Basri,et al.  Feature Matching with Bounded Distortion , 2014, ACM Trans. Graph..

[39]  Alan L. Yuille,et al.  Non-Rigid Point Set Registration by Preserving Global and Local Structures , 2016, IEEE Transactions on Image Processing.

[40]  Andrew Vardy,et al.  An Orientation Invariant Visual Homing Algorithm , 2012, J. Intell. Robotic Syst..

[41]  Minh N. Do,et al.  CODE: Coherence Based Decision Boundaries for Feature Correspondence , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[42]  Michael Werman,et al.  A Linear Time Histogram Metric for Improved SIFT Matching , 2008, ECCV.

[43]  Yu Zhou,et al.  Visual Homing via Guided Locality Preserving Matching , 2018, 2018 IEEE International Conference on Robotics and Automation (ICRA).

[44]  Yansheng Li,et al.  Feature guided Gaussian mixture model with semi-supervised EM and local geometric constraint for retinal image registration , 2017, Inf. Sci..

[45]  Ji Zhao,et al.  Non-rigid visible and infrared face registration via regularized Gaussian fields criterion , 2015, Pattern Recognit..

[46]  Paul J. Besl,et al.  A Method for Registration of 3-D Shapes , 1992, IEEE Trans. Pattern Anal. Mach. Intell..

[47]  Andriy Myronenko,et al.  Point Set Registration: Coherent Point Drift , 2009, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[48]  Zhuowen Tu,et al.  Robust Point Matching via Vector Field Consensus , 2014, IEEE Transactions on Image Processing.

[49]  Alan L. Yuille,et al.  Semi-Supervised Sparse Representation Based Classification for Face Recognition With Insufficient Labeled Samples , 2016, IEEE Transactions on Image Processing.

[50]  Vincent Lepetit,et al.  DAISY: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[51]  Martial Hebert,et al.  A spectral technique for correspondence problems using pairwise constraints , 2005, Tenth IEEE International Conference on Computer Vision (ICCV'05) Volume 1.

[52]  Vladimir G. Kim,et al.  Blended intrinsic maps , 2011, SIGGRAPH 2011.

[53]  Mongi A. Abidi,et al.  Gaussian fields: a new criterion for 3D rigid registration , 2004, Pattern Recognit..

[54]  Jitendra Malik,et al.  Shape matching and object recognition using shape contexts , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[55]  OngSim Heng,et al.  A robust global and local mixture distance based non-rigid point set registration , 2015 .

[56]  D. Schroeter On the Robustness of Visual Homing under Landmark Uncertainty 1 , 2008 .

[57]  Ji Zhao,et al.  Visual homing by robust interpolation for sparse motion flow , 2017, 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS).

[58]  Andrew Vardy,et al.  Local visual homing by matched-filter descent in image distances , 2006, Biological Cybernetics.

[59]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[60]  Xiaochun Cao,et al.  Good match exploration using triangle constraint , 2012, Pattern Recognit. Lett..

[61]  Charles A. Micchelli,et al.  On Learning Vector-Valued Functions , 2005, Neural Computation.

[62]  Ming-Ming Cheng,et al.  Robust Non-parametric Data Fitting for Correspondence Modeling , 2013, 2013 IEEE International Conference on Computer Vision.

[63]  Stefan Winkler,et al.  California-ND: An annotated dataset for near-duplicate detection in personal photo collections , 2013, 2013 Fifth International Workshop on Quality of Multimedia Experience (QoMEX).

[64]  B. Ripley,et al.  Robust Statistics , 2018, Encyclopedia of Mathematical Geosciences.

[65]  Yasuyuki Matsushita,et al.  GMS: Grid-Based Motion Statistics for Fast, Ultra-robust Feature Correspondence , 2017, 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[66]  Roland Siegwart,et al.  Visual Homing From Scale With an Uncalibrated Omnidirectional Camera , 2013, IEEE Transactions on Robotics.

[67]  Minh N. Do,et al.  Bilateral Functions for Global Motion Modeling , 2014, ECCV.