Scalable object instance recognition based on keygraph matching

Abstract We propose a generalisation of the local feature matching framework, where keypoints are replaced by k-keygraphs, i.e., isomorphic directed attributed graphs of cardinality k whose vertices are keypoints. Keygraphs have structural and topological properties which are discriminative and efficient to compute, based on graph edge length and orientation as well as vertex scale and orientation. Keypoint matching is performed based on descriptor similarity. Next, 2-keygraphs are calculated; as a result, the number of incorrect keypoint matches reduced in 75% (while the correct keypoint matches were preserved). Then, 3-keygraphs are calculated, followed by 4-keygraphs; this yielded a significant reduction of 99% in the number of remaining incorrect keypoint matches. The stage that finds 2-keygraphs has a computational cost equal to a small fraction of the cost of the keypoint matching stage, while the stages that find 3-keygraphs or 4-keygraphs have a negligible cost. In the final stage, RANSAC finds object poses represented as affine transformations mapping images. Our experiments concern large-scale object instance recognition subject to occlusion, background clutter and appearance changes. By using 4-keygraphs, RANSAC needed 1% of the iterations in comparison with 2-keygraphs or simple keypoints. As a result, using 4-keygraphs provided a better efficiency as well as allowed a larger number of initial keypoints matches to be established, which increased performance.

[1]  J. van Leeuwen,et al.  Graph Based Representations in Pattern Recognition , 2003, Lecture Notes in Computer Science.

[2]  Tsuhan Chen,et al.  Image retrieval with geometry-preserving visual phrases , 2011, CVPR 2011.

[3]  Yannis Avrithis,et al.  Hough Pyramid Matching: Speeded-Up Geometry Re-ranking for Large Scale Image Retrieval , 2014, International Journal of Computer Vision.

[4]  Mario Fernando Montenegro Campos,et al.  On the Improvement of Image Feature Matching under Perspective Transformations , 2010, 2010 23rd SIBGRAPI Conference on Graphics, Patterns and Images.

[5]  Nanning Zheng,et al.  Exploiting local linear geometric structure for identifying correct matches , 2014, Comput. Vis. Image Underst..

[6]  Andrea Vedaldi,et al.  Vlfeat: an open and portable library of computer vision algorithms , 2010, ACM Multimedia.

[7]  Mohammed Bennamoun,et al.  A Comprehensive Performance Evaluation of 3D Local Feature Descriptors , 2015, International Journal of Computer Vision.

[8]  Federico Tombari,et al.  BOLD Features to Detect Texture-less Objects , 2013, 2013 IEEE International Conference on Computer Vision.

[9]  Torsten Sattler,et al.  SCRAMSAC: Improving RANSAC's efficiency with a spatial consistency filter , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[10]  Martha Larson,et al.  Pairwise geometric matching for large-scale object retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[11]  Wei Wei,et al.  Pairwise Matching through Max-Weight Bipartite Belief Propagation , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[12]  Roberto Marcondes Cesar Junior,et al.  Efficient Object Recognition Using Sampling of Keypoint Triples and Keygraph Structure , 2016, 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI).

[13]  Nico Blodow,et al.  Fast Point Feature Histograms (FPFH) for 3D registration , 2009, 2009 IEEE International Conference on Robotics and Automation.

[14]  Roberto Marcondes Cesar Junior,et al.  Object Detection by Keygraph Classification , 2009, GbRPR.

[15]  Feng Wu,et al.  3D visual phrases for landmark recognition , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[16]  Nassir Navab,et al.  N3M: Natural 3D Markers for Real-Time Object Detection and Pose Estimation , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[17]  Alvaro Collet,et al.  Making specific features less discriminative to improve point-based 3D object recognition , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  Tibério S. Caetano,et al.  Unified graph matching in Euclidean spaces , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[19]  Yannis Avrithis,et al.  Scalable triangulation-based logo recognition , 2011, ICMR.

[20]  Martial Hebert,et al.  Fast and Scalable Approximate Spectral Matching for Higher Order Graph Matching , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[21]  Dario Maio,et al.  Saliency-based keypoint selection for fast object detection and matching , 2015, Pattern Recognit. Lett..

[22]  Adrian Hilton,et al.  Multi-modal Visual Data Registration and Web-based Visualisation , 2017 .

[23]  Rahul Sukthankar,et al.  MatchNet: Unifying feature and metric learning for patch-based matching , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[24]  Federico Tombari,et al.  Performance Evaluation of 3D Keypoint Detectors , 2012, International Journal of Computer Vision.

[25]  David G. Lowe,et al.  Scalable Nearest Neighbor Algorithms for High Dimensional Data , 2014, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[26]  Feng Wu,et al.  Efficient 2D-to-3D Correspondence Filtering for Scalable 3D Object Recognition , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Silvio Savarese,et al.  Universal Correspondence Network , 2016, NIPS.

[28]  Vincent Lepetit,et al.  Learning to Assign Orientations to Feature Points , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[29]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[30]  Stefano Soatto,et al.  Domain-size pooling in local descriptors: DSP-SIFT , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[31]  Gustavo Carneiro,et al.  Flexible spatial models for grouping local image features , 2004, Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2004. CVPR 2004..

[32]  David G. Lowe,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004, International Journal of Computer Vision.

[33]  Robert C. Bolles,et al.  Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography , 1981, CACM.

[34]  Javier Ruiz-del-Solar,et al.  Object recognition using local invariant features for robotic applications: A survey , 2016, Pattern Recognit..

[35]  Simon A. J. Winder,et al.  Object instance recognition using triplets of feature symbols , 2007 .

[36]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Adrian Hilton,et al.  Hybrid 3D feature description and matching for multi-modal data registration , 2014, 2014 IEEE International Conference on Image Processing (ICIP).