Ranking consistency for image matching and object retrieval

The goal of object retrieval is to rank a set of images by the similarity of their contents to those of a query image. However, it is difficult to measure image content similarity due to visual changes caused by varying viewpoint and environment. In this paper, we propose a simple, efficient method to more effectively measure content similarity from image measurements. Our method is based on the ranking information available from existing retrieval systems. We observe that images within the set which, when used as queries, yield similar ranking lists are likely to be relevant to each other and vice versa. In our method, ranking consistency is used as a verification method to efficiently refine an existing ranking list, in much the same fashion that spatial verification is employed. The efficiency of our method is achieved by a list-wise min-Hash scheme, which allows rapid calculation of an approximate similarity ranking. Experimental results demonstrate the effectiveness of the proposed framework and its applications. HighlightsWe propose an image matching and retrieval framework based on ranking consistency.A list-wise min-Hash method ensures the efficiency of ranking verification.Our method is flexible and thus has a variety of retrieval-related applications.

[1]  Ricardo da Silva Torres,et al.  Image Re-ranking and Rank Aggregation Based on Similarity of Ranked Lists , 2011, CAIP.

[2]  Panu Turcot,et al.  Better matching with fewer features: The selection of useful features in large database recognition problems , 2009, 2009 IEEE 12th International Conference on Computer Vision Workshops, ICCV Workshops.

[3]  Proceedings of the ACM International Conference on Multimedia, MM '14, Orlando, FL, USA, November 03 - 07, 2014 , 2014, ACM Multimedia.

[4]  Thorsten Joachims,et al.  Optimizing search engines using clickthrough data , 2002, KDD.

[5]  David G. Lowe,et al.  The viewpoint consistency constraint , 2015, International Journal of Computer Vision.

[6]  Jiri Matas,et al.  Efficient representation of local geometry for large scale object retrieval , 2009, CVPR.

[7]  Ilaria Bartolini,et al.  Efficient and effective similarity-based video retrieval , 2010, SISAP.

[8]  Alistair Moffat,et al.  A similarity measure for indefinite rankings , 2010, TOIS.

[9]  C. Spearman.  The proof and measurement of association between two things. , 2015, International journal of epidemiology.

[10]  Cordelia Schmid,et al.  A Comparison of Affine Region Detectors , 2005, International Journal of Computer Vision.

[11]  Michael Isard,et al.  Total Recall: Automatic Query Expansion with a Generative Feature Model for Object Retrieval , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[12]  Nenghai Yu,et al.  Distance metric learning from uncertain side information for automated photo tagging , 2011, TIST.

[13]  Winston H. Hsu,et al.  Query expansion for hash-based image object retrieval , 2009, ACM Multimedia.

[14]  Michael McGill,et al.  Introduction to Modern Information Retrieval , 1983 .

[15]  Ricardo da Silva Torres,et al.  Exploiting Contextual Information for Image Re-ranking , 2010, CIARP.

[16]  Fabio Roli,et al.  Bayesian relevance feedback for content-based image retrieval , 2004, Pattern Recognit..

[17]  Nenghai Yu,et al.  Flickr distance , 2008, ACM Multimedia.

[18]  Andrew Zisserman,et al.  Near Duplicate Image Detection: min-Hash and tf-idf Weighting , 2008, BMVC.

[19]  Marcel Worring,et al.  Content-Based Image Retrieval at the End of the Early Years , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[20]  Andrew Zisserman,et al.  Video Google: a text retrieval approach to object matching in videos , 2003, Proceedings Ninth IEEE International Conference on Computer Vision.

[21]  Anton van den Hengel,et al.  Boosting Object Retrieval With Group Queries , 2012, IEEE Signal Processing Letters.

[22]  Jiri Matas,et al.  Total recall II: Query expansion revisited , 2011, CVPR 2011.

[23]  Andrew Zisserman,et al.  Three things everyone should know to improve object retrieval , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Bernhard P. Wrobel,et al.  Multiple View Geometry in Computer Vision , 2001 .

[25]  Andrei Z. Broder,et al.  On the resemblance and containment of documents , 1997, Proceedings. Compression and Complexity of SEQUENCES 1997 (Cat. No.97TB100171).

[26]  Michael Isard,et al.  Descriptor Learning for Efficient Retrieval , 2010, ECCV.

[27]  M. Kendall.  A NEW MEASURE OF RANK CORRELATION , 1938 .

[28]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Andrew Zisserman,et al.  Multiple queries for large scale specific object retrieval , 2012, BMVC.

[30]  Michael Isard,et al.  Lost in quantization: Improving particular object retrieval in large scale image databases , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[31]  B. Ripley,et al.  Pattern Recognition , 1968, Nature.

[32]  O. Chum,et al.  Geometric min-Hashing: Finding a (thick) needle in a haystack , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[33]  Rong Jin,et al.  Web image retrieval re-ranking with relevance model , 2003, Proceedings IEEE/WIC International Conference on Web Intelligence (WI 2003).

[34]  Nenghai Yu,et al.  Flickr Distance: A Relationship Measure for Visual Concepts , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[35]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[36]  Andrew Zisserman,et al.  Object Mining Using a Matching Graph on Very Large Image Collections , 2008, 2008 Sixth Indian Conference on Computer Vision, Graphics & Image Processing.

[37]  Andrew Zisserman,et al.  Geometric LDA: A Generative Model for Particular Object Discovery , 2008, BMVC.

[38]  Tsuhan Chen,et al.  Image retrieval with geometry-preserving visual phrases , 2011, CVPR 2011.

[39]  Nenghai Yu,et al.  Multiple-instance ranking: Learning to rank images for image retrieval , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[40]  Cordelia Schmid,et al.  Improving Bag-of-Features for Large Scale Image Search , 2010, International Journal of Computer Vision.

[41]  Minsu Cho,et al.  Authority-shift clustering: Hierarchical clustering by authority seeking on graphs , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[42]  Raimondo Schettini,et al.  Content-based similarity retrieval of trademarks using relevance feedback , 2001, Pattern Recognit..

[43]  Michael Isard,et al.  General Theory , 1969 .

[44]  Adriana Kovashka,et al.  WhittleSearch: Image search with relative attribute feedback , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[45]  C. Spearman.  The proof and measurement of association between two things. By C. Spearman, 1904. , 1987, The American journal of psychology.

[46]  Luc Van Gool,et al.  Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors , 2011, CVPR 2011.