Normalized similarity measurement and query adaptive fusion for graph based visual reranking

Developing effective fusion schemes for multiple feature types has always been a hot issue in content-based image retrieval. In this paper, we propose a novel method for graph based visual reranking, which addresses two major limitations in existing methods. Firstly, in the phase of graph construction, our method introduces fine-grained measurements for image relations, by assigning the edge weights using normalized similarity. Furthermore, in the phase of graph fusion, rather than summing up all the graphs for different single features indiscriminately, we propose to estimate the reliability of each feature through a statistical model, and selectively fuse the single graphs via query-adaptive fusion weights. Our method is evaluated on three public datasets, by fusing SIFT and CNN, two complementary features. Experimental results demonstrate the effectiveness of the proposed method, which yields superior results than the competing methods.

[1]  Shiliang Zhang,et al.  Semantic-Aware Co-Indexing for Image Retrieval. , 2015, IEEE transactions on pattern analysis and machine intelligence.

[2]  Qi Tian,et al.  Packing and Padding: Coupled Multi-index for Accurate Image Retrieval , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[4]  Qi Tian,et al.  Rank-aware graph fusion with contextual dissimilarity measurement for image retrieval , 2015, 2015 IEEE International Conference on Image Processing (ICIP).

[5]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).

[6]  Qi Tian,et al.  Bayes Merging of Multiple Vocabularies for Scalable Image Retrieval , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[7]  Rongrong Ji,et al.  Visual Reranking through Weakly Supervised Multi-graph Learning , 2013, 2013 IEEE International Conference on Computer Vision.

[8]  Larry S. Davis,et al.  Submodular Reranking with Multiple Feature Modalities for Image Retrieval , 2014, ACCV.

[9]  Cordelia Schmid,et al.  Hamming Embedding and Weak Geometric Consistency for Large Scale Image Search , 2008, ECCV.

[10]  Fahad Shahbaz Khan,et al.  Color attributes for object detection , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[12]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[13]  Michael Isard,et al.  Object retrieval with large vocabularies and fast spatial matching , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Qi Tian,et al.  Coupled Binary Embedding for Large-Scale Image Retrieval , 2014, IEEE Transactions on Image Processing.

[15]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[16]  Stefan Carlsson,et al.  CNN Features Off-the-Shelf: An Astounding Baseline for Recognition , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition Workshops.

[17]  Matthijs Douze,et al.  Bag-of-colors for improved image search , 2011, ACM Multimedia.

[18]  Ming Yang,et al.  Query Specific Fusion for Image Retrieval , 2012, ECCV.

[19]  Qi Tian,et al.  Lp-Norm IDF for Large Scale Image Search , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[20]  Yuan Dong,et al.  An efficient graph-based visual reranking , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[21]  Ying Wu,et al.  Object retrieval and localization with spatially-constrained similarity measure and k-NN re-ranking , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[22]  Qi Tian,et al.  Query-adaptive late fusion for image search and person re-identification , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]  Andrew Zisserman,et al.  Three things everyone should know to improve object retrieval , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[24]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[25]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[26]  Luc Van Gool,et al.  Hello neighbor: Accurate object retrieval with k-reciprocal nearest neighbors , 2011, CVPR 2011.

[27]  Qi Tian,et al.  Visual reranking with improved image graph , 2014, 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).