MultiVCRank With Applications to Image Retrieval

In this paper, we propose and develop a multi-visual-concept ranking (MultiVCRank) scheme for image retrieval. The key idea is that an image can be represented by several visual concepts, and a hypergraph is built based on visual concepts as hyperedges, where each edge contains images as vertices to share a specific visual concept. In the constructed hypergraph, the weight between two vertices in a hyperedge is incorporated, and it can be measured by their affinity in the corresponding visual concept. A ranking scheme is designed to compute the association scores of images and the relevance scores of visual concepts by employing input query vectors to handle image retrieval. In the scheme, the association and relevance scores are determined by an iterative method to solve limiting probabilities of a multi-dimensional Markov chain arising from the constructed hypergraph. The convergence analysis of the iteration method is studied and analyzed. Moreover, a learning algorithm is also proposed to set the parameters in the scheme, which makes it simple to use. Experimental results on the MSRC, Corel, and Caltech256 data sets have demonstrated the effectiveness of the proposed method. In the comparison, we find that the retrieval performance of MultiVCRank is substantially better than those of HypergraphRank, ManifoldRank, TOPHITS, and RankSVM.

[1]  Yunming Ye,et al.  MultiRank: co-ranking for objects and relations in multi-relational data , 2011, KDD.

[2]  Ashish Mohan Yadav,et al.  A Survey on Content Based Image Retrieval Systems , 2014 .

[3]  Yan Gao,et al.  A Review of Region-Based Image Retrieval , 2010, J. Signal Process. Syst..

[4]  Xuelong Li,et al.  Asymmetric bagging and random subspace for support vector machines-based relevance feedback in image retrieval , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[5]  Ming Yang,et al.  DeepFace: Closing the Gap to Human-Level Performance in Face Verification , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Yoshida Yuichi,et al.  Multiclass VisualRank: Image Ranking Method in Clustered Subsets Based on Visual Features , 2008 .

[7]  Koen E. A. van de Sande,et al.  Evaluating Color Descriptors for Object and Scene Recognition , 2010, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[8]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[9]  R. B. Kellogg,et al.  Uniqueness in the Schauder fixed point theorem , 1976 .

[10]  Jürgen Schmidhuber,et al.  Multi-column deep neural networks for image classification , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[11]  Yunming Ye,et al.  HAR: Hub, Authority and Relevance Scores in Multi-Relational Data for Query Search , 2012, SDM.

[12]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[13]  Hichem Sahbi,et al.  Manifold learning using robust Graph Laplacian for interactive image search , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Dacheng Tao,et al.  Biased Discriminant Euclidean Embedding for Content-Based Image Retrieval , 2010, IEEE Transactions on Image Processing.

[15]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[16]  Qingshan Liu,et al.  Image retrieval via probabilistic hypergraph ranking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[17]  Jitendra Malik,et al.  Normalized cuts and image segmentation , 1997, Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[18]  James Ze Wang,et al.  SIMPLIcity: Semantics-Sensitive Integrated Matching for Picture LIbraries , 2000, IEEE Trans. Pattern Anal. Mach. Intell..

[19]  Shi-Min Hu,et al.  Global contrast based salient region detection , 2011, CVPR 2011.

[20]  Brett W. Bader,et al.  The TOPHITS Model for Higher-Order Web Link Analysis∗ , 2006 .

[21]  Yuichi Yoshida,et al.  Multiclass VisualRank: image ranking method in clustered subsets based on visual features , 2009, SIGIR.

[22]  Ieee Xplore,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence Information for Authors , 2022, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[23]  Jitendra Malik,et al.  Blobworld: Image Segmentation Using Expectation-Maximization and Its Application to Image Querying , 2002, IEEE Trans. Pattern Anal. Mach. Intell..

[24]  智一 吉田,et al.  Efficient Graph-Based Image Segmentationを用いた圃場図自動作成手法の検討 , 2014 .

[25]  LiXuelong,et al.  Asymmetric Bagging and Random Subspace for Support Vector Machines-Based Relevance Feedback in Image Retrieval , 2006 .

[26]  Liqing Zhang,et al.  Saliency Detection: A Spectral Residual Approach , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[27]  Jingrui He,et al.  Manifold-ranking based image retrieval , 2004, MULTIMEDIA '04.

[28]  Ying Liu,et al.  A survey of content-based image retrieval with high-level semantics , 2007, Pattern Recognit..

[29]  Tieniu Tan,et al.  Deep semantic ranking based hashing for multi-label image retrieval , 2015, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[30]  Qingshan Liu,et al.  Hypergraph with sampling for image retrieval , 2011, Pattern Recognit..

[31]  Yang Song,et al.  Learning Fine-Grained Image Similarity with Deep Ranking , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[32]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[33]  Bo Zhang,et al.  Relevance feedback in region-based image retrieval , 2004, IEEE Transactions on Circuits and Systems for Video Technology.

[34]  Chun Chen,et al.  Music recommendation by unified hypergraph: combining social media information and music content , 2010, ACM Multimedia.

[35]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[36]  Jason Weston,et al.  Large scale image annotation: learning to rank with joint word-image embeddings , 2010, Machine Learning.

[37]  Hanjiang Lai,et al.  Supervised Hashing for Image Retrieval via Image Representation Learning , 2014, AAAI.

[38]  Jun Zhang,et al.  Content Based Image Retrieval Using Unclean Positive Examples , 2009, IEEE Transactions on Image Processing.

[39]  Camille Couprie,et al.  Learning Hierarchical Features for Scene Labeling , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[40]  Ying Liu,et al.  Region-based image retrieval with high-level semantics using decision tree learning , 2008, Pattern Recognit..

[41]  Hung-Khoon Tan,et al.  Modeling video hyperlinks with hypergraph for web video reranking , 2008, ACM Multimedia.

[42]  Philip S. Yu,et al.  Efficient Relevance Feedback for Content-Based Image Retrieval by Mining User Navigation Patterns , 2011, IEEE Transactions on Knowledge and Data Engineering.

[43]  Yihong Gong,et al.  Unsupervised Image Categorization by Hypergraph Partition , 2011, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  G LoweDavid,et al.  Distinctive Image Features from Scale-Invariant Keypoints , 2004 .

[45]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[46]  Thorsten Joachims,et al.  Training linear SVMs in linear time , 2006, KDD '06.

[47]  Jingrui He,et al.  Generalized Manifold-Ranking-Based Image Retrieval , 2006, IEEE Transactions on Image Processing.