HyperSSR: A hypergraph based semi-supervised ranking method for visual search reranking

Abstract Recently, considerable efforts have been made in visual search reranking towards refining initial text-based image search results. In this paper, we propose a hypergraph based semi-supervised ranking method called HyperSSR for image search reranking. According to the basic visual consistency principle that visually similar images should have similar ranking scores, we introduce the hypergraph to capture the intrinsic geometrical structure of the data distribution. To build a robust hypergraph, a novel hypergraph construction approach is developed to incorporate relevance and pseudo relevance degree information from labeled and unlabeled samples, respectively. Based on the premise that a ranking model should work better with the prior pairwise preferences, we jointly incorporate the hypergraph regularizer and the prior pairwise preferences information into a unified ranking learning framework. Experimental results on MSRA-MM 1.0 dataset suggest our proposed approach produces superior performances compared with several state-of-the-art methods.

[1]  Shih-Fu Chang,et al.  Video search reranking through random walk over document-level context graph , 2007, ACM Multimedia.

[2]  Zechao Li,et al.  Projective nonnegative matrix factorization for social image retrieval , 2016, Neurocomputing.

[3]  Jaana Kekäläinen,et al.  Cumulated gain-based evaluation of IR techniques , 2002, TOIS.

[4]  Yan Liu,et al.  Ordinal Regression via Manifold Learning , 2011, AAAI.

[5]  Chaoqun Hong,et al.  Hypergraph-based multi-example ranking with sparse representation for transductive learning image retrieval , 2013, Neurocomputing.

[6]  Yue Gao,et al.  Attribute-Augmented Semantic Hierarchy: Towards a Unified Framework for Content-Based Image Retrieval , 2014, TOMM.

[7]  Rong Yan,et al.  Multimedia Search with Pseudo-relevance Feedback , 2003, CIVR.

[8]  Alan Hanjalic,et al.  Prototype-Based Image Search Reranking , 2012, IEEE Transactions on Multimedia.

[9]  Yuxiao Hu,et al.  Face recognition using Laplacianfaces , 2005, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[10]  Xian-Sheng Hua,et al.  Bayesian video search reranking , 2008, ACM Multimedia.

[11]  Tat-Seng Chua,et al.  Discrete Image Hashing Using Large Weakly Annotated Photo Collections , 2016, AAAI.

[12]  Tao Li,et al.  A Joint Local-Global Approach for Medical Terminology Assignment , 2014, MedIR@SIGIR.

[13]  Xiangyang Xue,et al.  Web image retrieval reranking with multi-view clustering , 2009, WWW '09.

[14]  Meng Wang,et al.  Multimedia answering: enriching text QA with media information , 2011, SIGIR.

[15]  Meng Wang,et al.  Adaptive Hypergraph Learning and its Application in Image Classification , 2012, IEEE Transactions on Image Processing.

[16]  Jianru Xue,et al.  Image re-ranking with an alternating optimization , 2015, Neurocomputing.

[17]  Yang Yang,et al.  Robust (Semi) Nonnegative Graph Embedding , 2014, IEEE Transactions on Image Processing.

[18]  Qi Tian,et al.  Multimedia search reranking: A literature survey , 2014, CSUR.

[19]  Shih-Fu Chang,et al.  Video search reranking via information bottleneck principle , 2006, MM '06.

[20]  Xian-Sheng Hua,et al.  Active Reranking for Web Image Search , 2010, IEEE Transactions on Image Processing.

[21]  Xian-Sheng Hua,et al.  Content-aware Ranking for visual search , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[22]  Vipin Kumar,et al.  Hypergraph Based Clustering in High-Dimensional Data Sets: A Summary of Results , 1998, IEEE Data Eng. Bull..

[23]  Jane You,et al.  Image clustering by hyper-graph regularized non-negative matrix factorization , 2014, Neurocomputing.

[24]  Lei Zhang,et al.  Dual-ranking for web image retrieval , 2010, CIVR '10.

[25]  Tie-Yan Liu,et al.  Adapting ranking SVM to document retrieval , 2006, SIGIR.

[26]  Boris Babenko,et al.  ImprovingWeb-based Image Search via Content Based Clustering , 2006, 2006 Conference on Computer Vision and Pattern Recognition Workshop (CVPRW'06).

[27]  Yuqing He,et al.  Semi-supervised LPP algorithms for learning-to-rank-based visual search reranking , 2015, Inf. Sci..

[28]  Xiaogang Wang,et al.  Query-specific visual semantic spaces for web image re-ranking , 2011, CVPR 2011.

[29]  Xuelong Li,et al.  Image Annotation by Multiple-Instance Learning With Discriminative Feature Mapping and Selection , 2014, IEEE Transactions on Cybernetics.

[30]  Xuelong Li,et al.  Ranking Graph Embedding for Learning to Rerank , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[31]  Yan Liu,et al.  Semi-supervised manifold ordinal regression for image ranking , 2011, MM '11.

[32]  Alan Hanjalic,et al.  Supervised reranking for web image search , 2010, ACM Multimedia.

[33]  Ricardo da Silva Torres,et al.  Exploiting pairwise recommendation and clustering strategies for image re-ranking , 2012, Inf. Sci..

[34]  Ralf Herbrich,et al.  Large margin rank boundaries for ordinal regression , 2000 .

[35]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[36]  Xiangwei Kong,et al.  Large-scale image retrieval based on boosting iterative quantization hashing with query-adaptive reranking , 2013, Neurocomputing.

[37]  Jianping Fan,et al.  A regularized optimization framework for tag completion and image retrieval , 2015, Neurocomputing.

[38]  Michael R. Lyu,et al.  A Multimodal and Multilevel Ranking Framework for Content-Based Video Retrieval , 2007, 2007 IEEE International Conference on Acoustics, Speech and Signal Processing - ICASSP '07.

[39]  Qingshan Liu,et al.  Image retrieval via probabilistic hypergraph ranking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[40]  Meng Wang,et al.  Learning Visual Semantic Relationships for Efficient Visual Retrieval , 2015, IEEE Transactions on Big Data.

[41]  TianQi,et al.  Multimedia search reranking , 2014 .

[42]  Yi-Liang Zhao,et al.  Bridging the Vocabulary Gap between Health Seekers and Healthcare Knowledge , 2015, IEEE Transactions on Knowledge and Data Engineering.

[43]  S T Roweis,et al.  Nonlinear dimensionality reduction by locally linear embedding. , 2000, Science.

[44]  Tao Mei,et al.  CrowdReranking: exploring multiple search engines for visual search reranking , 2009, SIGIR.

[45]  Tao Mei,et al.  Optimizing Visual Search Reranking via Pairwise Learning , 2011, IEEE Transactions on Multimedia.

[46]  Meng Wang,et al.  Harvesting visual concepts for image search with complex queries , 2012, ACM Multimedia.

[47]  Hung-Khoon Tan,et al.  Beyond search: Event-driven summarization for web videos , 2011, TOMCCAP.

[48]  Shaogang Gong,et al.  Constructing Robust Affinity Graphs for Spectral Clustering , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[49]  Tie-Yan Liu,et al.  Learning to rank: from pairwise approach to listwise approach , 2007, ICML '07.

[50]  Xiangyu Wang,et al.  Learning to Recommend Descriptive Tags for Questions in Social Forums , 2014, TOIS.

[51]  Shumeet Baluja,et al.  Pagerank for product image search , 2008, WWW.

[52]  Frédéric Jurie,et al.  Improving web image search results using query-relative classifiers , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[53]  Zhongfei Zhang,et al.  Visual search reranking with RElevant Local Discriminant Analysis , 2016, Neurocomputing.

[54]  Alan Hanjalic,et al.  Learning from search engine and human supervision for web image search , 2011, MM '11.