Scalable graph based non-negative multi-view embedding for image ranking

Abstract Due to the well-known semantic gap, content based image retrieval task is still a challenge problem. The performance of image ranking highly depends on feature representation. In this paper, trying to make a more discriminative feature, we propose a multi-graph based non-negative feature embedding framework for image ranking. In this framework, various image features are embedded into a unified latent space by a learned graph based non-negative multi-view embedding model. In this model, a multi-graph based regularization term, which discovers the intrinsic geometrical and the discriminating structure of the data space, is imposed into the non-negative matrix factorization. The framework learns to find an optimized combination of different Laplacian matrices to approximate the intrinsic manifold automatically. Meanwhile, multiple anchor graphs are utilized to reduce the complexity of computational. Finally, ranking is conducted according to the relevance score inferred by a Markov random field. Extensive experiments prove the effectiveness of proposed method.

[1]  João Magalhães,et al.  Assisted news reading with automated illustration , 2010, ACM Multimedia.

[2]  Jianru Xue,et al.  Fast Democratic Aggregation and Query Fusion for Image Search , 2015, ICMR.

[3]  Yue Gao,et al.  Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval , 2013, ACM Multimedia.

[4]  Jason Weston,et al.  Large scale image annotation: learning to rank with joint word-image embeddings , 2010, Machine Learning.

[5]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[6]  James Allan,et al.  A Case For Shorter Queries, and Helping Users Create Them , 2007, NAACL.

[7]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[8]  Trevor Darrell,et al.  Caffe: Convolutional Architecture for Fast Feature Embedding , 2014, ACM Multimedia.

[9]  Guiguang Ding,et al.  Robust Nonnegative Matrix Factorization with Discriminability for image representation , 2015, 2015 IEEE International Conference on Multimedia and Expo (ICME).

[10]  Thomas S. Huang,et al.  Graph Regularized Nonnegative Matrix Factorization for Data Representation. , 2011, IEEE transactions on pattern analysis and machine intelligence.

[11]  Frédéric Jurie,et al.  Image re-ranking based on statistics of frequent patterns , 2014, ICMR.

[12]  Zhongfei Zhang,et al.  Visual search reranking with RElevant Local Discriminant Analysis , 2016, Neurocomputing.

[13]  David A. Forsyth,et al.  Describing objects by their attributes , 2009, CVPR.

[14]  J. Laurie Snell,et al.  Markov Random Fields and Their Applications , 1980 .

[15]  Yi Yang,et al.  Ranking with local regression and global alignment for cross media retrieval , 2009, ACM Multimedia.

[16]  Josep-Lluís Larriba-Pey,et al.  Massive Query Expansion by Exploiting Graph Knowledge Bases for Image Retrieval , 2014, ICMR.

[17]  Ji Wan,et al.  Deep Learning for Content-Based Image Retrieval: A Comprehensive Study , 2014, ACM Multimedia.

[18]  Ke Lu,et al.  Multi-manifold Sparse Graph Embedding for Multi-modal Image Classification , 2016, Neurocomputing.

[19]  W. Bruce Croft,et al.  Learning concept importance using a weighted dependence model , 2010, WSDM '10.

[20]  Qiang Chen,et al.  Network In Network , 2013, ICLR.

[21]  Yinglin Wang,et al.  Low rank approximation with sparse integration of multiple manifolds for data representation , 2014, Applied Intelligence.

[22]  Pascal Vincent,et al.  Unsupervised Feature Learning and Deep Learning: A Review and New Perspectives , 2012, ArXiv.

[23]  Hongtao Lu,et al.  Non-negative Matrix Factorization with Pairwise Constraints and Graph Laplacian , 2015, Neural Processing Letters.

[24]  Meng Wang,et al.  Disease Inference from Health-Related Questions via Sparse Deep Learning , 2015, IEEE Transactions on Knowledge and Data Engineering.

[25]  Sebastian Nowozin,et al.  Weighted Substructure Mining for Image Analysis , 2007, 2007 IEEE Conference on Computer Vision and Pattern Recognition.

[26]  Meng Wang,et al.  Oracle in Image Search: A Content-Based Approach to Performance Prediction , 2012, TOIS.

[27]  Gang Wang,et al.  Object image retrieval by exploiting online knowledge resources , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[28]  Niranjan Balasubramanian,et al.  Exploring reductions for long web queries , 2010, SIGIR.

[29]  Chuang Lin,et al.  Graph Regularized Nonnegative Matrix Factorization with Sparse Coding , 2015 .

[30]  Xin Gao,et al.  Adaptive graph regularized Nonnegative Matrix Factorization via feature selection , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[31]  Yong Wang,et al.  Low-Rank Matrix Factorization With Adaptive Graph Regularizer , 2016, IEEE Transactions on Image Processing.

[32]  Jane You,et al.  Low-rank matrix factorization with multiple Hypergraph regularizer , 2015, Pattern Recognit..

[33]  Chun Chen,et al.  EMR: A Scalable Graph-Based Ranking Model for Content-Based Image Retrieval , 2015, IEEE Transactions on Knowledge and Data Engineering.

[34]  Hai Jin,et al.  Content-Based Visual Landmark Search via Multimodal Hypergraph Learning , 2015, IEEE Transactions on Cybernetics.

[35]  James Allan,et al.  Regression Rank: Learning to Meet the Opportunity of Descriptive Queries , 2009, ECIR.

[36]  W. Bruce Croft,et al.  Discovering key concepts in verbose queries , 2008, SIGIR '08.

[37]  Fuming Sun,et al.  Graph regularized and sparse nonnegative matrix factorization with hard constraints for data representation , 2016, Neurocomputing.

[38]  Wei Liu,et al.  Large Graph Construction for Scalable Semi-Supervised Learning , 2010, ICML.

[39]  Fei Su,et al.  Efficient multi-modal hypergraph learning for social image classification with complex label correlations , 2016, Neurocomputing.

[40]  W. Bruce Croft,et al.  Parameterized concept weighting in verbose queries , 2011, SIGIR.

[41]  Meng Wang,et al.  Scalable Semi-Supervised Learning by Efficient Anchor Graph Regularization , 2016, IEEE Transactions on Knowledge and Data Engineering.

[42]  Hongxun Yao,et al.  Affective Image Retrieval via Multi-Graph Learning , 2014, ACM Multimedia.

[43]  Yi Yang,et al.  A Multimedia Retrieval Framework Based on Semi-Supervised Ranking and Relevance Feedback , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[44]  Lin Wu,et al.  Efficient image and tag co-ranking: a bregman divergence optimization method , 2013, ACM Multimedia.

[45]  Trevor Darrell,et al.  DeCAF: A Deep Convolutional Activation Feature for Generic Visual Recognition , 2013, ICML.

[46]  Xiaofeng Zhu,et al.  Multi-view multi-sparsity kernel reconstruction for multi-class image classification , 2015, Neurocomputing.

[47]  H. Sebastian Seung,et al.  Learning the parts of objects by non-negative matrix factorization , 1999, Nature.

[48]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[49]  Yue Gao,et al.  Beyond Text QA: Multimedia Answer Generation by Harvesting Web Information , 2013, IEEE Transactions on Multimedia.

[50]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[51]  H. Sebastian Seung,et al.  Algorithms for Non-negative Matrix Factorization , 2000, NIPS.

[52]  Yueting Zhuang,et al.  A locally weighted sparse graph regularized Non-Negative Matrix Factorization method , 2015, Neurocomputing.

[53]  John W. Fisher,et al.  Loopy Belief Propagation: Convergence and Effects of Message Errors , 2005, J. Mach. Learn. Res..

[54]  James Allan,et al.  Effective and efficient user interaction for long queries , 2008, SIGIR '08.