An Attribute-Assisted Reranking Model for Web Image Search

Image search reranking is an effective approach to refine the text-based image search result. Most existing reranking approaches are based on low-level visual features. In this paper, we propose to exploit semantic attributes for image search reranking. Based on the classifiers for all the predefined attributes, each image is represented by an attribute feature consisting of the responses from these classifiers. A hypergraph is then used to model the relationship between images by integrating low-level visual features and attribute features. Hypergraph ranking is then performed to order the images. Its basic principle is that visually similar images should have similar ranking scores. In this paper, we propose a visual-attribute joint hypergraph learning approach to simultaneously explore two information sources. A hypergraph is constructed to model the relationship of all images. We conduct experiments on more than 1,000 queries in MSRA-MMV2.0 data set. The experimental results demonstrate the effectiveness of our approach.

[1]  Kristen Grauman,et al.  Beyond Comparing Image Pairs: Setwise Active Learning for Relative Attributes , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Xiaogang Wang,et al.  IntentSearch: Capturing User Intention for One-Click Internet Image Search , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[3]  Alan Hanjalic,et al.  Supervised reranking for web image search , 2010, ACM Multimedia.

[4]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR Forum.

[5]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[6]  Frédéric Jurie,et al.  Improving Image Classification Using Semantic Attributes , 2012, International Journal of Computer Vision.

[7]  GaoYue,et al.  Attribute-Augmented Semantic Hierarchy , 2014 .

[8]  Shree K. Nayar,et al.  FaceTracer: A Search Engine for Large Collections of Images with Faces , 2008, ECCV.

[9]  Rogério Schmidt Feris,et al.  Attribute-based people search in surveillance environments , 2009, 2009 Workshop on Applications of Computer Vision (WACV).

[10]  Xian-Sheng Hua,et al.  Towards a Relevant and Diverse Search of Social Images , 2010, IEEE Transactions on Multimedia.

[11]  Dragomir R. Radev,et al.  Edge Weight Regularization over Multiple Graphs for Similarity Learning , 2010, 2010 IEEE International Conference on Data Mining.

[12]  Yang Wang,et al.  A Discriminative Latent Model of Object Classes and Attributes , 2010, ECCV.

[13]  Shih-Fu Chang,et al.  Label diagnosis through self tuning for web image search , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[14]  Antonio Criminisi,et al.  Harvesting Image Databases from the Web , 2007, 2007 IEEE 11th International Conference on Computer Vision.

[15]  Yue Gao,et al.  Tag-based social image search with visual-text joint hypergraph learning , 2011, ACM Multimedia.

[16]  Meng Wang,et al.  Adaptive Hypergraph Learning and its Application in Image Classification , 2012, IEEE Transactions on Image Processing.

[17]  Shengjin Wang,et al.  Visual Phraselet: Refining Spatial Constraints for Large Scale Image Search , 2013, IEEE Signal Processing Letters.

[18]  Nicu Sebe,et al.  Image retrieval using wavelet-based salient points , 2001, J. Electronic Imaging.

[19]  Jing Wang,et al.  Robust object removal with an exemplar-based image inpainting approach , 2014, Neurocomputing.

[20]  Shree K. Nayar,et al.  Attribute and simile classifiers for face verification , 2009, 2009 IEEE 12th International Conference on Computer Vision.

[21]  Shiliang Zhang,et al.  Affective Visualization and Retrieval for Music Video , 2010, IEEE Transactions on Multimedia.

[22]  Larry S. Davis,et al.  Image ranking and retrieval based on multi-attribute queries , 2011, CVPR 2011.

[23]  Cordelia Schmid,et al.  Combining attributes and Fisher vectors for efficient image retrieval , 2011, CVPR 2011.

[24]  Tao Mei,et al.  Optimizing Visual Search Reranking via Pairwise Learning , 2011, IEEE Transactions on Multimedia.

[25]  Zheng-Jun Zha,et al.  Evaluation of histogram based interest point detector in web image classification and search , 2010, 2010 IEEE International Conference on Multimedia and Expo.

[26]  Jingdong Wang,et al.  Robust visual reranking via sparsity and ranking constraints , 2011, ACM Multimedia.

[27]  Yue Gao,et al.  When Amazon Meets Google: Product Visualization by Exploring Multiple Web Sources , 2013, TOIT.

[28]  Rongrong Ji,et al.  Weak attributes for large-scale image retrieval , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[29]  Qi Tian,et al.  Multimedia search reranking: A literature survey , 2014, CSUR.

[30]  Shiliang Zhang,et al.  Learning attribute-aware dictionary for image classification and search , 2013, ICMR.

[31]  Xian-Sheng Hua,et al.  Bayesian video search reranking , 2008, ACM Multimedia.

[32]  Wei-Ying Ma,et al.  IGroup: web image search results clustering , 2006, MM '06.

[33]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Kristen Grauman,et al.  Relative attributes , 2011, 2011 International Conference on Computer Vision.

[35]  Yi-Hsuan Yang,et al.  Unsupervised auxiliary visual words discovery for large-scale image object retrieval , 2011, CVPR 2011.

[36]  Christoph H. Lampert,et al.  Learning to detect unseen object classes by between-class attribute transfer , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[37]  Xian-Sheng Hua,et al.  MSRA-MM: Bridging Research and Industrial Societies for Multimedia Information Retrieval , 2009 .

[38]  Ali Farhadi,et al.  Describing objects by their attributes , 2009, 2009 IEEE Conference on Computer Vision and Pattern Recognition.

[39]  Bernhard Schölkopf,et al.  Learning with Hypergraphs: Clustering, Classification, and Embedding , 2006, NIPS.

[40]  Adriana Kovashka,et al.  WhittleSearch: Image search with relative attribute feedback , 2012, 2012 IEEE Conference on Computer Vision and Pattern Recognition.

[41]  Shih-Fu Chang,et al.  Video search reranking via information bottleneck principle , 2006, MM '06.

[42]  Qi Tian,et al.  Attribute-assisted reranking for web image retrieval , 2012, ACM Multimedia.

[43]  Qingshan Liu,et al.  Image retrieval via probabilistic hypergraph ranking , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[44]  Rong Yan,et al.  Multimedia Search with Pseudo-relevance Feedback , 2003, CIVR.

[45]  Kristen Grauman,et al.  Interactively building a discriminative vocabulary of nameable attributes , 2011, CVPR 2011.

[46]  Ning He,et al.  Content-based similarity for 3D model retrieval and classification , 2009 .

[47]  Yue Gao,et al.  3-D Object Retrieval and Recognition With Hypergraph Analysis , 2012, IEEE Transactions on Image Processing.

[48]  Qi Tian,et al.  Semi-automatic Flickr Group Suggestion , 2011, MMM.

[49]  Xian-Sheng Hua,et al.  Bayesian Visual Reranking , 2011, IEEE Transactions on Multimedia.

[50]  Yue Gao,et al.  Attribute-augmented semantic hierarchy: towards bridging semantic gap and intention gap in image retrieval , 2013, ACM Multimedia.