Social-oriented visual image search

Many research have been focusing on how to match the textual query with visual images and their surrounding texts or tags for Web image search. The returned results are often unsatisfactory due to their deviation from user intentions, particularly for queries with heterogeneous concepts (such as ''apple'', ''jaguar'') or general (non-specific) concepts (such as ''landscape'', ''hotel''). In this paper, we exploit social data from social media platforms to assist image search engines, aiming to improve the relevance between returned images and user intentions (i.e., social relevance). Facing the challenges of social data sparseness, the tradeoff between social relevance and visual relevance, and the complex social and visual factors, we propose a community-specific Social-Visual Ranking (SVR) algorithm to rerank the Web images returned by current image search engines. The SVR algorithm is implemented by PageRank over a hybrid image link graph, which is the combination of an image social-link graph and an image visual-link graph. By conducting extensive experiments, we demonstrated the importance of both visual factors and social factors, and the advantages of social-visual ranking algorithm for Web image search.

[1]  Yi Yang,et al.  Harmonizing Hierarchical Manifolds for Multimedia Document Semantics Understanding and Cross-Media Retrieval , 2008, IEEE Transactions on Multimedia.

[2]  Qi Tian,et al.  Latent visual context learning for web image applications , 2011, Pattern Recognit..

[3]  Gang Hua,et al.  Descriptive visual words and visual phrases for image applications , 2009, ACM Multimedia.

[4]  Tao Mei,et al.  Modeling social strength in social media community via kernel-based learning , 2011, ACM Multimedia.

[5]  Xian-Sheng Hua,et al.  Video search re-ranking via multi-graph propagation , 2007, ACM Multimedia.

[6]  Kotagiri Ramamohanarao,et al.  Mining web multi-resolution community-based popularity for information retrieval , 2007, CIKM '07.

[7]  Li Wang,et al.  Query aware visual similarity propagation for image search reranking , 2009, MM '09.

[8]  Andrei Z. Broder,et al.  Graph structure in the Web , 2000, Comput. Networks.

[9]  Dong Liu,et al.  Hybrid social media network , 2012, ACM Multimedia.

[10]  Shih-Fu Chang,et al.  Video search reranking via information bottleneck principle , 2006, MM '06.

[11]  Xian-Sheng Hua,et al.  The role of attractiveness in web image search , 2011, ACM Multimedia.

[12]  Yi Yang,et al.  A Multimedia Retrieval Framework Based on Semi-Supervised Ranking and Relevance Feedback , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[13]  Sourav S. Bhowmick,et al.  Quantifying tag representativeness of visual content of social images , 2010, ACM Multimedia.

[14]  Risi Kondor,et al.  Diffusion kernels on graphs and other discrete structures , 2002, ICML 2002.

[15]  Xian-Sheng Hua,et al.  Bayesian video search reranking , 2008, ACM Multimedia.

[16]  Taher H. Haveliwala.  Topic-Sensitive PageRank: A Context-Sensitive Ranking Algorithm for Web Search , 2003, IEEE Trans. Knowl. Data Eng..

[17]  Changsheng Xu,et al.  Exploiting user information for image tag refinement , 2011, MM '11.

[18]  John D. Lafferty,et al.  Diffusion Kernels on Graphs and Other Discrete Input Spaces , 2002, ICML.

[19]  Jaana Kekäläinen,et al.  IR evaluation methods for retrieving highly relevant documents , 2000, SIGIR '00.

[20]  Shih-Fu Chang,et al.  Video search reranking through random walk over document-level context graph , 2007, ACM Multimedia.

[21]  Shuicheng Yan,et al.  SIFT-Bag kernel for video event analysis , 2008, ACM Multimedia.

[22]  Soumen Chakrabarti,et al.  Dynamic personalized pagerank in entity-relation graphs , 2007, WWW '07.

[23]  Daniel Gatica-Perez,et al.  Analyzing Flickr groups , 2008, CIVR '08.

[24]  Martha Larson,et al.  Reading between the tags to predict real-world size-class for visually depicted objects in images , 2011, MM '11.

[25]  Chun Chen,et al.  Music recommendation by unified hypergraph: combining social media information and music content , 2010, ACM Multimedia.

[26]  Shumeet Baluja,et al.  VisualRank: Applying PageRank to Large-Scale Image Search , 2008, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[27]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[28]  Pinar Duygulu Sahin,et al.  Re-ranking of web image search results using a graph algorithm , 2008, 2008 19th International Conference on Pattern Recognition.

[29]  Rong Yan,et al.  Multimedia Search with Pseudo-relevance Feedback , 2003, CIVR.

[30]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[31]  Yi-Hsuan Yang,et al.  ContextSeer: context search and recommendation at query time for shared consumer photos , 2008, ACM Multimedia.

[32]  Marcel Worring,et al.  Learning tag relevance by neighbor voting for social image retrieval , 2008, MIR '08.

[33]  David Nistér,et al.  Scalable Recognition with a Vocabulary Tree , 2006, 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR'06).