Diversifying image search with user generated content

Large-scale image retrieval on the Web relies on the availability of short snippets of text associated with the image. This user-generated content is a primary source of information about the content and context of an image. While traditional information retrieval models focus on finding the most relevant document without consideration for diversity, image search requires results that are both diverse and relevant. This is problematic for images because they are represented very sparsely by text, and as with all user-generated content the text for a given image can be extremely noisy. The contribution of this paper is twofold. First, we present a retrieval model which provides diverse results as a property of the model itself, rather than in a post-retrieval step. Relevance models offer a unified framework to afford the greatest diversity without harming precision. Second, we show that it is possible to minimize the trade-offs between precision and diversity, and estimating the query model from the distribution of tags favors the dominant sense of a query. Relevance models operating only on tags offers the highest level of diversity with no significant decrease in precision.

[1]  Simone Santini,et al.  Exploratory Image Databases: Content-Based Retrieval , 2001 .

[2]  Donna K. Harman,et al.  Overview of the TREC 2002 Novelty Track , 2002, TREC.

[3]  W. Bruce Croft,et al.  Cross-lingual relevance models , 2002, SIGIR '02.

[4]  R. Manmatha,et al.  A Model for Learning the Semantics of Pictures , 2003, NIPS.

[5]  R. Manmatha,et al.  Automatic image annotation and retrieval using cross-media relevance models , 2003, SIGIR.

[6]  Donna K. Harman,et al.  Overview of the TREC 2003 Novelty Track , 2003, TREC.

[7]  Ian Soboroff,et al.  Overview of the TREC 2004 Novelty Track , 2004, TREC.

[8]  Djemel Ziou,et al.  Image Retrieval from the World Wide Web: Issues, Techniques, and Systems , 2004, CSUR.

[9]  Hua Li,et al.  Improving web search results using affinity graph , 2005, SIGIR '05.

[10]  W. Bruce Croft,et al.  A Translation Model for Sentence Retrieval , 2005, HLT.

[11]  Sean M. McNee,et al.  Improving recommendation lists through topic diversification , 2005, WWW '05.

[12]  Kai Song,et al.  Diversifying the image retrieval results , 2006, MM '06.

[13]  Fernando Diaz,et al.  Improving the estimation of relevance models using large external corpora , 2006, SIGIR.

[14]  Nicu Sebe,et al.  Content-based multimedia information retrieval: State of the art and challenges , 2006, TOMCCAP.

[15]  Mor Naaman,et al.  HT06, tagging paper, taxonomy, Flickr, academic article, to read , 2006, HYPERTEXT '06.

[16]  Vanessa Murdock,et al.  Aspects of sentence retrieval , 2007, SIGF.

[17]  James Ze Wang,et al.  Image retrieval: Ideas, influences, and trends of the new age , 2008, CSUR.

[18]  Roelof van Zwol,et al.  Flickr tag recommendation based on collective knowledge , 2008, WWW.