Exploiting User Comments for Audio-Visual Content Indexing and Retrieval

State-of-the-art content sharing platforms often require users to assign tags to pieces of media in order to make them easily retrievable. Since this task is sometimes perceived as tedious or boring, annotations can be sparse. Commenting on the other hand is a frequently used means of expressing user opinion towards shared media items. This work makes use of time series analyses in order to infer potential tags and indexing terms for audio-visual content from user comments. In this way, we mitigate the vocabulary gap between queries and document descriptors. Additionally, we show how large-scale encyclopaedias such as Wikipedia can aid the task of tag prediction by serving as surrogates for high-coverage natural language vocabulary lists. Our evaluation is conducted on a corpus of several million real-world user comments from the popular video sharing platform YouTube, and demonstrates significant improvements in retrieval performance.

[1]  Nenghai Yu,et al.  Distance metric learning from uncertain side information with application to automated photo tagging , 2009, ACM Multimedia.

[2]  Gabriella Kazai,et al.  In Search of Quality in Crowdsourcing for Search Engine Evaluation , 2011, ECIR.

[3]  Jiangchuan Liu,et al.  Understanding the Characteristics of Internet Short Video Sharing: YouTube as a Case Study , 2007, ArXiv.

[4]  Ee-Peng Lim,et al.  Comments-oriented blog summarization by sentence extraction , 2007, CIKM '07.

[5]  Gilad Mishne,et al.  Leave a Reply: An Analysis of Weblog Comments , 2006 .

[6]  Stephen E. Robertson,et al.  Simple BM25 extension to multiple weighted fields , 2004, CIKM '04.

[7]  Karl Aberer,et al.  Neighborhood-Based Tag Prediction , 2009, ESWC.

[8]  Djoerd Hiemstra,et al.  A probabilistic justification for using tf×idf term weighting in information retrieval , 2000, International Journal on Digital Libraries.

[9]  M. de Rijke,et al.  Predicting IMDB Movie Ratings Using Social Media , 2012, ECIR.

[10]  Matthew Hurst,et al.  A Language Model Approach to Keyphrase Extraction , 2003, ACL 2003.

[11]  Padmini Srinivasan,et al.  Quality through flow and immersion: gamifying crowdsourced relevance assessments , 2012, SIGIR '12.

[12]  Lora Aroyo,et al.  The Semantic Web: Research and Applications , 2009, Lecture Notes in Computer Science.

[13]  Thierry Bertin-Mahieux,et al.  Automatic Generation of Social Tags for Music Recommendation , 2007, NIPS.

[14]  Peter Ingwersen,et al.  Developing a Test Collection for the Evaluation of Integrated Search , 2010, ECIR.

[15]  Christian Wartena,et al.  Keyword extraction using co-occurrence. : IEEE Proceedings of the 7th International Workshop on Text-based Information Retrieval (TIR-10), DEXA 2010 Bilbao, Spain. , 2010 .

[16]  Ophir Frieder,et al.  Are Web User Comments Useful for Search? , 2009, LSDS-IR@SIGIR.

[17]  Mark Sanderson,et al.  Automatic video tagging using content redundancy , 2009, SIGIR.

[18]  and software — performance evaluation , .

[19]  Mohammad Soleymani,et al.  Automatic tagging and geotagging in video collections and communities , 2011, ICMR.

[20]  Hector Garcia-Molina,et al.  Social tag prediction , 2008, SIGIR '08.

[21]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[22]  Keith B. Hall,et al.  Improved video categorization from text metadata and user comments , 2011, SIGIR '11.

[23]  James P. Callan,et al.  Combining document representations for known-item search , 2003, SIGIR.

[24]  Giorgio Gambosi,et al.  On relevance, time and query expansion , 2011, CIKM '11.

[25]  Mitsuru Ishizuka,et al.  Keyword extraction from a single document using word co-occurrence statistical information , 2004, Int. J. Artif. Intell. Tools.