Tag-based Video Retrieval with Social Tag Relevance Learning

High-quality tags play an important role in many applications such as multimedia information retrieval. This paper proposes a social tag relevance learning method using a data-driven approach to improving tag-based video retrieval performance. The tag relevance means how a tag is relevant to multimedia content. To learn the tag relevance, we apply a well-known tag neighbor voting algorithm, which accumulates votes from visual neighbors. However, an imbalance in the number of tags among the datasets causes a loss in the accuracy of tag voting. Therefore, in the proposed method, we examine a formula for calculating the tag relevance score considering the tag occurrence frequency imbalance. We conduct experiments on the YouTube-8M dataset, and the results show that our approach is effective and efficient.

[1]  Chaoran Cui,et al.  Social tag relevance learning via ranking-oriented neighbor voting , 2016, Multimedia Tools and Applications.

[2]  Hugo Zaragoza,et al.  The Probabilistic Relevance Framework: BM25 and Beyond , 2009, Found. Trends Inf. Retr..

[3]  Apostol Natsev,et al.  YouTube-8M: A Large-Scale Video Classification Benchmark , 2016, ArXiv.

[4]  Marcel Worring,et al.  Learning Social Tag Relevance by Neighbor Voting , 2009, IEEE Transactions on Multimedia.

[5]  Yongdong Zhang,et al.  Web video retagging , 2011, Multimedia Tools and Applications.

[6]  Alberto Del Bimbo,et al.  Data-driven approaches for social image and video tagging , 2015, Multimedia Tools and Applications.