Long-tail Hashtag Recommendation for Micro-videos with Graph Convolutional Network

Hashtags, a user provides to a micro-video, are the ones which can well describe the semantics of the micro-video's content in his/her mind. At the same time, hashtags have been widely used to facilitate various micro-video retrieval scenarios (e.g., search, browse, and categorization). Despite their importance, numerous micro-videos lack hashtags or contain inaccurate or incomplete hashtags. In light of this, hashtag recommendation, which suggests a list of hashtags to a user when he/she wants to annotate a post, becomes a crucial research problem. However, little attention has been paid to micro-video hashtag recommendation, mainly due to the following three reasons: 1) lack of benchmark dataset; 2) the temporal and multi-modality characteristics of micro-videos; and 3) hashtag sparsity and long-tail distributions. In this paper, we recommend hashtags for micro-videos by presenting a novel multi-view representation interactive embedding model with graph-based information propagation. It is capable of boosting the performance of micro-videos hashtag recommendation by jointly considering the sequential feature learning, the video-user-hashtag interaction, and the hashtag correlations. Extensive experiments on a constructed dataset demonstrate our proposed method outperforms state-of-the-art baselines. As a side research contribution, we have released our dataset and codes to facilitate the research in this community.

[1]  Qiang Li,et al.  Conditional Graphical Lasso for Multi-label Image Classification , 2016, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]  Xuanjing Huang,et al.  Learning Topical Translation Model for Microblog Hashtag Suggestion , 2013, IJCAI.

[3]  Serge J. Belongie,et al.  Separating Self-Expression and Visual Content in Hashtag Supervision , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[4]  Xuanjing Huang,et al.  Hashtag Recommendation for Multimodal Microblog Using Co-Attention Network , 2017, IJCAI.

[5]  Dragomir Anguelov,et al.  Capturing Long-Tail Distributions of Object Subcategories , 2014, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[6]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[7]  Joshua B. Tenenbaum,et al.  Learning to share visual appearance for multiclass object detection , 2011, CVPR 2011.

[8]  Xuanjing Huang,et al.  Hashtag Recommendation Using Dirichlet Process Mixture Models Incorporating Types of Hashtags , 2015, EMNLP.

[9]  Tat-Seng Chua,et al.  Shorter-is-Better: Venue Category Estimation from Micro-Video , 2016, ACM Multimedia.

[10]  Mohan S. Kankanhalli,et al.  Temporal encoded F-formation system for social interaction detection , 2013, ACM Multimedia.

[11]  Marjan Kaedi,et al.  Recommending the long tail items through personalized diversification , 2019, Knowl. Based Syst..

[12]  Wei-Ying Ma,et al.  Hashtag-Based Sub-Event Discovery Using Mutually Generative LDA in Twitter , 2016, AAAI.

[13]  Martha Palmer,et al.  Verb Semantics and Lexical Selection , 1994, ACL.

[14]  Miles Efron,et al.  Hashtag retrieval in a microblogging environment , 2010, SIGIR.

[15]  Li Peng,et al.  A Capsule Network for Recommendation and Explaining What You Like and Dislike , 2019, SIGIR.

[16]  Jane Yung-jen Hsu,et al.  Who likes it more?: mining worth-recommending items from long tails by modeling relative preference , 2014, WSDM.

[17]  Johannes Gehrke,et al.  Hashtag Recommendation for Enterprise Applications , 2016, CIKM.

[18]  Mohan S. Kankanhalli,et al.  A Multi-sensor Framework for Personal Presentation Analytics , 2019, ACM Trans. Multim. Comput. Commun. Appl..

[19]  Changsheng Xu,et al.  Hashtag-centric Immersive Search on Social Media , 2017, ACM Multimedia.

[20]  Qi Zhang,et al.  Hashtag Recommendation Using Attention-Based Convolutional Neural Network , 2016, IJCAI.

[21]  Niloy Ganguly,et al.  LMPP: A Large Margin Point Process Combining Reinforcement and Competition for Modeling Hashtag Popularity , 2017, IJCAI.

[22]  Ee-Peng Lim,et al.  On Recommending Hashtags in Twitter Networks , 2012, SocInfo.

[23]  Junmo Kim,et al.  HARRISON: A Benchmark on HAshtag Recommendation for Real-world Images in Social Networks , 2016, ArXiv.

[24]  Baoxin Li,et al.  CLARE: A Joint Approach to Label Classification and Tag Recommendation , 2017, AAAI.

[25]  Lei Shi,et al.  Trading-off among accuracy, similarity, diversity, and long-tail: a graph-based recommendation approach , 2013, RecSys.

[26]  Kiyoharu Aizawa,et al.  FolkPopularityRank: Tag Recommendation for Enhancing Social Popularity using Text Tags in Content Sharing Services , 2017, IJCAI.

[27]  Yang Li,et al.  Hashtag Recommendation with Topical Attention-Based LSTM , 2016, COLING.

[28]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[29]  Mohan S. Kankanhalli,et al.  Aspect-Aware Latent Factor Model: Rating Prediction with Ratings and Reviews , 2018, WWW.

[30]  Xin Li,et al.  Multi-label Image Classification with A Probabilistic Label Enhancement Model , 2014, UAI.

[31]  Xiu-Shen Wei,et al.  Multi-Label Image Recognition With Graph Convolutional Networks , 2019, 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).

[32]  Mohan S. Kankanhalli,et al.  MMALFM , 2018, ACM Trans. Inf. Syst..

[33]  Max Welling,et al.  Semi-Supervised Classification with Graph Convolutional Networks , 2016, ICLR.

[34]  Mohan S. Kankanhalli,et al.  A^3NCF: An Adaptive Aspect Attention Model for Rating Prediction , 2018, IJCAI.

[35]  Mohan S. Kankanhalli,et al.  Multi-sensor Self-Quantification of Presentations , 2015, ACM Multimedia.

[36]  Liqiang Nie,et al.  Neural Multimodal Cooperative Learning Toward Micro-Video Understanding , 2020, IEEE Transactions on Image Processing.

[37]  Yu-Chiang Frank Wang,et al.  Multi-label Zero-Shot Learning with Structured Knowledge Graphs , 2017, 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition.

[38]  Tat-Seng Chua,et al.  Neural Collaborative Filtering , 2017, WWW.

[39]  Tat-Seng Chua,et al.  Micro Tells Macro: Predicting the Popularity of Micro-Videos via a Transductive Model , 2016, ACM Multimedia.

[40]  Emily Denton,et al.  User Conditional Hashtag Prediction for Images , 2015, KDD.

[41]  Tian Gan,et al.  Sentiment Analysis for Social Sensor , 2017, PCM.

[42]  Nenghai Yu,et al.  Learning to tag , 2009, WWW '09.

[43]  Xiaolong Wang,et al.  Topic sentiment analysis in twitter: a graph-based hashtag sentiment classification approach , 2011, CIKM '11.

[44]  Cees Snoek,et al.  Exploring the Long Tail of Social Media Tags , 2016, MMM.

[45]  Meng Wang,et al.  Towards Micro-video Understanding by Joint Sequential-Sparse Modeling , 2017, ACM Multimedia.

[46]  Yoon-Joo Park,et al.  The Adaptive Clustering Method for the Long Tail Problem of Recommender Systems , 2013, IEEE Transactions on Knowledge and Data Engineering.

[47]  Jie Liu,et al.  Hashtag2Vec: Learning Hashtag Representation with Relational Hierarchical Embedding Model , 2018, IJCAI.

[48]  Mohan S. Kankanhalli,et al.  ConTagNet: Exploiting User Context for Image Tag Recommendation , 2016, ACM Multimedia.