Metadata Connector: Exploiting Hashtag and Tag for Cross-OSN Event Search

Social media has revolutionized the way people understand and keep track of real-world events. Various related multimedia information in different modalities such as texts, images and videos is updated on social media and reflects the events. These quantities of information distributes on different Online Social Networks (OSNs), which provides rich, wide coverage, comprehensive information about the trending events. Faced with such large amounts of data, searching has become a handy tool for event understanding and tracking on social media. However, existing single-OSN search mainly involves with single modality on single platform. Moreover, most OSNs usually focus on biased perspective of events, which significantly limits the coverage and diversity of single-OSN based event search. In this paper, we introduce a novel cross-OSN framework to help integrate these cross-OSN information regarding the same event and provide an immersive experience for information retrieval. Since social media information is widely distributed in different OSNs where semantic gap exists among these heterogeneous spaces, we propose to utilize hashtag and tag, which are user-generated metadata for organizing and labeling in many OSNs, as bridges to connect between different OSNs. In our four-stage solution framework, various methods are adopted for hashtag and tag filtering, search results representation, clustering and demonstration. Given an event query, in the first stage we generate related items with corresponding tags and hashtags from OSNs and filter the hashtags and tags we need. Then, topical representation is generated for hashtag and tag. The third stage leverages the derived representation for cross-OSN hashtag and tag clustering. Finally, demonstration for each query is produced and the results are organized hierarchically. Experiments on a dataset containing hundreds of search queries and related items demonstrate the effectiveness of our cross-OSN event search framework.

[1]  Changsheng Xu,et al.  Cross-OSN User Modeling by Homogeneous Behavior Quantification and Local Social Regularization , 2015, IEEE Transactions on Multimedia.

[2]  Nikos Pelekis,et al.  DataStories at SemEval-2017 Task 4: Deep LSTM with Attention for Message-level and Topic-based Sentiment Analysis , 2017, *SEMEVAL.

[3]  Inderjit S. Dhillon,et al.  A generalized maximum entropy approach to bregman co-clustering and matrix approximation , 2004, J. Mach. Learn. Res..

[4]  QianXueming,et al.  Tag-Based Image Search by Social Re-ranking , 2016 .

[5]  Martha Larson,et al.  Tags as bridges between domains: improving recommendation with tag-induced cross-domain collaborative filtering , 2011, UMAP'11.

[6]  Yongfeng Huang,et al.  Twitter100k: A Real-World Dataset for Weakly Supervised Cross-Media Retrieval , 2017, IEEE Transactions on Multimedia.

[7]  Judit Bar-Ilan,et al.  Methods for comparing rankings of search engine results , 2005, Comput. Networks.

[8]  Amy Nicole Langville,et al.  A Survey of Eigenvector Methods for Web Information Retrieval , 2005, SIAM Rev..

[9]  Dong Liu,et al.  Tag ranking , 2009, WWW '09.

[10]  Michael I. Jordan,et al.  On Spectral Clustering: Analysis and an algorithm , 2001, NIPS.

[11]  Yanjun Zhang,et al.  Equal Representation by Search Engines? A Comparison of Websites across Countries and Domains , 2007, J. Comput. Mediat. Commun..

[12]  Shih-Fu Chang,et al.  Video search reranking through random walk over document-level context graph , 2007, ACM Multimedia.

[13]  Joydeep Ghosh,et al.  Cluster Ensembles --- A Knowledge Reuse Framework for Combining Multiple Partitions , 2002, J. Mach. Learn. Res..

[14]  Nick Barnes,et al.  Deep0Tag: Deep Multiple Instance Learning for Zero-Shot Image Tagging , 2020, IEEE Transactions on Multimedia.

[15]  Thomas L. Griffiths,et al.  Hierarchical Topic Models and the Nested Chinese Restaurant Process , 2003, NIPS.

[16]  Ying Yang,et al.  Personalized Recommendation of Social Images by Constructing a User Interest Tree With Deep Features and Tag Trees , 2019, IEEE Transactions on Multimedia.

[17]  Changsheng Xu,et al.  Social Multimedia Ming: From Special to General , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[18]  Inderjit S. Dhillon,et al.  Clustering with Bregman Divergences , 2005, J. Mach. Learn. Res..

[19]  Changsheng Xu,et al.  Hashtag-centric Immersive Search on Social Media , 2017, ACM Multimedia.

[20]  Lei Yang,et al.  We know what @you #tag: does the dual role affect hashtag adoption? , 2012, WWW.

[21]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[22]  Changsheng Xu,et al.  Personalized Celebrity Video Search Based on Cross-Space Mining , 2012, PCM.

[23]  M. Shamim Hossain,et al.  YouTube Video Promotion by Cross-Network Association: @Britney to Advertise Gangnam Style , 2015, IEEE Transactions on Multimedia.

[24]  Francesco Ricci,et al.  Cold-Start Management with Cross-Domain Collaborative Filtering and Tags , 2013, EC-Web.

[25]  Tao Mei,et al.  SocialTransfer: cross-domain transfer learning from social streams for media applications , 2012, ACM Multimedia.

[26]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[27]  Bernhard Schölkopf,et al.  Learning with Local and Global Consistency , 2003, NIPS.

[28]  Xueming Qian,et al.  Tag-Based Image Search by Social Re-ranking , 2016, IEEE Transactions on Multimedia.

[29]  Emily Denton,et al.  User Conditional Hashtag Prediction for Images , 2015, KDD.

[30]  Gert R. G. Lanckriet,et al.  User-centered design of a social game to tag music , 2009, HCOMP '09.

[31]  Beth Logan,et al.  Speechbot: an experimental speech-based search engine for multimedia content on the web , 2002, IEEE Trans. Multim..

[32]  Mor Naaman,et al.  Why we tag: motivations for annotation in mobile and online media , 2007, CHI.

[33]  Hongfei Yan,et al.  Comparing Twitter and Traditional Media Using Topic Models , 2011, ECIR.

[34]  Winston H. Hsu,et al.  Learning From Cross-Domain Media Streams for Event-of-Interest Discovery , 2018, IEEE Transactions on Multimedia.

[35]  R. Graham,et al.  Spearman's Footrule as a Measure of Disarray , 1977 .

[36]  M. Shamim Hossain,et al.  Cross-Platform Emerging Topic Detection and Elaboration from Multimedia Streams , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[37]  Rajeev Motwani,et al.  The PageRank Citation Ranking : Bringing Order to the Web , 1999, WWW 1999.

[38]  Adrian Popescu,et al.  Social media driven image retrieval , 2011, ICMR.

[39]  Chong-Wah Ngo,et al.  Hierarchical Visualization of Video Search Results for Topic-Based Browsing , 2016, IEEE Transactions on Multimedia.

[40]  Changsheng Xu,et al.  Unified YouTube Video Recommendation via Cross-network Collaboration , 2015, ICMR.