A survey of event analysis and mining from social multimedia

In recent years, with the popularity of mobile devices and mobile Internet, more and more social media sites are growing in an explosive way. Therefore, the social hot event will be rapidly fermented by the interaction of a large number of network users, and a large amount of multimedia data (such as texts, images and videos) will be generated. Therefore, it is important and necessary to conduct the research of multimedia social event analysis to know the evolutionary trend of social event over time automatically. This paper provides a survey and summarizes major progresses in multimedia social event analysis. We focus on four areas: (1) multimedia social event representation; (2) multimedia social event detection and tracking; (3) multimedia social event evolutionary analysis; and (4) multimedia social event topic mining.

[1]  Yiming Yang,et al.  Topic-conditioned novelty detection , 2002, KDD.

[2]  Michael I. Jordan,et al.  Modeling annotated data , 2003, SIGIR.

[3]  Jianwen Zhang,et al.  Evolutionary hierarchical dirichlet processes for multiple correlated time-varying corpora , 2010, KDD.

[4]  Changsheng Xu,et al.  Multi-modal Multi-view Topic-opinion Mining for Social Event Analysis , 2016, ACM Multimedia.

[5]  Rohini K. Srihari,et al.  Graph-based text representation and knowledge discovery , 2007, SAC '07.

[6]  U. Berkeley.  Exploring Content Models for Multi-Document Summarization , 2018 .

[7]  Cordelia Schmid,et al.  Multimodal semi-supervised learning for image classification , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[8]  Gang Hua,et al.  Semantic Model Vectors for Complex Video Event Recognition , 2012, IEEE Transactions on Multimedia.

[9]  Wolfgang Nejdl,et al.  Bringing order to your photos: event-driven classification of flickr images based on social knowledge , 2010, CIKM.

[10]  Long Zhu,et al.  A Hybrid Neural Network-Latent Topic Model , 2012, AISTATS.

[11]  Henri Theil,et al.  Relations between two sets of variates: The bits of information provided by each variate in each set , 1988 .

[12]  Xia Feng,et al.  Latent Dirichlet allocation (LDA) and topic modeling: models, applications, a survey , 2017, Multimedia Tools and Applications.

[13]  Véronique Prinet,et al.  Towards Optimal Naive Bayes Nearest Neighbor , 2010, ECCV.

[14]  James Allan,et al.  Detection As Multi-Topic Tracking , 2002, Information Retrieval.

[15]  David D. Lewis,et al.  Naive (Bayes) at Forty: The Independence Assumption in Information Retrieval , 1998, ECML.

[16]  T. Landauer,et al.  Indexing by Latent Semantic Analysis , 1990 .

[17]  Gerard Salton,et al.  A vector space model for automatic indexing , 1975, CACM.

[18]  Kevin Lane Keller.  Conceptualizing, Measuring, and Managing Customer-Based Brand Equity , 1993 .

[19]  Changsheng Xu,et al.  Boosted Multi-modal Supervised Latent Dirichlet Allocation for Social Event Classification , 2014, 2014 22nd International Conference on Pattern Recognition.

[20]  Martin Ester,et al.  On the design of LDA models for aspect-based opinion mining , 2012, CIKM.

[21]  Luo Si,et al.  Mining contrastive opinions on political texts using cross-perspective topic model , 2012, WSDM '12.

[22]  David M. Blei,et al.  Supervised Topic Models , 2007, NIPS.

[23]  Prasenjit Mitra,et al.  Event detection with spatial latent Dirichlet allocation , 2011, JCDL '11.

[24]  Eric P. Xing,et al.  Dynamic Non-Parametric Mixture Models and the Recurrent Chinese Restaurant Process: with Applications to Evolutionary Clustering , 2008, SDM.

[25]  Roger Levy,et al.  A new approach to cross-modal multimedia retrieval , 2010, ACM Multimedia.

[26]  Richard A. Harshman,et al.  Indexing by latent semantic indexing , 1990 .

[27]  Hector Garcia-Molina,et al.  Clustering the tagged web , 2009, WSDM '09.

[28]  Vikas Sindhwani,et al.  Emerging topic detection using dictionary learning , 2011, CIKM '11.

[29]  Meng Wang,et al.  Multimodal Graph-Based Reranking for Web Image Search , 2012, IEEE Transactions on Image Processing.

[30]  Haixun Wang,et al.  Tracking and Connecting Topics via Incremental Hierarchical Dirichlet Processes , 2011, 2011 IEEE 11th International Conference on Data Mining.

[31]  Changsheng Xu,et al.  Right buddy makes the difference: an early exploration of social relation analysis in multimedia applications , 2012, ACM Multimedia.

[32]  Andrew McCallum,et al.  A comparison of event models for naive bayes text classification , 1998, AAAI 1998.

[33]  Arnaud Doucet,et al.  Generalized Polya Urn for Time-varying Dirichlet Process Mixtures , 2007, UAI.

[34]  Yun Chi,et al.  Evolutionary spectral clustering by incorporating temporal smoothness , 2007, KDD '07.

[35]  M. E. Maron,et al.  Automatic Indexing: An Experimental Inquiry , 1961, JACM.

[36]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.

[37]  Yun Liu,et al.  A Forecasting System of Micro-blog Public Opinion Based on Artificial Neural Network , 2014, 2014 Tenth International Conference on Intelligent Information Hiding and Multimedia Signal Processing.

[38]  Juhan Nam,et al.  Multimodal Deep Learning , 2011, ICML.

[39]  Lei Zhang,et al.  A Survey of Opinion Mining and Sentiment Analysis , 2012, Mining Text Data.

[40]  Jing Jiang,et al.  A Latent Variable Model for Viewpoint Discovery from Threaded Forum Posts , 2013, NAACL.

[41]  David B. Dunson,et al.  The dynamic hierarchical Dirichlet process , 2008, ICML '08.

[42]  Helena Ahonen-Myka,et al.  Simple Semantics in Topic Detection and Tracking , 2004, Information Retrieval.

[43]  Chong-Wah Ngo,et al.  Multimodal News Story Clustering With Pairwise Visual Near-Duplicate Constraint , 2008, IEEE Transactions on Multimedia.

[44]  Mark Steyvers,et al.  Finding scientific topics , 2004, Proceedings of the National Academy of Sciences of the United States of America.

[45]  Changsheng Xu,et al.  Multi-Modal Event Topic Model for Social Event Analysis , 2016, IEEE Transactions on Multimedia.

[46]  Gabriela Csurka,et al.  Visual categorization with bags of keypoints , 2002, eccv 2004.

[47]  Jianjun Yu,et al.  Towards Topic Trend Prediction on a Topic Evolution Model with Social Connection , 2012, 2012 IEEE/WIC/ACM International Conferences on Web Intelligence and Intelligent Agent Technology.

[48]  Jing Yu,et al.  Cross-modal topic correlations for multimedia retrieval , 2012, Proceedings of the 21st International Conference on Pattern Recognition (ICPR2012).

[49]  Martin Ester,et al.  ILDA: interdependent LDA model for learning latent aspects and their ratings from online product reviews , 2011, SIGIR.

[50]  Dimitrios Tzovaras,et al.  Multi-Objective Optimization for Multimodal Visualization , 2014, IEEE Transactions on Multimedia.

[51]  Tao Mei,et al.  Joint multi-label multi-instance learning for image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[52]  Changsheng Xu,et al.  Social Multimedia Ming: From Special to General , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[53]  Jun Zhang,et al.  A Unified Framework for Fine-Grained Opinion Mining from Online Reviews , 2016, 2016 49th Hawaii International Conference on System Sciences (HICSS).

[54]  Andrew McCallum,et al.  Group and topic discovery from relations and text , 2005, LinkKDD '05.

[55]  Nitish Srivastava,et al.  Multimodal learning with deep Boltzmann machines , 2012, J. Mach. Learn. Res..

[56]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[57]  Fabrizio Sebastiani,et al.  Supervised term weighting for automated text categorization , 2003, SAC '03.

[58]  Meng Wang,et al.  Learning Visual Semantic Relationships for Efficient Visual Retrieval , 2015, IEEE Transactions on Big Data.

[59]  Thorsten Joachims,et al.  Text categorization with support vector machines , 1999 .

[60]  Eli Shechtman,et al.  In defense of Nearest-Neighbor based image classification , 2008, 2008 IEEE Conference on Computer Vision and Pattern Recognition.

[61]  John Shawe-Taylor,et al.  Canonical Correlation Analysis: An Overview with Application to Learning Methods , 2004, Neural Computation.

[62]  Mor Naaman,et al.  Diamonds in the rough: Social media visual analytics for journalistic inquiry , 2010, 2010 IEEE Symposium on Visual Analytics Science and Technology.

[63]  Hagai Attias,et al.  Topic regression multi-modal Latent Dirichlet Allocation for image annotation , 2010, 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.

[64]  Roger Zimmermann,et al.  Flickr Circles: Aesthetic Tendency Discovery by Multi-View Regularized Topic Modeling , 2016, IEEE Transactions on Multimedia.

[65]  Dong Liu,et al.  Multi-Scale Triplet CNN for Person Re-Identification , 2016, ACM Multimedia.

[66]  M. Shamim Hossain,et al.  Social Event Classification via Boosted Multimodal Supervised Latent Dirichlet Allocation , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[67]  Changsheng Xu,et al.  Cross-Domain Collaborative Learning in Social Multimedia , 2015, ACM Multimedia.

[68]  N. Altman.  An Introduction to Kernel and Nearest-Neighbor Nonparametric Regression , 1992 .

[69]  Thomas Hofmann,et al.  Probabilistic latent semantic indexing , 1999, SIGIR '99.

[70]  Yueting Zhuang,et al.  Cross-modal correlation learning for clustering on image-audio dataset , 2007, ACM Multimedia.

[71]  Changsheng Xu,et al.  Cross-Domain Feature Learning in Multimedia , 2015, IEEE Transactions on Multimedia.

[72]  Xindong Wu,et al.  Learning on Big Graph: Label Inference and Regularization with Anchor Hierarchy , 2017, IEEE Transactions on Knowledge and Data Engineering.

[73]  Yasushi Sakurai,et al.  Online multiscale dynamic topic models , 2010, KDD.

[74]  Ning Chen,et al.  Gibbs max-margin topic models with data augmentation , 2013, J. Mach. Learn. Res..

[75]  Yonghong Yan,et al.  Customer voice sensor: A comprehensive opinion mining system for call center conversation , 2016, 2016 IEEE International Conference on Cloud Computing and Big Data Analysis (ICCCBDA).

[76]  James Allan,et al.  Text classification and named entities for new event detection , 2004, SIGIR '04.

[77]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[78]  Chong Wang,et al.  Continuous Time Dynamic Topic Models , 2008, UAI.

[79]  Meng Wang,et al.  Scalable Semi-Supervised Learning by Efficient Anchor Graph Regularization , 2016, IEEE Transactions on Knowledge and Data Engineering.

[80]  Rajarshi Das,et al.  Gaussian LDA for Topic Models with Word Embeddings , 2015, ACL.

[81]  Dacheng Tao,et al.  Multi-View Object Retrieval via Multi-Scale Topic Models. , 2016, IEEE transactions on image processing : a publication of the IEEE Signal Processing Society.

[82]  Naonori Ueda,et al.  Topic Tracking Model for Analyzing Consumer Purchase Behavior , 2009, IJCAI.

[83]  M. Shamim Hossain,et al.  Word-of-Mouth Understanding: Entity-Centric Multimodal Aspect-Opinion Mining in Social Media , 2015, IEEE Transactions on Multimedia.