Information cascades prediction with attention neural network

Cascade prediction helps us uncover the basic mechanisms that govern collective human behavior in networks, and it also is very important in extensive other applications, such as viral marketing, online advertising, and recommender systems. However, it is not trivial to make predictions due to the myriad factors that influence a user’s decision to reshare content. This paper presents a novel method for predicting the increment size of the information cascade based on an end-to-end neural network. Learning the representation of a cascade in an end-to-end manner circumvents the difficulties inherent to blue the design of hand-crafted features. An attention mechanism, which consists of the intra-attention and inter-gate module, was designed to obtain and fuse the temporal and structural information learned from the observed period of the cascade. The experiments were performed on two real-world scenarios, i.e., predicting the size of retweet cascades on Twitter and predicting the citation of papers in AMiner. Extensive results demonstrated that our method outperformed the state-of-the-art cascade prediction methods, including both feature-based and generative approaches.

[1]  Chethan Pandarinath,et al.  Inferring single-trial neural population dynamics using sequential auto-encoders , 2017, Nature Methods.

[2]  Emily B. Fox,et al.  A Bayesian Approach for Predicting the Popularity of Tweets , 2013, ArXiv.

[3]  Duncan J. Watts,et al.  Exploring Limits to Prediction in Complex Social Systems , 2016, WWW.

[4]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[5]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[6]  Tomasz Trzcinski,et al.  Understanding Multimodal Popularity Prediction of Social Media Videos With Self-Attention , 2018, IEEE Access.

[7]  Utkarsh Upadhyay,et al.  Recurrent Marked Temporal Point Processes: Embedding Event History to Vector , 2016, KDD.

[8]  Cheng Li,et al.  DeepCas: An End-to-end Predictor of Information Cascades , 2016, WWW.

[9]  Raffay Hamid,et al.  What makes an image popular? , 2014, WWW.

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Berkant Barla Cambazoglu,et al.  On the feasibility of predicting popular news at cold start , 2017, J. Assoc. Inf. Sci. Technol..

[12]  Jure Leskovec,et al.  SEISMIC: A Self-Exciting Point Process Model for Predicting Tweet Popularity , 2015, KDD.

[13]  Richard Socher,et al.  Dynamic Memory Networks for Visual and Textual Question Answering , 2016, ICML.

[14]  Lawrence Carin,et al.  Predicting Smoking Events with a Time-Varying Semi-Parametric Hawkes Process Model , 2018, MLHC.

[15]  Joon Son Chung,et al.  Deep Audio-Visual Speech Recognition , 2018, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Dan Cosley,et al.  Predictability of Popularity: Gaps between Prediction and Understanding , 2016, ICWSM.

[17]  R. Desimone,et al.  Neural mechanisms of selective visual attention. , 1995, Annual review of neuroscience.

[18]  Steven Skiena,et al.  DeepWalk: online learning of social representations , 2014, KDD.

[19]  Amit P. Sheth,et al.  Predictive Analysis on Twitter: Techniques and Applications , 2018 .

[20]  Masoud Makrehchi,et al.  Activity-based Twitter sampling for content-based and user-centric prediction models , 2017, Human-centric Computing and Information Sciences.

[21]  Bernardo A. Huberman,et al.  Predicting the popularity of online content , 2008, Commun. ACM.

[22]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[23]  Hongyuan Zha,et al.  Learning Parametric Models for Social Infectivity in Multi-Dimensional Hawkes Processes , 2014, AAAI.

[24]  Wei Zhang,et al.  User-guided Hierarchical Attention Network for Multi-modal Social Image Popularity Prediction , 2018, WWW.

[25]  Rediet Abebe Can Cascades be Predicted? , 2014 .

[26]  H. Damasio,et al.  IEEE Transactions on Pattern Analysis and Machine Intelligence: Special Issue on Perceptual Organization in Computer Vision , 1998 .

[27]  Masahiro Kimura,et al.  Prediction of Information Diffusion Probabilities for Independent Cascade Model , 2008, KES.

[28]  Swapnil Mishra,et al.  Feature Driven and Point Process Approaches for Popularity Prediction , 2016, CIKM.

[29]  Rita Cucchiara,et al.  Predicting Human Eye Fixations via an LSTM-Based Saliency Attentive Model , 2016, IEEE Transactions on Image Processing.

[30]  Nitin Agarwal,et al.  Emerging Research Challenges and Opportunities in Computational Social Network Analysis and Mining , 2019, Lecture Notes in Social Networks.

[31]  Hui Liu,et al.  Predicting popularity via a generative model with adaptive peeking window , 2019, Physica A: Statistical Mechanics and its Applications.

[32]  Razvan Pascanu,et al.  On the difficulty of training recurrent neural networks , 2012, ICML.

[33]  John Salvatier,et al.  Theano: A Python framework for fast computation of mathematical expressions , 2016, ArXiv.

[34]  Siqi Shen,et al.  Predicting the implicit and the explicit video popularity in a User Generated Content site with enhanced social features , 2018, Comput. Networks.

[35]  Przemysław Rokita,et al.  Predicting Popularity of Online Videos Using Support Vector Regression , 2017, IEEE Transactions on Multimedia.

[36]  Wenji Mao,et al.  Exploring Trends and Patterns of Popularity Stage Evolution in Social Media , 2020, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[37]  Raúl Monroy,et al.  Fusing pattern discovery and visual analytics approaches in tweet propagation , 2019, Inf. Fusion.

[38]  Lars Backstrom,et al.  Structural diversity in social contagion , 2012, Proceedings of the National Academy of Sciences.

[39]  F. Xavier Roca,et al.  Pay Attention to the Activations: A Modular Attention Mechanism for Fine-Grained Image Recognition , 2020, IEEE Transactions on Multimedia.

[40]  Hilde van der Togt,et al.  Publisher's Note , 2003, J. Netw. Comput. Appl..

[41]  Filippo Menczer,et al.  Predicting Successful Memes Using Network and Community Structure , 2014, ICWSM.

[42]  Amir Masoud Rahmani,et al.  Personality classification based on profiles of social networks’ users and the five-factor model of personality , 2018, Human-centric Computing and Information Sciences.

[43]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[44]  Yoshua Bengio,et al.  Fine-grained attention mechanism for neural machine translation , 2018, Neurocomputing.

[45]  Christof Koch,et al.  A Model of Saliency-Based Visual Attention for Rapid Scene Analysis , 2009 .

[46]  Richard Socher,et al.  Ask Me Anything: Dynamic Memory Networks for Natural Language Processing , 2015, ICML.

[47]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[48]  Fei Wang,et al.  Uncovering and predicting the dynamic process of information cascades with survival model , 2017, Knowledge and Information Systems.