Modeling Temporal Patterns of Cyberbullying Detection with Hierarchical Attention Networks

Cyberbullying is rapidly becoming one of the most serious online risks for adolescents. This has motivated work on machine learning methods to automate the process of cyberbullying detection, which have so far mostly viewed cyberbullying as one-off incidents that occur at a single point in time. Comparatively less is known about how cyberbullying behavior occurs and evolves over time. This oversight highlights a crucial open challenge for cyberbullying-related research, given that cyberbullying is typically defined as intentional acts of aggression via electronic communication that occur repeatedly and persistently. In this article, we center our discussion on the challenge of modeling temporal patterns of cyberbullying behavior. Specifically, we investigate how temporal information within a social media session, which has an inherently hierarchical structure (e.g., words form a comment and comments form a session), can be leveraged to facilitate cyberbullying detection. Recent findings from interdisciplinary research suggest that the temporal characteristics of bullying sessions differ from those of non-bullying sessions and that the temporal information from users’ comments can improve cyberbullying detection. The proposed framework consists of three distinctive features: (1) a hierarchical structure that reflects how a social media session is formed in a bottom-up manner; (2) attention mechanisms applied at the word- and comment-level to differentiate the contributions of words and comments to the representation of a social media session; and (3) the incorporation of temporal features in modeling cyberbullying behavior at the comment-level. Quantitative and qualitative evaluations are conducted on a real-world dataset collected from Instagram, the social networking site with the highest percentage of users reporting cyberbullying experiences. Results from empirical evaluations show the significance of the proposed methods, which are tailored to capture temporal patterns of cyberbullying detection.

[1]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[2]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[3]  Jin-Hyuk Hong,et al.  Semantic Sentence Matching with Densely-connected Recurrent and Co-attentive Information , 2018, AAAI.

[4]  Nanyun Peng,et al.  Multi-task Learning for Universal Sentence Embeddings: A Thorough Evaluation using Transfer and Auxiliary Tasks , 2018, 1804.07911.

[5]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[6]  Tianqi Chen,et al.  XGBoost: A Scalable Tree Boosting System , 2016, KDD.

[7]  Lu Cheng,et al.  PI-Bully: Personalized Cyberbullying Detection with Peer Influence , 2019, IJCAI.

[8]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[9]  Qianjia Huang,et al.  Cyber Bullying Detection Using Social and Textual Analysis , 2014, SAM '14.

[10]  Hal Daumé,et al.  Deep Unordered Composition Rivals Syntactic Methods for Text Classification , 2015, ACL.

[11]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[12]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[13]  R. Ordelman,et al.  Improved cyberbullying detection using gender information , 2012 .

[14]  Parma Nand,et al.  “How Bullying is this Message?”: A Psychometric Thermometer for Bullying , 2016, COLING.

[15]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[16]  James W. Pennebaker,et al.  Linguistic Inquiry and Word Count (LIWC2007) , 2007 .

[17]  Oksana Smal,et al.  POLITICAL DISCOURSE CONTENT ANALYSIS: A CRITICAL OVERVIEW OF A COMPUTERIZED TEXT ANALYSIS PROGRAM LINGUISTIC INQUIRY AND WORD COUNT (LIWC) , 2020, Naukovì zapiski Nacìonalʹnogo unìversitetu «Ostrozʹka akademìâ». Serìâ «Fìlologìâ».

[18]  Chong Wang,et al.  TopicRNN: A Recurrent Neural Network with Long-Range Semantic Dependency , 2016, ICLR.

[19]  Lu Cheng,et al.  Session-Based Cyberbullying Detection: Problems and Challenges , 2021, IEEE Internet Computing.

[20]  Shivakant Mishra,et al.  Prediction of Cyberbullying Incidents on the Instagram Social Network , 2015, ArXiv.

[21]  Chaoyi Pang,et al.  Sentiment Analysis for Effective Detection of Cyber Bullying , 2012, APWeb.

[22]  Ruocheng Guo,et al.  A Survey of Learning Causality with Data , 2018, ACM Comput. Surv..

[23]  Vivek K. Singh,et al.  Time Reveals All Wounds: Modeling Temporal Characteristics of Cyberbullying , 2018, ICWSM.

[24]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[25]  Fred Morstatter,et al.  Aggressive, Repetitive, Intentional, Visible, and Imbalanced: Refining Representations for Cyberbullying Classification , 2020, ICWSM.

[26]  Anna Cinzia Squicciarini,et al.  Identification and characterization of cyberbullying dynamics in an online social network , 2022 .

[27]  Yann LeCun,et al.  Very Deep Convolutional Networks for Text Classification , 2016, EACL.

[28]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[29]  Henry Lieberman,et al.  Modeling the Detection of Textual Cyberbullying , 2011, The Social Mobile Web.

[30]  Tong Zhang,et al.  Effective Use of Word Order for Text Categorization with Convolutional Neural Networks , 2014, NAACL.

[31]  Ruocheng Guo,et al.  Robust Cyberbullying Detection with Causal Interpretation , 2019, WWW.

[32]  Tao Shen,et al.  DiSAN: Directional Self-Attention Network for RNN/CNN-free Language Understanding , 2017, AAAI.

[33]  Kelly Reynolds,et al.  Detecting cyberbullying: query terms and techniques , 2013, WebSci.

[34]  Tong Zhang,et al.  Semi-supervised Convolutional Neural Networks for Text Categorization via Region Embedding , 2015, NIPS.

[35]  Yulan He,et al.  Approaches to Automated Detection of Cyberbullying: A Survey , 2020, IEEE Transactions on Affective Computing.

[36]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[37]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[38]  Ruocheng Guo,et al.  Hierarchical Attention Networks for Cyberbullying Detection on the Instagram Social Network , 2019, SDM.

[39]  E. Cambria,et al.  Deep Learning--based Text Classification , 2020, ACM Comput. Surv..

[40]  Lukás Burget,et al.  Recurrent Neural Network Based Language Modeling in Meeting Recognition , 2011, INTERSPEECH.

[41]  Charalampos Chelmis,et al.  Minority Report: Cyberbullying Prediction on Instagram , 2019, WebSci.

[42]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[43]  Lu Cheng,et al.  XBully: Cyberbullying Detection within a Multi-Modal Context , 2019, WSDM.

[44]  Yasin N. Silva,et al.  Temporal Properties of Cyberbullying on Instagram , 2020, WWW.

[45]  Justin Zhijun Zhan,et al.  Using deep learning for short text understanding , 2017, Journal of Big Data.

[46]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[47]  Peter K. Smith,et al.  Cyberbullying: its nature and impact in secondary school pupils. , 2008, Journal of child psychology and psychiatry, and allied disciplines.

[48]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[49]  Yiming Yang,et al.  Deep Learning for Extreme Multi-label Text Classification , 2017, SIGIR.

[50]  Daphney-Stavroula Zois,et al.  Cyberbullying Ends Here: Towards Robust Detection of Cyberbullying in Social Media , 2019, WWW.

[51]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[52]  Huan Liu,et al.  Sentiment Informed Cyberbullying Detection in Social Media , 2017, ECML/PKDD.

[53]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[54]  Xiaojun Wan,et al.  Attention-based LSTM Network for Cross-Lingual Sentiment Classification , 2016, EMNLP.

[55]  Jun-Ming Xu,et al.  Learning from Bullying Traces in Social Media , 2012, NAACL.

[56]  Brian D. Davison,et al.  Detection of Harassment on Web 2.0 , 2009 .

[57]  Xue Li,et al.  An Effective Approach for Cyberbullying Detection , 2013 .