5W1H-Based Semantic Segmentation of Tweets for Event Detection Using BERT

Detection of events from Twitter has been one of the significant areas in the Text Mining domain due to the volume of content generated by online users. Twitter is considered as one of the top sources for disseminating information to the users. Due to the short length of texts on Twitter, the content generated is often noisy, which makes the detection of events very difficult. Though research on Twitter event detection has been in existence, most of them focused on implementing statistical measures rather than exploiting the semantics. The work presented in this paper presents an approach for the semantic segmentation of Twitter texts (tweets) by adopting the concept of 5W1H (Who, What, When, Where, Why and How). 5W1H represent the semantic constituents (subject, object and modifiers) of a sentence and the actions of verbs on them. The relationship between a verb and the semantic constituents of a sentence forms the basis for representation of an event. The basic approach of the proposed system is to segment the tweets based on the 5W1H contextual word embeddings generated with the help of recent state-of-the-art technology and then clustering the tweets for the representation of possible events. We compared our approach with a simple baseline system that does not segment the tweets. We evaluated the performance of both the approaches by measuring the cosine similarity of the tweets under a cluster. Our 5W1H segmentation approach produced a similarity score above 82% for the most similar tweets in a cluster against the baseline system that scored below 70%.

[1]  S. Dudoit,et al.  A prediction-based resampling method for estimating the number of clusters in a dataset , 2002, Genome Biology.

[2]  Richard K. G. Do,et al.  Convolutional neural networks: an overview and application in radiology , 2018, Insights into Imaging.

[3]  Chenliang Li,et al.  Twevent: segment-based event detection from tweets , 2012, CIKM.

[4]  Hila Becker,et al.  Beyond Trending Topics: Real-World Event Identification on Twitter , 2011, ICWSM.

[5]  Anbupalam Thalamuthu,et al.  Gene expression Evaluation and comparison of gene clustering methods in microarray analysis , 2006 .

[6]  Geoffrey E. Hinton,et al.  Visualizing Data using t-SNE , 2008 .

[7]  Wael Khreich,et al.  A Survey of Techniques for Event Detection in Twitter , 2015, Comput. Intell..

[8]  John M. Hancock,et al.  K -Means Clustering. , 2010 .

[9]  Mehmet A. Orgun,et al.  A survey on real-time event detection from the Twitter data stream , 2018, J. Inf. Sci..

[10]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[11]  Iryna Gurevych,et al.  Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks , 2019, EMNLP.

[12]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[13]  Hosung Park,et al.  What is Twitter, a social network or a news media? , 2010, WWW '10.

[14]  Mehmet A. Orgun,et al.  Real-time event detection from the Twitter data stream using the TwitterNews+ Framework , 2019, Inf. Process. Manag..

[15]  Shi Feng,et al.  Determining the Topic Hashtags for Chinese Microblogs Based on 5W Model , 2016, BigCom.

[16]  Alex Sherstinsky,et al.  Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network , 2018, Physica D: Nonlinear Phenomena.

[17]  James Allan,et al.  Topic detection and tracking: event-based information organization , 2002 .

[18]  Daniel Jurafsky,et al.  Automatic Labeling of Semantic Roles , 2002, CL.

[19]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[20]  Yutaka Matsuo,et al.  Earthquake shakes Twitter users: real-time event detection by social sensors , 2010, WWW '10.

[21]  John B. Lowe,et al.  The Berkeley FrameNet Project , 1998, ACL.

[22]  KhreichWael,et al.  A Survey of Techniques for Event Detection in Twitter , 2015, CI 2015.

[23]  Keval Morabia,et al.  SEDTWik: Segmentation-based Event Detection from Tweets Using Wikipedia , 2019, NAACL.

[24]  Kevin Heaslip,et al.  Developing a Twitter-based traffic event detection model using deep learning architectures , 2019, Expert Syst. Appl..

[25]  Daniel Gildea,et al.  The Proposition Bank: An Annotated Corpus of Semantic Roles , 2005, CL.

[26]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[27]  Amitava Das,et al.  Deep Semantic Role Labeling for Tweets using 5W1H: Who, What, When, Where, Why and How , 2019, Computación y Sistemas.

[28]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.