A Transformer based Approach for Identification of Tweet Acts

Speech acts help in uncovering and understanding the communicative intention and behavior of a speaker utterance. This is pertinent for communication on any platform that embodies social media platforms like Twitter. This paper presents a supervised speech act (tweet act in our case) classifier for tweets for assessing the content and intent of tweets, thereby, exploring the valuable communication amongst the tweeters. With the recent success of Bi-directional Encoder Representations from Transformers (BERT), a newly introduced language representation model that provides pre-trained deep bi-directional representations of vast unlabeled data, we introduce BERT-extended that is built on top of BERT. Our model is based on calculating attention weights over the representations of tokens of a sequence to identify tweet acts. The proposed model attained a benchmark accuracy of 75.97% and outperformed several strong baselines and state-of-the-art approaches on an open-source, tweet act annotated Twitter dataset.

[1]  Adelheit Stein,et al.  Modelling the Illocutionary Aspects of Information-Seeking Dialogues , 1992, Inf. Process. Manag..

[2]  Csr Young,et al.  How to Do Things With Words , 2009 .

[3]  David Vilar,et al.  Dialogue act classification using a Bayesian approach ∗ , 2004 .

[4]  J. Searle.  Expression and Meaning: A taxonomy of illocutionary acts , 1975 .

[5]  Rodney D. Nielsen,et al.  Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network , 2016, COLING.

[6]  John M. Budd,et al.  Discourse Analysis: Method and Application in the Study of Information , 1996, Inf. Process. Manag..

[7]  Christophe Cerisara,et al.  Multi-task dialog act and sentiment recognition on Mastodon , 2018, COLING.

[8]  Wenjie Li,et al.  What Are Tweeters Doing: Recognizing Speech Acts in Twitter , 2011, Analyzing Microtext.

[9]  Soroush Vosoughi,et al.  Tweet Acts: A Speech Act Classifier for Twitter , 2016, ICWSM.

[10]  Soroush Vosoughi,et al.  Automatic detection and verification of rumors on Twitter , 2015 .

[11]  George Kurian,et al.  Google's Neural Machine Translation System: Bridging the Gap between Human and Machine Translation , 2016, ArXiv.

[12]  Andreas Stolcke,et al.  Dialogue act modeling for automatic tagging and recognition of conversational speech , 2000, CL.

[13]  Jian Sun,et al.  Deep Residual Learning for Image Recognition , 2015, 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[14]  Yun Lei,et al.  Using Context Information for Dialog Act Classification in DNN Framework , 2017, EMNLP.

[15]  Welch Bl.  THE GENERALIZATION OF ‘STUDENT'S’ PROBLEM WHEN SEVERAL DIFFERENT POPULATION VARLANCES ARE INVOLVED , 1947 .

[16]  Sriparna Saha,et al.  Tweet Act Classification : A Deep Learning based Classifier for Recognizing Speech Acts in Twitter , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[17]  Avinash Chandra Pandey,et al.  Twitter sentiment analysis using hybrid cuckoo search method , 2017, Inf. Process. Manag..

[18]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[19]  Harshit Kumar,et al.  Dialogue Act Sequence Labeling using Hierarchical encoder with CRF , 2017, AAAI.

[20]  Gary Geunbae Lee,et al.  Semi-supervised Speech Act Recognition in Emails and Forums , 2009, EMNLP.

[21]  Giuseppe Castellucci,et al.  Multi-lingual Intent Detection and Slot Filling in a Joint BERT-based Model , 2019, ArXiv.

[22]  Sriparna Saha,et al.  Exploring Machine Learning and Deep Learning Frameworks for Task-Oriented Dialogue Act Classification , 2019, 2019 International Joint Conference on Neural Networks (IJCNN).

[23]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[24]  Phil Blunsom,et al.  Recurrent Convolutional Neural Networks for Discourse Compositionality , 2013, CVSM@ACL.

[25]  Joel D. Martin,et al.  Sentiment, emotion, purpose, and style in electoral tweets , 2015, Inf. Process. Manag..

[26]  A. Koller,et al.  Speech Acts: An Essay in the Philosophy of Language , 1969 .

[27]  Abbas Rajabifard,et al.  Event relatedness assessment of Twitter messages for emergency response , 2017, Inf. Process. Manag..

[28]  Dirk Heylen,et al.  DIALOGUE-ACT TAGGING USING SMART FEATURE SELECTION; RESULTS ON MULTIPLE CORPORA , 2006, 2006 IEEE Spoken Language Technology Workshop.

[29]  B. L. Welch.  The generalisation of student's problems when several different population variances are involved. , 1947, Biometrika.

[30]  John J. Godfrey,et al.  SWITCHBOARD: telephone speech corpus for research and development , 1992, [Proceedings] ICASSP-92: 1992 IEEE International Conference on Acoustics, Speech, and Signal Processing.

[31]  Wen Wang,et al.  BERT for Joint Intent Classification and Slot Filling , 2019, ArXiv.