Complaint Identification in Social Media with Transformer Networks

Complaining is a speech act extensively used by humans to communicate a negative inconsistency between reality and expectations. Previous work on automatically identifying complaints in social media has focused on using feature-based and task-specific neural network models. Adapting state-of-the-art pre-trained neural language models and their combinations with other linguistic information from topics or sentiment for complaint prediction has yet to be explored. In this paper, we evaluate a battery of neural models underpinned by transformer networks which we subsequently combine with linguistic information. Experiments on a publicly available data set of complaints demonstrate that our models outperform previous state-of-the-art methods by a large margin achieving a macro F1 up to 87.

[1]  Febina Merson,et al.  A Text Mining Approach to Identify and Analyse Prominent issues from Public Complaints , 2017 .

[2]  Louis-Philippe Morency,et al.  Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors , 2018, AAAI.

[3]  Louis-Philippe Morency,et al.  Integrating Multimodal Information in Large Pretrained Transformers , 2020, ACL.

[4]  Camilla Vásquez.  Complaints online: The case of TripAdvisor , 2011 .

[5]  A. Cohen,et al.  The Production of Speech Acts by EFL Learners , 1993 .

[6]  Aditya Gupta,et al.  Effective Use of Transformer Networks for Entity Tracking , 2019, EMNLP.

[7]  Yijun Li,et al.  Service Failure Complaints Identification in Social Media: A Text Classification Approach , 2013, ICIS.

[8]  M. Shahrokhi,et al.  Gender Differences in Complaint Strategies among Iranian Upper Intermediate EFL Students , 2016 .

[9]  Louis-Philippe Morency,et al.  M-BERT: Injecting Multimodal Information in the BERT Structure , 2019, ArXiv.

[10]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[11]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[12]  Jian Pei,et al.  Detecting Customer Complaint Escalation with Recurrent Neural Networks and Manually-Engineered Features , 2019, NAACL.

[13]  J Forster,et al.  A Cognitive Computing Approach for Classification of Complaints in the Insurance Industry , 2017 .

[14]  Enkhbold Bataa,et al.  An Investigation of Transfer Learning-Based Sentiment Analysis in Japanese , 2019, ACL.

[15]  Svitlana Volkova,et al.  Inferring Perceived Demographics from User Emotional Tone and User-Environment Emotional Contrast , 2016, ACL.

[16]  Nikolaos Aletras,et al.  An analysis of the user occupational class through Twitter content , 2015, ACL.

[17]  Nikolaos Aletras,et al.  Analyzing Political Parody in Social Media , 2020, ACL.

[18]  Aron Culotta,et al.  Identifying Leading Indicators of Product Recalls from Online Reviews Using Positive Unlabeled Learning and Domain Adaptation , 2017, ICWSM.

[19]  Kevin Seppi,et al.  Humor Detection: A Transformer Gets the Last Laugh , 2019, EMNLP.

[20]  Sebastian Ruder,et al.  Universal Language Model Fine-tuning for Text Classification , 2018, ACL.

[21]  A. Trosborg.  Interlanguage Pragmatics: Requests, Complaints, and Apologies , 1994 .

[22]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[23]  P. Ekman.  An argument for basic emotions , 1992 .

[24]  Yoram Bachrach,et al.  Studying User Income through Language, Behaviour and Affect in Social Media , 2015, PloS one.

[25]  Richard Socher,et al.  Regularizing and Optimizing LSTM Language Models , 2017, ICLR.

[26]  A. Abt.  Interlanguage Pragmatics Requests Complaints And Apologies , 2016 .

[27]  Nikolaos Aletras,et al.  Automatically Identifying Complaints in Social Media , 2019, ACL.

[28]  Kristof Coussement,et al.  Improving Customer Complaint Management by Automatic Email Classification Using Linguistic Style Features as Predictors , 2007 .

[29]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[30]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[31]  Beverley Sparks,et al.  Complaining in Cyberspace: The Motives and Forms of Hotel Guests' Complaints Online , 2010 .

[32]  D Gunawan,et al.  Building automatic customer complaints filtering application based on Twitter in Bahasa Indonesia , 2018 .

[33]  Suhatati Tjandra,et al.  Determining citizen complaints to the appropriate government departments using KNN algorithm , 2015, 2015 13th International Conference on ICT and Knowledge Engineering (ICT & Knowledge Engineering 2015).

[34]  Elite Olshtain,et al.  10. Complaints: A study of speech act behavior among native and non-native speakers of Hebrew , 1987 .

[35]  S. Sumpeno,et al.  Sentiment analysis of public complaints using lexical resources between Indonesian sentiment lexicon and Sentiwordnet , 2017, 2017 International Seminar on Intelligent Technology and Its Applications (ISITIA).

[36]  Ayu Purwarianti,et al.  Indonesian Twitter text authority classification for government in Bandung , 2014, 2014 International Conference of Advanced Informatics: Concept, Theory and Application (ICAICTA).

[37]  Yiming Yang,et al.  XLNet: Generalized Autoregressive Pretraining for Language Understanding , 2019, NeurIPS.

[38]  Robin M. Kowalski,et al.  Complaints and complaining: functions, antecedents, and consequences. , 1996, Psychological bulletin.

[39]  Girish Keshav Palshikar,et al.  Deciphering Review Comments: Identifying Suggestions, Appreciations and Complaints , 2015, NLDB.