Sentiment analysis for customer relationship management: an incremental learning approach

In recent years there has been a significant rethinking of corporate management, which is increasingly based on customer orientation principles. As a matter of fact, customer relationship management processes and systems are ever more popular and crucial to facing today’s business challenges. However, the large number of available customer communication stimuli coming from different (direct and indirect) channels, require automatic language processing techniques to help filter and qualify such stimuli, determine priorities, facilitate the routing of requests and reduce the response times. In this scenario, sentiment analysis plays an important role in measuring customer satisfaction, tracking consumer opinion, interacting with consumers and building customer loyalty. The research described in this paper proposes an approach based on Hierarchical Attention Networks for detecting the sentiment polarity of customer communications. Unlike other existing approaches, after initial training, the defined model can improve over time during system operation using the feedback provided by CRM operators thanks to an integrated incremental learning mechanism. The paper also describes the developed prototype as well as the dataset used for training the model which includes over 30.000 annotated items. The results of two experiments aimed at measuring classifier performance and validating the retraining mechanism are also presented and discussed. In particular, the classifier accuracy turned out to be better than that of other algorithms for the supported languages (macro-averaged f1-score of 0.89 and 0.79 for Italian and English respectively) and the retraining mechanism was able to improve the classification accuracy on new samples without degrading the overall system performance.

[1]  Ting Liu,et al.  Deep learning for sentiment analysis: successful approaches and future challenges , 2015, WIREs Data Mining Knowl. Discov..

[2]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[3]  Misha Denil,et al.  From Group to Individual Labels Using Deep Features , 2015, KDD.

[4]  Phillip E. Pfeifer,et al.  Marketing Metrics: The Definitive Guide to Measuring Marketing Performance , 2010 .

[5]  Samaneh Moghaddam,et al.  Beyond Sentiment Analysis: Mining Defects and Improvements from Customer Feedback , 2015, ECIR.

[6]  Purnima S. Sangle,et al.  Contemporary challenges in CRM technology adoption: a multichannel view , 2014 .

[7]  Youngja Park,et al.  Towards real-time measurement of customer satisfaction using automatically generated call transcripts , 2009, CIKM.

[8]  Guy Lapalme,et al.  A systematic analysis of performance measures for classification tasks , 2009, Inf. Process. Manag..

[9]  LiuTing,et al.  Deep learning for sentiment analysis , 2015 .

[10]  Sisi Liu,et al.  A Hybrid Sentiment Analysis Framework for Large Email Data , 2015, 2015 10th International Conference on Intelligent Systems and Knowledge Engineering (ISKE).

[11]  Yaxin Bi,et al.  Improved lexicon-based sentiment analysis for social media analytics , 2015, Security Informatics.

[12]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[13]  Jana Diesner,et al.  Sentiment Analysis with Incremental Human-in-the-Loop Learning and Lexical Resource Customization , 2015, HT.

[14]  Anthony V. Robins,et al.  Catastrophic Forgetting, Rehearsal and Pseudorehearsal , 1995, Connect. Sci..

[15]  Dale Schuurmans,et al.  Language and Task Independent Text Categorization with Simple Language Models , 2003, NAACL.

[16]  Philip S. Yu,et al.  A holistic lexicon-based approach to opinion mining , 2008, WSDM '08.

[17]  Sisi Liu,et al.  Discovering sentiment sequence within email data through trajectory representation , 2018, Expert Syst. Appl..

[18]  Kristof Coussement,et al.  Improving customer attrition prediction by integrating emotions from client/company interaction emails and evaluating multiple classifiers , 2009, Expert Syst. Appl..

[19]  Sebastian Raschka,et al.  Model Evaluation, Model Selection, and Algorithm Selection in Machine Learning , 2018, ArXiv.

[20]  Vadlamani Ravi,et al.  CUDA-Self-Organizing feature map based visual sentiment analysis of bank customer complaints for Analytical CRM , 2019, ArXiv.

[21]  Milam Aiken,et al.  An Updated Evaluation of Google Translate Accuracy , 2019, Studies in Linguistics and Literature.

[22]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[23]  Jeffrey Heer,et al.  MUSE: reviving memories using email archives , 2011, UIST.

[24]  Jugal K. Kalita,et al.  Sentiment Analysis of Restaurant Reviews on Yelp with Incremental Learning , 2016, 2016 15th IEEE International Conference on Machine Learning and Applications (ICMLA).

[25]  Li Yang,et al.  Learn#: A Novel incremental learning method for text classification , 2020, Expert Syst. Appl..

[26]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[27]  Shankar Ganesan.  Determinants of Long-Term Orientation in Buyer-Seller Relationships , 1994 .

[28]  Sameer Pradhan,et al.  OntoNotes: Large Scale Multi-Layer, Multi-Lingual, Distributed Annotation , 2017 .

[29]  Shankar Ganesan.  Determinants of Long-Term Orientation in Buyer-Seller Relationships , 1994 .

[30]  Hinrich Schütze,et al.  Introduction to information retrieval , 2008 .

[31]  Razvan Pascanu,et al.  Overcoming catastrophic forgetting in neural networks , 2016, Proceedings of the National Academy of Sciences.

[32]  Timothy Baldwin,et al.  langid.py: An Off-the-shelf Language Identification Tool , 2012, ACL.

[33]  Lei Zhang,et al.  Sentiment Analysis and Opinion Mining , 2017, Encyclopedia of Machine Learning and Data Mining.

[34]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[35]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[36]  Amy Van Looy.  Sentiment Analysis and Opinion Mining (Business Intelligence 1) , 2016 .

[37]  Pawel Cichosz,et al.  A Case Study in Text Mining of Discussion Forum Posts: Classification with Bag of Words and Global Vectors , 2018, Int. J. Appl. Math. Comput. Sci..

[38]  Nicola Capuano,et al.  Multi-attribute Categorization of MOOC Forum Posts and Applications to Conversational Agents , 2019, 3PGCIC.

[39]  Maruthi Rohit Ayyagari.  A Framework for Analytical CRM Assessments Challenges and Recommendations , 2019 .

[40]  Alessandro Moschitti,et al.  Twitter Sentiment Analysis with Deep Convolutional Neural Networks , 2015, SIGIR.

[41]  Quoc V. Le,et al.  Don't Decay the Learning Rate, Increase the Batch Size , 2017, ICLR.

[42]  Marc Parizeau,et al.  Evaluating and Characterizing Incremental Learning from Non-Stationary Data , 2018, ArXiv.

[43]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[44]  Bracha Shapira,et al.  Sentiment Analysis in Transcribed Utterances , 2015, PAKDD.

[45]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[46]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.