Developing an LSTM-based Classification Model of IndiHome Customer Feedbacks

The development of companies in the service sector is inseparable from problems with the customer service process. The latest trends show that customers are more comfortable using social media and chat services as feedback delivery media compared to sending emails.In this study, we classify user feedback from an Internet Service Provider (ISP) in Indonesia named IndiHome. In this study, there are two test scenarios carried out to classify, the first scenario uses Short Term Memory combined with Word Embedding, and the second scenario uses Naive Bayes combined with TF-IDF. based on the test results, obtained accuracy on the LSTM method combined with Word Embedding with an f1 score of 87.98 % and accuracy obtained at Naive Bayes combined with TF-IDF with an f1 score of 76.77 %. From the tests that have been done, different results are obtained and have a big difference, which is 11.21%. The difference in the results of this classification is influenced by several factors, namely the first factor is the extraction of features used and the second is the data used. In feature selection, TF-IDF is more ideal for data that has a large or long document size because the representation seen from the whole document, in contrast to Word Embedding that is not affected by the data size because the representation is seen from words. so this is what causes the method of using LSTM combined with Word Embedding to produce higher results.

[1]  Suyanto Suyanto,et al.  Indonesian Chatbot of University Admission Using a Question Answering System Based on Sequence-to-Sequence Model , 2019, Procedia Computer Science.

[2]  David Madigan,et al.  On the Naive Bayes Model for Text Categorization , 2003, AISTATS.

[3]  George Knox,et al.  Customer Complaints and Recovery Effectiveness: A Customer Base Approach , 2014 .

[4]  Hermann Ney,et al.  LSTM Neural Networks for Language Modeling , 2012, INTERSPEECH.

[5]  Kwangsoo Kim,et al.  Customer Complaints Analysis Using Text Mining and Outcome-Driven Innovation Method for Market-Oriented Product Development , 2018, Sustainability.

[6]  Mani Shankar,et al.  Multiclass Text Classification and Analytics for Improving Customer Support Response through different Classifiers , 2018, 2018 International Conference on Advances in Computing, Communications and Informatics (ICACCI).

[7]  Suyanto Suyanto,et al.  The Impact of Local Attention in LSTM for Abstractive Text Summarization , 2019, 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI).

[8]  Elizabeth Shriberg,et al.  The ICSI Meeting Recorder Dialog Act (MRDA) Corpus , 2004, SIGDIAL Workshop.

[9]  Dev Kumar Chaudhary,et al.  Sentiment Analysis Using Rnn and Google Translator , 2018, 2018 8th International Conference on Cloud Computing, Data Science & Engineering (Confluence).

[10]  Omar Khadeer Hussain,et al.  A Survey on Chatbot Implementation in Customer Service Industry through Deep Neural Networks , 2018, 2018 IEEE 15th International Conference on e-Business Engineering (ICEBE).

[11]  Neetesh Gupta,et al.  Real Time Twitter Sentiment Analysis for Product Reviews Using Naive Bayes Classifier , 2018, Lecture Notes on Data Engineering and Communications Technologies.

[12]  Adiwijaya Adiwijaya,et al.  Implementasi Mutual Information Dan Naive Bayes Untuk Klasifikasi Data Microarray , 2017 .

[13]  Suyanto Suyanto,et al.  Indonesian Abstractive Text Summarization Using Bidirectional Gated Recurrent Unit , 2019 .

[14]  Giuseppe Di Fabbrizio,et al.  EMOTION DETECTION IN EMAIL CUSTOMER CARE , 2013, Comput. Intell..

[15]  Yi-Shun Wang,et al.  Consumer responses to online retailer's service recovery after a service failure , 2011 .

[16]  Sfenrianto Sfenrianto,et al.  Sentiment Analysis about E-Commerce from Tweets Using Decision Tree, K-Nearest Neighbor, and Naïve Bayes , 2018, 2018 International Conference on Orange Technologies (ICOT).

[17]  Din J. Wasem,et al.  Mining of Massive Datasets , 2014 .

[18]  M. B. Kristanda,et al.  AUTOMATED COMPLAINTS CLASSIFICATION USING MODIFIED NAZIEF-ADRIANI STEMMING ALGORITHM AND NAIVE BAYES CLASSIFIER , 2019 .

[19]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[20]  M. de Rijke,et al.  Short Text Similarity with Word Embeddings , 2015, CIKM.

[21]  Mingyong Liu,et al.  An improvement of TFIDF weighting in text categorization , .

[22]  Souham Meshoul,et al.  Deep learning for online harassment detection in tweets , 2018, 2018 3rd International Conference on Pattern Analysis and Intelligent Systems (PAIS).

[23]  Suyanto Suyanto,et al.  Typographic-Based Data Augmentation to Improve a Question Retrieval in Short Dialogue System , 2019, 2019 International Seminar on Research of Information Technology and Intelligent Systems (ISRITI).

[24]  Franck Dernoncourt,et al.  Sequential Short-Text Classification with Recurrent and Convolutional Neural Networks , 2016, NAACL.

[25]  Suharjito Suharjito,et al.  Classification Model of Contact Center Customers Emails Using Machine Learning , 2020 .

[26]  Rafael E. Banchs,et al.  The fifth dialog state tracking challenge , 2016, 2016 IEEE Spoken Language Technology Workshop (SLT).

[27]  Kristof Coussement,et al.  Improving Customer Complaint Management by Automatic Email Classification Using Linguistic Style Features as Predictors , 2007 .