A semi-supervised approach in detecting sentiment and emotion based on digital payment reviews

This paper investigates the sentiment and emotion of digital payment application consumers using a hybrid approach consisting of both supervised and unsupervised machine learning techniques. Support vector machine, random forest and Naive Bayes were modeled for sentiment and emotion analyses, whereas latent Dirichlet allocation was administered to identify top emerging topics based on English textual reviews from three digital payment applications. Random forest produced the best results for sentiment (F1 score = 73.8%; Cohen’s Kappa = 52.2%) and emotion (F1 score = 58.8%; Cohen’s Kappa = 44.7%) analyses based on a tenfold cross-validation. Latent Dirichlet allocation revealed best clusters at k = 5 and items = 25, with the top topics being App Service, Transaction, Reload Features, Connectivity and Reward. Findings are presented and discussed in general and also based on each application.

[1]  Jon Rokne,et al.  Emotion detection from text and speech: a survey , 2018, Social Network Analysis and Mining.

[2]  Amlan Chakrabarti,et al.  Demonetization and its aftermath: an analysis based on twitter sentiments , 2018, Sādhanā.

[3]  Jacob Cohen.  A Coefficient of Agreement for Nominal Scales , 1960 .

[4]  Prathamesh Apte,et al.  Text-Based Emotion Analysis: Feature Selection Techniques and Approaches , 2018, Advances in Intelligent Systems and Computing.

[5]  S. Strejilevich,et al.  Stability of facial emotion recognition performance in bipolar disorder , 2016, Psychiatry Research.

[6]  Devika Nadarajah,et al.  Millennials’ Perception on Mobile Payment Services in Malaysia , 2017 .

[7]  Ha Jin Hwang,et al.  Measuring the Effectiveness of E-Wallet in Malaysia , 2018, Big Data, Cloud Computing, Data Science & Engineering.

[8]  Vimala Balakrishnan,et al.  Improving sentiment scoring mechanism: a case study on airline services , 2018, Ind. Manag. Data Syst..

[9]  Neelam Chaplot,et al.  Sentiment Analysis of Movie Reviews using Machine Learning Techniques , 2017 .

[10]  Gunasekaran Manogaran,et al.  An enhanced graph-based semi-supervised learning algorithm to detect fake users on Twitter , 2019, The Journal of Supercomputing.

[11]  Ida Rosmini Othman,et al.  Responsiveness of Smart Card in Restaurants: Factor Analysis Approach☆ , 2013 .

[12]  Justin Zhijun Zhan,et al.  Sentiment analysis using product review data , 2015, Journal of Big Data.

[13]  Paul Rad,et al.  Are you emotional or depressed? Learning about your emotional state from your music using machine learning , 2018, The Journal of Supercomputing.

[14]  P. Ekman.  An argument for basic emotions , 1992 .

[15]  Vishal. A. Kharde,et al.  Sentiment Analysis of Twitter Data : A Survey of Techniques , 2016, ArXiv.

[16]  Herbert Kimura,et al.  Stock price prediction using support vector regression on daily and up to the minute prices , 2018, The Journal of Finance and Data Science.

[17]  D. Reiss.  Is money going digital? An alternative perspective on the current hype , 2018, Financial Innovation.

[18]  J. Christopher Westland,et al.  Methods to Assess the Value of New Technologies: the Case of Consumer Sentiment Towards Digital Wallet Technology , 2017 .

[19]  Thomas Roelleke,et al.  TF-IDF uncovered: a study of theories and probabilities , 2008, SIGIR '08.

[20]  Kamal Eddine El Kadiri,et al.  Random Forest and Support Vector Machine based Hybrid Approach to Sentiment Analysis , 2018 .

[21]  Pallavi Maindola,et al.  Sentiment Analysis of Digital Wallets and UPI Systems in India Post Demonetization Using IBM Watson , 2018, 2018 International Conference on Computer Communication and Informatics (ICCCI).

[22]  Huan Liu,et al.  Sentiment Informed Cyberbullying Detection in Social Media , 2017, ECML/PKDD.

[23]  Roliana Ibrahim,et al.  Experimental study of support vector machines and Naïve Bayes classifier on automated subject area classification , 2017 .

[24]  Gintautas Garsva,et al.  SVM and k-Means Hybrid Method for Textual Data Sentiment Analysis , 2019, Balt. J. Mod. Comput..

[25]  Carlos Alberto Ochoa Ortíz Zezzatti,et al.  A Sentiment Analysis Model to Analyze Students Reviews of Teacher Performance Using Support Vector Machines , 2017, DCAI.

[26]  Patrizia Grifoni,et al.  Approaches, Tools and Applications for Sentiment Analysis Implementation , 2015 .

[27]  Raj K. Palkar,et al.  Comparative Evaluation of Supervised Learning Algorithms for Sentiment Analysis of Movie Reviews , 2016 .

[28]  Min Song,et al.  Developing a supervised learning-based social media business sentiment index , 2019, The Journal of Supercomputing.

[29]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[30]  Eojina Kim,et al.  More than words: Do emotional content and linguistic style matching matter on restaurant review helpfulness? , 2019, International Journal of Hospitality Management.

[31]  Amir Javed,et al.  A comparative analysis of detection mechanisms for emotion detection , 2019, Journal of Physics: Conference Series.

[32]  Theodoros Iliou,et al.  Features and classifiers for emotion recognition from speech: a survey from 2000 to 2011 , 2012, Artificial Intelligence Review.

[33]  Choochart Haruechaiyasak,et al.  Classifying emotion in Thai youtube comments , 2015, 2015 6th International Conference of Information and Communication Technology for Embedded Systems (IC-ICTES).

[34]  Stanimira Yordanova,et al.  Sentiment Classification of Hotel Reviews in Social Media with Decision Tree Learning , 2017 .

[35]  Paulius Danenas,et al.  SVM and k-Means Hybrid Method for Textual Data Sentiment Analysis , 2019, Balt. J. Mod. Comput..

[36]  G. Li.  Application of sentiment analysis : assessing the reliability and validity of the global airlines rating program , 2017 .

[37]  Mandar Deshpande,et al.  Depression detection using emotion artificial intelligence , 2017, 2017 International Conference on Intelligent Sustainable Systems (ICISS).

[38]  Fabio Crestani,et al.  Like It or Not , 2016, ACM Comput. Surv..

[39]  Yan Liu,et al.  A Topic based Approach for Sentiment Analysis on Twitter Data , 2016 .

[40]  Ethan Ligon,et al.  What explains low adoption of digital payment technologies? Evidence from small-scale merchants in Jaipur, India , 2019, PloS one.

[41]  Christopher Joseph Pal,et al.  EmoNets: Multimodal deep learning approaches for emotion recognition in video , 2015, Journal on Multimodal User Interfaces.

[42]  Nitesh V. Chawla,et al.  SMOTE: Synthetic Minority Over-sampling Technique , 2002, J. Artif. Intell. Res..

[43]  Ahmad Ali,et al.  Sentiment Analysis on Twitter Data using KNN and SVM , 2017 .