A Multi-Criteria Approach for Arabic Dialect Sentiment Analysis for Online Reviews: Exploiting Optimal Machine Learning Algorithm Selection

A sentiment analysis of Arabic texts is an important task in many commercial applications such as Twitter. This study introduces a multi-criteria method to empirically assess and rank classifiers for Arabic sentiment analysis. Prominent machine learning algorithms were deployed to build classification models for Arabic sentiment analysis classifiers. Moreover, an assessment of the top five machine learning classifiers’ performances measures was discussed to rank the performance of the classifier. We integrated the top five ranking methods with evaluation metrics of machine learning classifiers such as accuracy, recall, precision, F-measure, CPU Time, classification error, and area under the curve (AUC). The method was tested using Saudi Arabic product reviews to compare five popular classifiers. Our results suggest that deep learning and support vector machine (SVM) classifiers perform best with accuracy 85.25%, 82.30%; precision 85.30, 83.87%; recall 88.41%, 83.89; F-measure 86.81, 83.87%; classification error 14.75, 17.70; and AUC 0.93, 0.90, respectively. They outperform decision trees, K-nearest neighbours (K-NN), and Naïve Bayes classifiers.

[1]  Rehab Duwairi,et al.  A study of the effects of preprocessing strategies on sentiment analysis for Arabic text , 2014, J. Inf. Sci..

[2]  Sarah Alhumoud,et al.  Hybrid sentiment analyser for Arabic tweets using R , 2015, 2015 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K).

[3]  Jonathan Cheung-Wai Chan,et al.  Multiple Criteria for Evaluating Machine Learning Algorithms for Land Cover Classification from Satellite Data , 2000 .

[4]  Ferat Sahin,et al.  A survey on feature selection methods , 2014, Comput. Electr. Eng..

[5]  Belal Abu Ata,et al.  A rule-based stemmer for Arabic Gulf dialect , 2015, J. King Saud Univ. Comput. Inf. Sci..

[6]  Ram Gopal Raj,et al.  A Review on Arabic Sentiment Analysis: State-of-the-Art, Taxonomy and Open Research Challenges , 2019, IEEE Access.

[7]  Mohamed M. Mostafa,et al.  More than words: Social networks' text mining for consumer brand sentiments , 2013, Expert Syst. Appl..

[8]  Mahmoud Al-Ayyoub,et al.  Automatic categorization of Arabic articles based on their political orientation , 2018, Digit. Investig..

[9]  Pedro Antonio Gutiérrez,et al.  Monotonic classification: an overview on algorithms, performance measures and data sets , 2018, Neurocomputing.

[10]  Dong-Hong Ji,et al.  Towards Twitter sentiment classification by multi-level sentiment-enriched word embeddings , 2016, Neurocomputing.

[11]  David M. W. Powers,et al.  Evaluation: from precision, recall and F-measure to ROC, informedness, markedness and correlation , 2011, ArXiv.

[12]  MengChu Zhou,et al.  A Deep Latent Factor Model for High-Dimensional and Sparse Matrices in Recommender Systems , 2019, IEEE Transactions on Systems, Man, and Cybernetics: Systems.

[13]  Charles W. Anderson,et al.  EXTENDING THE KNOWLEDGE OF THE ARABIC SENTIMENT CLASSIFICATION USING A FOREIGN EXTERNAL LEXICAL SOURCE , 2016 .

[14]  Kannan Govindan,et al.  Multi criteria decision making approaches for green supplier evaluation and selection: a literature review , 2015 .

[15]  Sabrina Tiun,et al.  Comparison of machine learning approaches on Arabic twitter sentiment analysis , 2016 .

[16]  Cagatay CATAL,et al.  A sentiment classification model based on multiple classifiers , 2017, Appl. Soft Comput..

[17]  Ram Gopal Raj,et al.  A systematic literature review on opinion types and sentiment analysis techniques: Tasks and challenges , 2017, Internet Res..

[18]  Mahmoud Al-Ayyoub,et al.  Hierarchical Classifiers for Multi-Way Sentiment Analysis of Arabic Reviews , 2016 .

[19]  A. Herlopian,et al.  Predictors of quality of life among multiple sclerosis patients: a comprehensive analysis , 2013, European journal of neurology.

[20]  Salwani Abdullah,et al.  Arabic senti-lexicon: Constructing publicly available language resources for Arabic sentiment analysis , 2018, J. Inf. Sci..

[21]  Pao-Long Chang,et al.  A fuzzy multi-criteria decision making method for technology transfer strategy selection in biotechnology , 1994 .

[22]  Igor Kononenko,et al.  Machine learning for medical diagnosis: history, state of the art and perspective , 2001, Artif. Intell. Medicine.

[23]  Amira Abdelwahab,et al.  An enhanced feature extraction technique for improving sentiment analysis in Arabic language , 2016, 2016 4th IEEE International Colloquium on Information Science and Technology (CiSt).

[24]  Basit Shahzad,et al.  Classification of Arabic Twitter Users: A Study Based on User Behaviour and Interests , 2016, Mob. Inf. Syst..

[25]  Ram Gopal Raj,et al.  Assessing consumers' satisfaction and expectations through online opinions: Expectation and disconfirmation approach , 2017, Comput. Hum. Behav..

[26]  Erik Cambria,et al.  A review of affective computing: From unimodal analysis to multimodal fusion , 2017, Inf. Fusion.

[27]  Honggang Wang,et al.  User preferences based software defect detection algorithms selection using MCDM , 2012, Inf. Sci..

[28]  Adel Hamdan Mohammad,et al.  Arabic Text Categorization Using Support vector machine, Naïve Bayes and Neural Network , 2016 .

[29]  Sendhil Mullainathan,et al.  Machine Learning: An Applied Econometric Approach , 2017, Journal of Economic Perspectives.

[30]  Erik Cambria,et al.  Enhancing Business Intelligence by Means of Suggestive Reviews , 2014, TheScientificWorldJournal.

[31]  Long Jin,et al.  RNN for Solving Perturbed Time-Varying Underdetermined Linear System With Double Bound Limits on Residual Errors and State Variables , 2019, IEEE Transactions on Industrial Informatics.

[32]  Yunming Ye,et al.  Sentiment analysis through critic learning for optimizing convolutional neural networks with rules , 2019, Neurocomputing.

[33]  Sungyoung Lee,et al.  Accurate multi-criteria decision making methodology for recommending machine learning algorithm , 2017, Expert Syst. Appl..

[34]  Pedro Antonio Gutiérrez,et al.  Classification of countries' progress toward a knowledge economy based on machine learning classification techniques , 2015, Expert Syst. Appl..

[35]  Felipe Bravo-Marquez,et al.  Meta-level sentiment models for big social data analysis , 2014, Knowl. Based Syst..

[36]  Anne E Carpenter,et al.  Opportunities and obstacles for deep learning in biology and medicine , 2017, bioRxiv.

[37]  Kewei Tu,et al.  Lexicalized Neural Unsupervised Dependency Parsing , 2019, Neurocomputing.

[38]  Francisco Herrera,et al.  Inconsistencies on TripAdvisor reviews: A unified index between users and Sentiment Analysis Methods , 2019, Neurocomputing.

[39]  Mahmoud Al-Ayyoub,et al.  Deep Recurrent neural network vs. support vector machine for aspect-based sentiment analysis of Arabic hotels' reviews , 2017, J. Comput. Sci..

[40]  Lars Kotthoff,et al.  Algorithm Selection for Combinatorial Search Problems: A Survey , 2012, AI Mag..

[41]  Yurong Liu,et al.  A survey of deep neural network architectures and their applications , 2017, Neurocomputing.

[42]  Erik Cambria,et al.  Computational Intelligence for Affective Computing and Sentiment Analysis [Guest Editorial] , 2019, IEEE Comput. Intell. Mag..

[43]  Safa Ben Hamouda,et al.  Social Networks ’ Text Mining for Sentiment Classification : The case of Facebook ’ statuses updates in the “ Arabic Spring ” Era , 2013 .

[44]  Yong Shi,et al.  ν-Nonparallel support vector machine for pattern classification , 2014, Neural Computing and Applications.

[45]  Charles W. Anderson,et al.  Word Clustering as a Feature for Arabic Sentiment Classification , 2017 .

[46]  Ashutosh Kumar Singh,et al.  Machine Learning for High-Throughput Stress Phenotyping in Plants. , 2016, Trends in plant science.

[47]  M. Schatz,et al.  Big Data: Astronomical or Genomical? , 2015, PLoS biology.

[48]  Khaled Shaalan,et al.  A Survey of Arabic Text Mining , 2018 .

[49]  Sabrina Tiun,et al.  Comparative analysis of ML POS on Arabic tweets , 2017 .

[50]  Ashraf Odeh,et al.  Arabic Text Categorization Algorithm using Vector Evaluation Method , 2015, ArXiv.

[51]  Mohammad Subhi Al-Batah,et al.  Arabic Sentiment Classification using MLP Network Hybrid with Naive Bayes Algorithm , 2018 .

[52]  Tarek F. Gharib,et al.  Arabic Text Classification Using Support Vector Machines , 2009, Int. J. Comput. Their Appl..

[53]  U. Rajendra Acharya,et al.  Automated detection of coronary artery disease using different durations of ECG segments with convolutional neural network , 2017, Knowl. Based Syst..

[54]  Taha Tobaili.  Arabizi Identification in Twitter Data , 2016, ACL.