Arabic Opinion Mining Using Combined CNN - LSTM Models

In the last few years, Sentiment Analysis regarding customers' reviews in order to comprehend the opinion polarity on social media has received considerable attention. However, the improvement of deep learning for sentiment analysis relating to customer reviews in Arabic language has received less attention. In fact, many users post and jot down their reviews in Arabic daily, so we ought to shed more light on Arabic sentiment analysis. Most likely all previous work depends on conventional classification techniques, such as KNN, Naïve Bayes (NB), etc. But in this work, we implement two deep learning models: Long Short Term Memory (LSTM) and Convolution Neural Networks (CNN), in addition to three traditional techniques: Naïve Bayes, K-Nearest Neighbor (KNN), Decision trees for sentiment analysis and compared the experimental results. Also, we offer a combined model from CNN and Recurrent Neural Network (RNN) architecture where this model collects local features through CNN as the input for RNN for Arabic sentiment analysis of short texts. An appropriate data preparation has been conducted for each utilized dataset. Our Conducted experiments for each dataset against traditional machine learning classifier; KNN, NB, and decision trees and regular deep learning models; CNN and LSTM, has resulted in impressive performance using our proposed combined (CNN-LSTM) model with an average accuracy of 85,83%, 86,88% for HTL and LABR datasets respectively.

[1]  Yann LeCun,et al.  Very Deep Convolutional Networks for Natural Language Processing , 2016, ArXiv.

[2]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[3]  Lukás Burget,et al.  Recurrent neural network based language model , 2010, INTERSPEECH.

[4]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[5]  Samhaa R. El-Beltagy,et al.  Combining Lexical Features and a Supervised Learning Approach for Arabic Sentiment Analysis , 2016, CICLing.

[6]  Gouda I. Salama,et al.  A novel approach for ontology-based dimensionality reduction for web text document classification , 2017, 2017 IEEE/ACIS 16th International Conference on Computer and Information Science (ICIS).

[7]  Gouda I. Salama,et al.  Sentiment Analysis on Twitter Data using Apache Spark Framework , 2018, 2018 13th International Conference on Computer Engineering and Systems (ICCES).

[8]  Christopher Meek,et al.  Semantic Parsing for Single-Relation Question Answering , 2014, ACL.

[9]  Mayuri A. Mehta,et al.  Techniques for sentiment analysis of Twitter data: A comprehensive survey , 2016, 2016 International Conference on Computing, Communication and Automation (ICCCA).

[10]  Yunfang Chen,et al.  A survey on sentiment analysis by using machine learning methods , 2017, 2017 IEEE 2nd Information Technology, Networking, Electronic and Automation Control Conference (ITNEC).

[11]  Ahmad Ali,et al.  Sentiment Analysis on Twitter Data using KNN and SVM , 2017 .

[12]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[13]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[14]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[15]  Min-Yuh Day,et al.  Deep Learning for Sentiment Analysis on Google Play Consumer Review , 2017, 2017 IEEE International Conference on Information Reuse and Integration (IRI).

[16]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Mustafa Hammad,et al.  Sentiment Analysis for Arabic Reviews in Social Networks Using Machine Learning , 2016 .

[18]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[19]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[20]  Peerapon Vateekul,et al.  A study of sentiment analysis using deep learning techniques on Thai Twitter data , 2016, 2016 13th International Joint Conference on Computer Science and Software Engineering (JCSSE).

[21]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[22]  Jiang Qian,et al.  Text sentiment analysis based on long short-term memory , 2016, 2016 First IEEE International Conference on Computer Communication and the Internet (ICCCI).

[23]  Hosny M. Ibrahim,et al.  Detecting Twitter Users' Opinions of Arabic Comments During Various Time Episodes via Deep Neural Network , 2017, AISI.

[24]  Geoffrey E. Hinton,et al.  ImageNet classification with deep convolutional neural networks , 2012, Commun. ACM.

[25]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[26]  Khaled Shaalan,et al.  Arabic Tweets Sentimental Analysis Using Machine Learning , 2017, IEA/AIE.

[27]  Samhaa R. El-Beltagy,et al.  Building Large Arabic Multi-domain Resources for Sentiment Analysis , 2015, CICLing.

[28]  Alessandro Moschitti,et al.  Twitter Sentiment Analysis with Deep Convolutional Neural Networks , 2015, SIGIR.

[29]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[30]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[31]  Li Deng,et al.  A tutorial survey of architectures, algorithms, and applications for deep learning , 2014, APSIPA Transactions on Signal and Information Processing.

[32]  Mohammad Rahmati,et al.  Sentiment analysis using deep learning on Persian texts , 2017, 2017 Iranian Conference on Electrical Engineering (ICEE).

[33]  Lixin Tao,et al.  Word embeddings for Arabic sentiment analysis , 2016, 2016 IEEE International Conference on Big Data (Big Data).

[34]  Amir F. Atiya,et al.  LABR: A Large Scale Arabic Book Reviews Dataset , 2013, ACL.

[35]  B. K. Tripathy,et al.  Investigation of recurrent neural networks in the field of sentiment analysis , 2017, 2017 International Conference on Communication and Signal Processing (ICCSP).

[36]  Taha Osman,et al.  Challenges in Sentiment Analysis for Arabic Social Networks , 2017, ACLING.

[37]  Khaled Nagi,et al.  Sentiment Analysis of Colloquial Arabic Tweets , 2014 .

[38]  Hazem M. Hajj,et al.  Deep Learning Models for Sentiment Analysis in Arabic , 2015, ANLP@ACL.

[39]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[40]  Walaa Medhat,et al.  Sentiment analysis algorithms and applications: A survey , 2014 .

[41]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[42]  Thorsten Joachims,et al.  Text Categorization with Support Vector Machines: Learning with Many Relevant Features , 1998, ECML.