Exploring the Use of Word Embedding and Deep Learning in Arabic Sentiment Analysis

In the past couple of years, improving Arabic sentiment Analysis systems has been one of the important fields of research. There are several challenges and issues facing existing systems, especially, handling multiple dialects and feature extraction. Most of those systems are generated using linear classification models and traditional bag-of-word features. In this work, we explore the use of word embedding as a modern feature representation, and Convolutional Neural Networks as a Deep Neural Network in a sentiment classification of Arabic texts. The application of our model on five benchmark datasets has yielded results that outperform previous works on 4 out of 5 datasets.

[1]  Trevor Darrell,et al.  Rich Feature Hierarchies for Accurate Object Detection and Semantic Segmentation , 2013, 2014 IEEE Conference on Computer Vision and Pattern Recognition.

[2]  Yann Dauphin,et al.  Convolutional Sequence to Sequence Learning , 2017, ICML.

[3]  Amir F. Atiya,et al.  ASTD: Arabic Sentiment Tweets Dataset , 2015, EMNLP.

[4]  Samhaa R. El-Beltagy,et al.  Combining Lexical Features and a Supervised Learning Approach for Arabic Sentiment Analysis , 2016, CICLing.

[5]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[6]  Mohsen Rashwan,et al.  Word Representations in Vector Space and their Applications for Arabic , 2015, CICLing.

[7]  Pengfei Duan,et al.  Word Embeddings and Convolutional Neural Network for Arabic Sentiment Classification , 2016, COLING.

[8]  Samhaa R. El-Beltagy,et al.  AraVec: A set of Arabic Word Embedding Models for use in Arabic NLP , 2017, ACLING.

[9]  Chris Callison-Burch,et al.  Machine Translation of Arabic Dialects , 2012, NAACL.

[10]  Hatem Haddad,et al.  Empirical Evaluation of Word Representations on Arabic Sentiment Analysis , 2017, ICALP.

[11]  Hazem M. Hajj,et al.  Deep Learning Models for Sentiment Analysis in Arabic , 2015, ANLP@ACL.

[12]  Zellig S. Harris,et al.  Distributional Structure , 1954 .

[13]  Verena Rieser,et al.  An Arabic Twitter Corpus for Subjectivity and Sentiment Analysis , 2014, LREC.

[14]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[15]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[16]  Saif Mohammad,et al.  How Translation Alters Sentiment , 2016, J. Artif. Intell. Res..

[17]  Yelong Shen,et al.  Learning semantic representations using convolutional neural networks for web search , 2014, WWW.

[18]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[19]  Gerald Penn,et al.  Applying Convolutional Neural Networks concepts to hybrid NN-HMM model for speech recognition , 2012, 2012 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).

[20]  Saif Mohammad,et al.  Sentiment after Translation: A Case-Study on Arabic Social Media Posts , 2015, NAACL.

[21]  Kareem Darwish,et al.  Subjectivity and Sentiment Analysis of Modern Standard Arabic and Arabic Microblogs , 2013, WASSA@NAACL-HLT.

[22]  Christopher Meek,et al.  Semantic Parsing for Single-Relation Question Answering , 2014, ACL.

[23]  Steven Skiena,et al.  Polyglot: Distributed Word Representations for Multilingual NLP , 2013, CoNLL.

[24]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[25]  Nitish Srivastava,et al.  Improving neural networks by preventing co-adaptation of feature detectors , 2012, ArXiv.

[26]  Samhaa R. El-Beltagy.  NileTMRG at SemEval-2016 Task 7: Deriving Prior Polarities for Arabic Sentiment Terms , 2016, SemEval@NAACL-HLT.

[27]  Kunihiko Fukushima,et al.  Neocognitron: A hierarchical neural network capable of visual pattern recognition , 1988, Neural Networks.

[28]  Raddouane Chiheb,et al.  Sentiment analysis in Arabic: A review of the literature , 2017, Ain Shams Engineering Journal.

[29]  Mahmoud Al-Ayyoub,et al.  Hierarchical Classifiers for Multi-Way Sentiment Analysis of Arabic Reviews , 2016 .

[30]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[31]  David R. Karger,et al.  Tackling the Poor Assumptions of Naive Bayes Text Classifiers , 2003, ICML.

[32]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[33]  Kaiming He,et al.  Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks , 2015, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[34]  Arzucan Özgür,et al.  Improving Named Entity Recognition for Morphologically Rich Languages Using Word Embeddings , 2014, 2014 13th International Conference on Machine Learning and Applications.