Evaluating SentiStrength for Arabic Sentiment Analysis

Social networking websites are used today as platforms enabling their users to write down almost anything about everything. Social media users express their opinions and feelings about lots of events occurring in their daily lives. Lots of studies are conducted to study the sentiments presented by social media users regarding different topics. Sentiment Analysis (SA) is a new field that is concerned with measuring the sentiment presented in a given text. Due to their wide set of applications, several SA tools are available. Most of them are designed for English text. As for other languages such as Arabic, the case is different since only few tools are available. In fact, many of these tools were originally designed for English and were later adapted to deal with Arabic. SentiStrength is an example of tools that are successful for English and were later adapted to Arabic. However, the adaptation has been done in a crude manner and no deep studies are available to measure the effectiveness of such tools for Arabic text. In this paper, we perform a comprehensive evaluation of SentiStrength using 11 Arabic datasets consisting of tens of thousands of reviews/comments from different domains and in different dialects. We perform the evaluation in terms of positive and negative sentiments. The evaluation results show that overall SentiStrength achieves 62% accuracy, 83.7% precision, 64% recall (positive correct), 68% F1 measure and 55% negative correct.

[1]  Suad Alhojely,et al.  Sentiment Analysis and Opinion Mining: A Survey , 2016 .

[2]  Miguel A. Alonso,et al.  The megaphone of the people? Spanish SentiStrength for real-time analysis of political tweets , 2015, J. Inf. Sci..

[3]  Mahmoud Al-Ayyoub,et al.  Arabic sentiment analysis: Lexicon-based and corpus-based , 2013, 2013 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT).

[4]  Mahmoud Al-Ayyoub,et al.  Enhancing the determination of aspect categories and their polarities in Arabic reviews using lexicon-based approaches , 2015, 2015 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT).

[5]  Pete Burnap,et al.  Arabic Event Detection in Social Media , 2015, CICLing.

[6]  Mahmoud Al-Ayyoub,et al.  An analytical study of Arabic sentiments: Maktoob case study , 2013, 8th International Conference for Internet Technology and Secured Transactions (ICITST-2013).

[7]  Mahmoud Al-Ayyoub,et al.  Using Aspect-Based Sentiment Analysis to Evaluate Arabic News Affect on Readers , 2015, 2015 IEEE/ACM 8th International Conference on Utility and Cloud Computing (UCC).

[8]  Mahmoud Al-Ayyoub,et al.  Framework for affective news analysis of Arabic news: 2014 Gaza attacks case study , 2016, 2016 7th International Conference on Information and Communication Systems (ICICS).

[9]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[10]  Mahmoud Al-Ayyoub,et al.  Towards Improving the Lexicon-Based Approach for Arabic Sentiment Analysis , 2014, Int. J. Inf. Technol. Web Eng..

[11]  Mahmoud Al-Ayyoub,et al.  Lexicon-based sentiment analysis of Arabic tweets , 2015, Int. J. Soc. Netw. Min..

[12]  Izzat Alsmadi,et al.  Sentiment analysis of arabic social media content: a comparative study , 2013, 8th International Conference for Internet Technology and Secured Transactions (ICITST-2013).

[13]  Samhaa R. El-Beltagy,et al.  Building Large Arabic Multi-domain Resources for Sentiment Analysis , 2015, CICLing.

[14]  Muhammad Abdul-Mageed,et al.  AWATIF: A Multi-Genre Corpus for Modern Standard Arabic Subjectivity and Sentiment Analysis , 2012, LREC.

[15]  Bashar Al Shboul,et al.  Multi-way sentiment classification of Arabic reviews , 2015, 2015 6th International Conference on Information and Communication Systems (ICICS).

[16]  Mahmoud Al-Ayyoub,et al.  Are emoticons good enough to train emotion classifiers of Arabic tweets? , 2016, 2016 7th International Conference on Computer Science and Information Technology (CSIT).

[17]  Mahmoud Al-Ayyoub,et al.  On the Use of Arabic Tweets to Predict Stock Market Changes in the Arab World , 2016 .

[18]  Mahmoud Al-Ayyoub,et al.  Automatic Lexicon Construction for Arabic Sentiment Analysis , 2014, 2014 International Conference on Future Internet of Things and Cloud.

[19]  Mahmoud Al-Ayyoub,et al.  Hierarchical Classifiers for Multi-Way Sentiment Analysis of Arabic Reviews , 2016 .

[20]  Amir F. Atiya,et al.  LABR: A Large Scale Arabic Book Reviews Dataset , 2013, ACL.

[21]  Amir F. Atiya,et al.  ASTD: Arabic Sentiment Tweets Dataset , 2015, EMNLP.

[22]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[23]  Luis Alfonso Ureña López,et al.  OCA: Opinion corpus for Arabic , 2011, J. Assoc. Inf. Sci. Technol..

[24]  Shiri Dori-Hacohen.  Controversy Detection and Stance Analysis , 2015, SIGIR.

[25]  Marie-Francine Moens,et al.  Automatic Sentiment Analysis in On-line Text , 2007, ELPUB.

[26]  Alaa M. El-Halees,et al.  Arabic Opinion Mining Using Combined Classification Approach , 2011 .

[27]  Mahmoud Al-Ayyoub,et al.  An extended analytical study of Arabic sentiments , 2014, Int. J. Big Data Intell..

[28]  Mahmoud Al-Ayyoub,et al.  Measuring the controversy level of Arabic trending topics on Twitter , 2016, 2016 7th International Conference on Information and Communication Systems (ICICS).

[29]  Mahmoud Al-Ayyoub,et al.  A prototype for a standard arabic sentiment analysis corpus , 2016, Int. Arab J. Inf. Technol..

[30]  Mahmoud Al-Ayyoub,et al.  Emotion analysis of Arabic articles and its impact on identifying the author's gender , 2015, 2015 IEEE/ACS 12th International Conference of Computer Systems and Applications (AICCSA).

[31]  TaboadaMaite,et al.  Lexicon-based methods for sentiment analysis , 2011 .