Reliability of Sentiment Mining Tools: A comparison of Semantria and Social Mention

Social Media platforms have become quintessential for user-generated content and consumer opinions. As a result, vast amounts of commercially and freely available sentiment mining tools have emerged, however it remains unclear how reliable these tools are. In this paper, I evaluate the reliability as well as the features of the sentiment mining tools Semantria and Social Mention. This study is a sentiment analysis of 12 different car models that were applied to both sentiment mining tools. In addition to presenting a comparison of outputs obtained from two different sentiment mining tools, an analysis was conducted that compares the sentiment and passion outputs for three social media platforms. The results show significant differences in outputs for sentiment mining tools. Furthermore, statistically signifcant as well as observational differences were also apparent for the outputs from three social media platforms. The results have theoretical as well practical implications for different groups, including academics and practitioners using sentiment mining tools as input for future research and decision-making.

[1]  Johan Bollen,et al.  Twitter mood predicts the stock market , 2010, J. Comput. Sci..

[2]  David A. Ostrowski.  Sentiment Mining within Social Media for Topic Identification , 2010, 2010 IEEE Fourth International Conference on Semantic Computing.

[3]  Jonathon Read,et al.  Using Emoticons to Reduce Dependency in Machine Learning Techniques for Sentiment Classification , 2005, ACL.

[4]  Wei Hu,et al.  Twitter Sentiment in Data Streams with Perceptron , 2014 .

[5]  Abeywardena Ishan Sudeera.  Public opinion on OER and MOOC: a sentiment analysis of twitter data , 2014 .

[6]  Gilad Mishne,et al.  Predicting Movie Sales from Blogger Sentiment , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[7]  Eric K. Ringger,et al.  Pulse: Mining Customer Opinions from Free Text , 2005, IDA.

[8]  Hong Yu,et al.  Towards Answering Opinion Questions: Separating Facts from Opinions and Identifying the Polarity of Opinion Sentences , 2003, EMNLP.

[9]  Bernard J. Jansen,et al.  Micro-blogging as online word of mouth branding , 2009, CHI Extended Abstracts.

[10]  Andrea Esuli,et al.  SENTIWORDNET: A Publicly Available Lexical Resource for Opinion Mining , 2006, LREC.

[11]  Patrick Paroubek,et al.  Twitter as a Corpus for Sentiment Analysis and Opinion Mining , 2010, LREC.

[12]  Lillian Lee,et al.  Opinion Mining and Sentiment Analysis , 2008, Found. Trends Inf. Retr..

[13]  H. Varian,et al.  Predicting the Present with Google Trends , 2009 .

[14]  Mike Thelwall,et al.  Sentiment in Twitter events , 2011, J. Assoc. Inf. Sci. Technol..

[15]  Simon Fong,et al.  Opinion mining over twitterspace: Classifying tweets programmatically using the R approach , 2012, Seventh International Conference on Digital Information Management (ICDIM 2012).

[16]  Ari Rappoport,et al.  Enhanced Sentiment Learning Using Twitter Hashtags and Smileys , 2010, COLING.

[17]  Michael I. Barrett,et al.  Methodological Implications of Critical Realism for Mixed-Methods Research , 2013, MIS Q..

[18]  Mike Thelwall,et al.  Information-centered research for large-scale analyses of new information sources , 2008 .

[19]  Bernardo A. Huberman,et al.  Predicting the Future with Social Media , 2010, Web Intelligence.

[20]  Janyce Wiebe,et al.  Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis , 2005, HLT.

[21]  Isabell M. Welpe,et al.  Predicting Elections with Twitter: What 140 Characters Reveal about Political Sentiment , 2010, ICWSM.

[22]  Johanna D. Moore,et al.  Twitter Sentiment Analysis: The Good the Bad and the OMG! , 2011, ICWSM.

[23]  Soo-Min Kim,et al.  Determining the Sentiment of Opinions , 2004, COLING.

[24]  Bernard J. Jansen,et al.  Twitter power: Tweets as electronic word of mouth , 2009 .

[25]  Shaowen Wang,et al.  Mapping the global Twitter heartbeat: The geography of Twitter , 2013, First Monday.

[26]  Aliza Sarlan,et al.  Twitter sentiment analysis , 2014, Proceedings of the 6th International Conference on Information Technology and Multimedia.