A Hybrid Approach for Sentiment Analysis Applied to Paper Reviews

Œis article discusses the problem of extracting sentiment and opinions about a collection of articles on scienti€c reviews conducted under an international conference on computing in Spanish language. Œe aim of this analysis is on the one hand to automatically determine the orientation of a review of an article and contrast this approach with the assessment made by the reviewer of the article. Œis would allow scientists to characterize and compare reviews crosswise, and more objectively support the overall assessment of a scienti€c article. A hybrid approach that combines an unsupervised machine learning algorithm with techniques from natural language processing is proposed to analyze reviews, and part-of-speech (POS) tagging to obtain the syntactic structure of a sentence. Œis syntactic structure, along with the use of dictionaries, allows to determine the semantic orientation of the review through a scoring algorithm. A set of experiments were conducted to evaluate the capability and performance of the proposed approaches relative to a baseline, using standard metrics, such as accuracy, precision, recall, and the F1-score. Œe results show improvements in the case of binary, ternary and a 5-point scale classi€cation in relation to classical machine learning algorithms such as SVM and NB, but they also present a challenge to improve the multiclass classi€cation in this domain.

[1]  Hanxiao Shi,et al.  A Supervised Fine-Grained Sentiment Analysis System for Online Reviews , 2015, Intell. Autom. Soft Comput..

[2]  David Zimbra,et al.  Twitter brand sentiment analysis: A hybrid system using n-gram analysis and dynamic artificial neural network , 2013, Expert Syst. Appl..

[3]  Lei Zhang,et al.  Combining lexicon-based and learning-based methods for twitter sentiment analysis , 2011 .

[4]  Erik Cambria,et al.  Affective Computing and Sentiment Analysis , 2016, IEEE Intelligent Systems.

[5]  "Thank you for your critical comments and helpful suggestions": compliance and conflict in authors' replies to referees' comments in peer reviews of scientific research papers , 2001 .

[6]  Tomas Mikolov,et al.  Bag of Tricks for Efficient Text Classification , 2016, EACL.

[7]  Iñaki Inza,et al.  Approaching Sentiment Analysis by using semi-supervised learning of multi-dimensional classifiers , 2012, Neurocomputing.

[8]  Yutaka Matsuo,et al.  Identifying Customer Preferences about Tourism Products Using an Aspect-based Opinion Mining Approach , 2013, KES.

[9]  Miss A.O. Penney.  (b) , 1974, The New Yale Book of Quotations.

[10]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[11]  Antonio Fernández Anta,et al.  Techniques for Sentiment Analysis and Topic Detection of Spanish Tweets: Preliminary Report , 2012 .

[12]  Ting Liu,et al.  Deep learning for sentiment analysis: successful approaches and future challenges , 2015, WIREs Data Mining Knowl. Discov..

[13]  Betty Samraj.  Discourse structure and variation in manuscript reviews: Implications for genre categorization , 2016 .

[14]  Dan Klein,et al.  Feature-Rich Part-of-Speech Tagging with a Cyclic Dependency Network , 2003, NAACL.

[15]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[16]  Michael Gamon,et al.  Customizing Sentiment Classifiers to New Domains: a Case Study , 2019 .

[17]  Rodica Potolea,et al.  Modeling Sentiment Polarity with Meta-features to Achieve Domain-Independence , 2014, IC3K.

[18]  Bo Xu,et al.  Parallel Recursive Deep Model for Sentiment Analysis , 2015, PAKDD.

[19]  Hugh Gosden,et al.  ‘Why not give us the full story?’: functions of referees’ comments in peer reviews of scientific research papers , 2003 .

[20]  Marie-Francine Moens,et al.  A machine learning approach to sentiment analysis in multilingual Web texts , 2009, Information Retrieval.

[21]  Ming Zhou,et al.  Coooolll: A Deep Learning System for Twitter Sentiment Classification , 2014, *SEMEVAL.

[22]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[23]  Miguel A. Alonso,et al.  A linguistic approach for determining the topics of Spanish Twitter messages , 2015, J. Inf. Sci..

[24]  Bing Liu,et al.  Web Data Mining: Exploring Hyperlinks, Contents, and Usage Data , 2006, Data-Centric Systems and Applications.

[25]  Ting Liu,et al.  Learning Semantic Representations of Users and Products for Document Level Sentiment Classification , 2015, ACL.

[26]  Gaël Varoquaux,et al.  Scikit-learn: Machine Learning in Python , 2011, J. Mach. Learn. Res..

[27]  Brian Paltridge,et al.  Referees' comments on submissions to peer-reviewed journals: when is a suggestion not a suggestion? , 2015 .

[28]  Erik Cambria,et al.  Sentic patterns: Dependency-based rules for concept-level sentiment analysis , 2014, Knowl. Based Syst..

[29]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[30]  Vadlamani Ravi,et al.  A survey on opinion mining and sentiment analysis: Tasks, approaches and applications , 2015, Knowl. Based Syst..

[31]  Miguel A. Alonso,et al.  A syntactic approach for opinion mining on Spanish reviews , 2013, Natural Language Engineering.

[32]  Enrique Herrera-Viedma,et al.  Sentiment analysis: A review and comparative analysis of web services , 2015, Inf. Sci..

[33]  Maite Taboada,et al.  Cross-Linguistic Sentiment Analysis: From English to Spanish , 2009, RANLP.

[34]  Thomas G. Dietterich,et al.  Solving Multiclass Learning Problems via Error-Correcting Output Codes , 1994, J. Artif. Intell. Res..

[35]  Lutz Bornmann,et al.  Scientific peer review , 2011, Annu. Rev. Inf. Sci. Technol..

[36]  Houda Benbrahim,et al.  An empirical study to address the problem of Unbalanced Data Sets in sentiment classification , 2012, 2012 IEEE International Conference on Systems, Man, and Cybernetics (SMC).

[37]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[38]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[39]  Mário Rodrigues,et al.  Advanced Applications of Natural Language Processing for Performing Information Extraction , 2015 .

[40]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[41]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[42]  Peter D. Turney.  Thumbs Up or Thumbs Down? Semantic Orientation Applied to Unsupervised Classification of Reviews , 2002, ACL.

[43]  Walaa Medhat,et al.  Sentiment analysis algorithms and applications: A survey , 2014 .