User reviews: Sentiment analysis using lexicon integrated two-channel CNN-LSTM​ family models

Abstract Sentiment analysis, which refers to the task of detecting whether a textual item (e.g., a product review and a blog post) expresses a positive or negative opinion in general or about a given entity (e.g., a product, person, or policy), has received increasing attention in recent years. It serves as an important role in natural language processing. User generated content, like tourism reviews, developed dramatically during the past years, generating a large amount of unstructured data from which it is hard to obtain useful information. Due to the changes in textual order, sequence length and complicated logic, it is still a challenging task to predict the exact sentiment polarities of the user reviews, especially for fine-grained sentiment classification. In this paper, we first propose sentiment padding, a novel padding method compared with zero padding, making the input data sample of a consistent size and improving the proportion of sentiment information in each review. Inspired by the most recent studies with respect to neural networks, we propose deep learning based sentiment analysis models named lexicon integrated two-channel CNN-LSTM family models, combining CNN and LSTM/BiLSTM branches in a parallel manner. Experiments on several challenging datasets, like Stanford Sentiment Treebank, demonstrate that the proposed method outperforms many baseline methods.

[1]  Eric P. Xing,et al.  Select-additive learning: Improving generalization in multimodal sentiment analysis , 2016, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[2]  Jong-Seok Lee,et al.  Data-driven integration of multiple sentiment dictionaries for lexicon-based sentiment classification of product reviews , 2014, Knowl. Based Syst..

[3]  Opinion Extraction , 2017, Encyclopedia of Machine Learning and Data Mining.

[4]  G. G. Stokes "J." , 1890, The New Yale Book of Quotations.

[5]  Saif Mohammad,et al.  Generating High-Coverage Semantic Orientation Lexicons From Overtly Marked Words and a Thesaurus , 2009, EMNLP.

[6]  Mahmoud Nabil,et al.  CUFE at SemEval-2016 Task 4: A Gated Recurrent Model for Sentiment Classification , 2016, *SEMEVAL.

[7]  Carlo Strapparava,et al.  WordNet Affect: an Affective Extension of WordNet , 2004, LREC.

[8]  Lingfeng Niu,et al.  Nonparallel Support Vector Ordinal Regression , 2017, IEEE Transactions on Cybernetics.

[9]  Cícero Nogueira dos Santos,et al.  Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts , 2014, COLING.

[10]  Erik Cambria,et al.  Benchmarking Multimodal Sentiment Analysis , 2017, CICLing.

[11]  Harith Alani,et al.  Contextual semantics for sentiment analysis of Twitter , 2016, Inf. Process. Manag..

[12]  Zhiyong Luo,et al.  Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts , 2016, COLING.

[13]  Lucy Vanderwende,et al.  MindNet: Acquiring and Structuring Semantic Information from Text , 1998, COLING-ACL.

[14]  Mathieu Cliche,et al.  BB_twtr at SemEval-2017 Task 4: Twitter Sentiment Analysis with CNNs and LSTMs , 2017, *SEMEVAL.

[15]  Haiqiang Chen,et al.  New Word Detection for Sentiment Analysis , 2014, ACL.

[16]  Hongfei Lin,et al.  Visual and Textual Sentiment Analysis of a Microblog Using Deep Convolutional Neural Networks , 2016, Algorithms.

[17]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[18]  Ming Zhou,et al.  Building Large-Scale Twitter-Specific Sentiment Lexicon : A Representation Learning Approach , 2014, COLING.

[19]  Milad Shokouhi,et al.  Query Expansion Using External Evidence , 2009, ECIR.

[20]  Yong Shi,et al.  DWWP: Domain-specific new words detection and word propagation system for sentiment analysis in the tourism domain , 2018, Knowl. Based Syst..

[21]  Patrik Lambert,et al.  Attention and Lexicon Regularized LSTM for Aspect-based Sentiment Analysis , 2019, ACL.

[22]  Erik Cambria,et al.  Convolutional MKL Based Multimodal Emotion Recognition and Sentiment Analysis , 2016, 2016 IEEE 16th International Conference on Data Mining (ICDM).

[23]  Yong Shi,et al.  Improved New Word Detection Method Used in Tourism Field , 2017, ICCS.

[24]  Alex Perelygin Semantic Compositionality and Deep Learning for Sentiment Analysis , 2012 .

[25]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[26]  Yu Zhou,et al.  A sequence level latent topic modeling method for sentiment analysis via CNN based Diversified Restrict Boltzmann Machine , 2016, 2016 International Conference on Machine Learning and Cybernetics (ICMLC).

[27]  Felipe Bravo-Marquez,et al.  Building a Twitter opinion lexicon from automatically-annotated tweets , 2016, Knowl. Based Syst..

[28]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[29]  Braja Gopal Patra,et al.  JU_CSE: A Conditional Random Field (CRF) Based Approach to Aspect Based Sentiment Analysis , 2014, *SEMEVAL.

[30]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[31]  Erik Cambria,et al.  Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-level Multimodal Sentiment Analysis , 2015, EMNLP.

[32]  Soumith Chintala Sentiment Analysis using neural architectures , 2011 .

[33]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[34]  P ? ? ? ? ? ? ? % ? ? ? ? , 1991 .

[35]  Minh-Quoc Nghiem,et al.  Combining Convolution and Recursive Neural Networks for Sentiment Analysis , 2017, SoICT.

[36]  Hsin-Hsi Chen,et al.  Opinion Extraction, Summarization and Tracking in News and Blog Corpora , 2006, AAAI Spring Symposium: Computational Approaches to Analyzing Weblogs.

[37]  Yunhong Wang,et al.  Visual and textual sentiment analysis using deep fusion convolutional neural networks , 2017, 2017 IEEE International Conference on Image Processing (ICIP).

[38]  Likun Qiu,et al.  Expanding Chinese Sentiment Dictionaries from Large Scale Unlabeled Corpus , 2010, PACLIC.

[39]  Usman Qamar,et al.  Lexicon based semantic detection of sentiments using expected likelihood estimate smoothed odds ratio , 2016, Artificial Intelligence Review.

[40]  Hong Yu,et al.  Neural Tree Indexers for Text Understanding , 2016, EACL.

[41]  SongYangqiu,et al.  Towards building a high-quality microblog-specific Chinese sentiment lexicon , 2016 .

[43]  Sasha Blair-Goldensohn,et al.  The viability of web-derived polarity lexicons , 2010, NAACL.

[44]  J. Koenderink Q… , 2014, Les noms officiels des communes de Wallonie, de Bruxelles-Capitale et de la communaute germanophone.

[45]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[46]  Shervin Minaee,et al.  Deep-Sentiment: Sentiment Analysis Using Ensemble of CNN and Bi-LSTM Models , 2019, ArXiv.

[47]  Zhiyong Huang,et al.  Chinese Micro-blog Sentiment Analysis Based on SVM and Complex Phrasing , 2016 .

[48]  Usman Qamar,et al.  SWIMS: Semi-supervised subjective feature weighting and intelligent model selection for sentiment analysis , 2016, Knowl. Based Syst..

[49]  Usman Qamar,et al.  eSAP: A decision support framework for enhanced sentiment analysis and polarity classification , 2016, Inf. Sci..

[50]  Zhongxia Zhang,et al.  Automatic Construction and Global Optimization of a Multisentiment Lexicon , 2016, Comput. Intell. Neurosci..

[51]  George A. Miller,et al.  Introduction to WordNet: An On-line Lexical Database , 1990 .

[52]  Multiclass Sentiment Prediction using Yelp Business Reviews , 2015 .

[53]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[54]  Taghi M. Khoshgoftaar,et al.  Improving deep neural network design with new text data representations , 2017, Journal of Big Data.

[55]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[56]  Walaa Medhat,et al.  Sentiment analysis algorithms and applications: A survey , 2014 .

[57]  Lijun Liu,et al.  Sentiment Analysis Using Convolutional Neural Network , 2015, 2015 IEEE International Conference on Computer and Information Technology; Ubiquitous Computing and Communications; Dependable, Autonomic and Secure Computing; Pervasive Intelligence and Computing.

[58]  Kyung-shik Shin,et al.  Attention-based long short-term memory network using sentiment lexicon embedding for aspect-level sentiment analysis in Korean , 2019, Inf. Process. Manag..

[59]  Hao Wang,et al.  Sentiment Analysis of Chinese Microblog Based on Stacked Bidirectional LSTM , 2018, 2018 15th International Symposium on Pervasive Systems, Algorithms and Networks (I-SPAN).

[60]  G. Wells,et al.  A multicenter, randomized, controlled clinical trial of transfusion requirements in critical care. Transfusion Requirements in Critical Care Investigators, Canadian Critical Care Trials Group. , 1999, The New England journal of medicine.

[61]  Xiaoyan Zhu,et al.  Measuring the Non-compositionality of Multiword Expressions , 2010, COLING.

[62]  Fangzhao Wu,et al.  Towards building a high-quality microblog-specific Chinese sentiment lexicon , 2016, Decis. Support Syst..

[63]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[64]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[65]  Hyeoncheol Kim,et al.  Multi-Channel Lexicon Integrated CNN-BiLSTM Models for Sentiment Analysis , 2017, ROCLING/IJCLCLP.

[66]  Guosheng Shao,et al.  Template-oriented synthesis of monodispersed SnS 2 @SnO 2 hetero-nanoflowers for Cr(VI) photoreduction , 2016 .

[67]  Yue Zhang,et al.  Gated Neural Networks for Targeted Sentiment Analysis , 2016, AAAI.

[68]  Yoshua Bengio,et al.  Understanding intermediate layers using linear classifier probes , 2016, ICLR.

[69]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[70]  Hongbo Xu,et al.  Adapting Naive Bayes to Domain Adaptation for Sentiment Analysis , 2009, ECIR.

[71]  Catuscia Palamidessi,et al.  Constructing elastic distinguishability metrics for location privacy , 2015, Proc. Priv. Enhancing Technol..

[72]  Steve Renals,et al.  Convolutional Neural Networks for Distant Speech Recognition , 2014, IEEE Signal Processing Letters.

[73]  Claire Cardie,et al.  Deep Recursive Neural Networks for Compositionality in Language , 2014, NIPS.

[74]  Bonggun Shin,et al.  Lexicon Integrated CNN Models with Attention for Sentiment Analysis , 2016, WASSA@EMNLP.

[75]  Jian Su,et al.  NLANGP at SemEval-2016 Task 5: Improving Aspect Based Sentiment Analysis using Neural Network Features , 2016, *SEMEVAL.

[76]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[77]  P. Deepa Shenoy,et al.  Aspect term extraction for sentiment analysis in large movie reviews using Gini Index feature selection method and SVM classifier , 2016, World Wide Web.

[78]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[79]  Ruixi Lin,et al.  Sentiment Classification of Food Reviews , 2016, ArXiv.

[80]  Ting Wang,et al.  Long Short-term Memory Network over Rhetorical Structure Theory for Sentence-level Sentiment Analysis , 2016, ACML.

[81]  Daling Wang,et al.  Unsupervised Learning Chinese Sentiment Lexicon from Massive Microblog Data , 2012, ADMA.

[82]  Erik Cambria,et al.  Sentic patterns: Dependency-based rules for concept-level sentiment analysis , 2014, Knowl. Based Syst..

[83]  K. Robert Lai,et al.  Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model , 2016, ACL.

[84]  Jiang Qian,et al.  Text sentiment analysis based on long short-term memory , 2016, 2016 First IEEE International Conference on Computer Communication and the Internet (ICCCI).

[85]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[86]  Mounia Mikram,et al.  A CNN-BiLSTM Model for Document-Level Sentiment Analysis , 2019, Mach. Learn. Knowl. Extr..

[87]  John G. Breslin,et al.  Financial Aspect and Sentiment Predictions with Deep Neural Networks: An Ensemble Approach , 2018, WWW.

[88]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[89]  Hazem M. Hajj,et al.  Deep Learning Models for Sentiment Analysis in Arabic , 2015, ANLP@ACL.