Text Sentiment Analysis Based on Parallel Recursive Constituency Tree-LSTM

With the rapid development of mobile Internet, online platforms such as social entertainment and e-commerce have brought a large number of unstructured texts containing sentiment polarity. Sentiment analysis of these texts can be used for public opinion analysis, product review and other tasks. Recent researches show great interest and success in using deep learning methods for sentiment analysis tasks. Among these contributions, Constituency Tree-LSTM has achieved good results because of its superior ability and tree structure to preserve sequence information over time. However, the model calculates the sentiment label of the node based mainly on its vector representation, while missing the sentiment information of its children. This paper proposes a parallel recursive Constituency Tree-LSTM (PRCT-LSTM), which combines recursive neural network with Constituency Tree-LSTM to achieve better performance. The node sentiment information propagates to the global evaluation through recursive neural networks. Therefore, the sentiment information of each node in the PRCT-LSTM structure consists of sentiment information generated by Constituency Tree-LSTM and the recursive model. In this way, we take more full usage of sentiment information. Experimental results manifest that the proposed model achieves a higher performance than the Constituency Tree-LSTM.

[1]  Mirella Lapata,et al.  Composition in Distributional Models of Semantics , 2010, Cogn. Sci..

[2]  Zhiyuan Liu,et al.  A C-LSTM Neural Network for Text Classification , 2015, ArXiv.

[3]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[4]  Jeffrey Pennington,et al.  Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions , 2011, EMNLP.

[5]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[6]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[7]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[8]  K. Robert Lai,et al.  Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model , 2016, ACL.

[9]  Hongyu Guo,et al.  Long Short-Term Memory Over Recursive Structures , 2015, ICML.

[10]  Yanjun Qi,et al.  Sentiment classification based on supervised latent n-gram analysis , 2011, CIKM '11.

[11]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[12]  Xiao Sun,et al.  Sentiment analysis for Chinese microblog based on deep neural networks with convolutional extension features , 2016, Neurocomputing.

[13]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[14]  Klaus-Robert Müller,et al.  Efficient BackProp , 2012, Neural Networks: Tricks of the Trade.

[15]  Samy Bengio,et al.  Show and tell: A neural image caption generator , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[16]  Ye Zhang,et al.  Rationale-Augmented Convolutional Neural Networks for Text Classification , 2016, EMNLP.

[17]  Horacio Saggion,et al.  Are Emojis Predictable? , 2017, EACL.

[18]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[19]  Geoffrey E. Hinton,et al.  Speech recognition with deep recurrent neural networks , 2013, 2013 IEEE International Conference on Acoustics, Speech and Signal Processing.

[20]  Yoram Singer,et al.  Adaptive Subgradient Methods for Online Learning and Stochastic Optimization , 2011, J. Mach. Learn. Res..

[21]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[22]  Quoc V. Le,et al.  Sequence to Sequence Learning with Neural Networks , 2014, NIPS.

[23]  Dan Klein,et al.  Accurate Unlexicalized Parsing , 2003, ACL.