SNNN: Promoting Word Sentiment and Negation in Neural Sentiment Classification

We mainly investigate word influence in neural sentiment classification, which results in a novel approach to promoting word sentiment and negation as attentions. Particularly, a sentiment and negation neural network (SNNN) is proposed, including a sentiment neural network (SNN) and a negation neural network (NNN). First, we modify the word level by embedding the word sentiment and negation information as the extra layers for the input. Second, we adopt a hierarchical LSTM model to generate the word-level, sentence-level and document-level representations respectively. After that, we enhance word sentiment and negation as attentions over the semantic level. Finally, the experiments conducting on the IMDB and Yelp data sets show that our approach is superior to the state-of-the-art methods. Furthermore, we draw the interesting conclusions that (1) LSTM performs better than CNN and RNN for neural sentiment classification; (2) word sentiment and negation are a strong alliance as attentions, while overfitting occurs when they are simultaneously applied at the embedding layer; and (3) word sentiment/negation can be singly implemented for better performance as both embedding layer and attention at the same time. Introduction and Motivation Many approaches in sentiment classification, utilize a supervised classifier and rely on extensive feature engineering (Go, Bhayani, and Huang 2009; Barbosa and Feng 2010; Pak and Paroubek 2010; Jiang et al. 2011; Mukherjee, Bhattacharyya, and others 2012; Hamdan, Béchet, and Bellot 2013; Mohammad, Kiritchenko, and Zhu 2013; Cheng et al. 2017). However, feature engineering costs extensive labour work and needs specific domain knowledge. Therefore, feature learning is an alternative way to learn discriminative features automatically from data. The work presented by (Socher et al. 2013; Yessenalina and Cardie 2011; Hu et al. 2016) proved that the features of a sentence/document could be learnt through its word embedding. Existing approaches of learning word embedding (Collobert et al. 2011; Mikolov et al. 2013b; Yang, Hu, and He 2015) then focused on modeling the syntactic context. After that, people turn to neural network for its learning ability of text representation (Glorot, Bordes, and Bengio 2011; Zhai and Zhang Copyright c © 2018, Association for the Advancement of Artificial Intelligence (www.aaai.org). All rights reserved. 2016; Socher et al. 2011a; 2011b; 2012; 2013; Kim 2014; Tang, Qin, and Liu 2015b; Yang et al. 2016; Chen et al. 2016; Ren et al. 2016a) Tang et al. (Tang, Qin, and Liu 2015b) proposed a neural network model to learn vector-based document representation in a CNN based sentiment classification, where the authors found that neural gates outperformed the traditional recurrent neural network. Then, Chen et al. (Chen et al. 2016) brought a hierarchical neural network to incorporate global user and product information as attentions. They mainly challenged Tang’s work that the characteristics of the user and product information should be reflected on the semantic level, instead of the word level. Based on these two pieces of the state-ofthe-art work, we aim to investigate further word influence in neural sentiment classification. The motivation is that it is theoretically feasible and sound by adding more word information from multiple dimensions in the word level as the input, since the quality of document/sentence representation highly depends on word representations. I positive it not negative output it