Short Text Sentiment Analysis Based on Multi-Channel CNN With Multi-Head Attention Mechanism

In view of the limited text features of short texts, features of short texts should be mined from various angles, and multiple sentiment feature combinations should be used to learn the hidden sentiment information. A novel sentiment analysis model based on multi-channel convolutional neural network with multi-head attention mechanism (MCNN-MA) is proposed. This model combines word features with part of speech features, position features and dependency syntax features separately to form three new combined features, and inputs them into the multi-channel convolutional neural network, as well as integrates the multi-head attention mechanism to more fully learn the sentiment information in the text. Finally, experiments are carried out on two Chinese short text data sets. The experimental results show that the MCNN-MA model has a higher classification accuracy and a relatively low training time cost compared with other baseline models.

[1]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[2]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[3]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[4]  Alan F. Smeaton,et al.  Classifying sentiment in microblogs: is brevity an advantage? , 2010, CIKM.

[5]  Maite Taboada,et al.  Lexicon-Based Methods for Sentiment Analysis , 2011, CL.

[6]  Yann LeCun,et al.  Very Deep Convolutional Networks for Text Classification , 2016, EACL.

[7]  Qiang Ye,et al.  Sentiment classification of online reviews to travel destinations by supervised machine learning approaches , 2009, Expert Syst. Appl..

[8]  Zhiwei Zhao,et al.  Attention-Based Convolutional Neural Networks for Sentence Classification , 2016, INTERSPEECH.

[9]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[10]  Xin Wang,et al.  Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory , 2015, ACL.

[11]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[12]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[13]  Marie-Francine Moens,et al.  A machine learning approach to sentiment analysis in multilingual Web texts , 2009, Information Retrieval.

[14]  Yangsen Zhang,et al.  Study of Sentiment Classification for Chinese Microblog Based on Recurrent Neural Network , 2016 .

[15]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[16]  Tao Chen,et al.  Combining Convolutional Neural Network and Support Vector Machine for Sentiment Classification , 2015, SMP.