Deep learning for sentiment analysis: A survey

Deep learning has emerged as a powerful machine learning technique that learns multiple layers of representations or features of the data and produces state‐of‐the‐art prediction results. Along with the success of deep learning in many application domains, deep learning is also used in sentiment analysis in recent years. This paper gives an overview of deep learning and then provides a comprehensive survey of its current applications in sentiment analysis.

[1]  Pushpak Bhattacharyya,et al.  Learning Cognitive Features from Gaze Data for Sentiment and Sarcasm Classification using Convolutional Neural Network , 2017, ACL.

[2]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[3]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[4]  Honglak Lee,et al.  Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations , 2009, ICML '09.

[5]  Zhiyuan Liu,et al.  Neural Sentiment Classification with User and Product Attention , 2016, EMNLP.

[6]  Claire Cardie,et al.  Deep Recursive Neural Networks for Compositionality in Language , 2014, NIPS.

[7]  Jeremy Barnes,et al.  Exploring Distributional Representations and Machine Translation for Aspect-based Cross-lingual Sentiment Classification , 2016, COLING.

[8]  K. Robert Lai,et al.  Refining Word Embeddings for Sentiment Analysis , 2017, EMNLP.

[9]  Jeffrey Pennington,et al.  Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions , 2011, EMNLP.

[10]  Iyad Rahwan,et al.  Using millions of emoji occurrences to learn any-domain representations for detecting sentiment, emotion and sarcasm , 2017, EMNLP.

[11]  Yue Zhang,et al.  Attention Modeling for Targeted Sentiment , 2017, EACL.

[12]  Hod Lipson,et al.  Re-embedding words , 2013, ACL.

[13]  Xin Li,et al.  Deep Multi-Task Learning for Aspect Term Extraction with Memory Interaction , 2017, EMNLP.

[14]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[15]  Siu Cheung Hui,et al.  Dyadic Memory Networks for Aspect-based Sentiment Analysis , 2017, CIKM.

[16]  K. Robert Lai,et al.  Dimensional Sentiment Analysis Using a Regional CNN-LSTM Model , 2016, ACL.

[17]  Xiaojun Wan,et al.  Attention-based LSTM Network for Cross-Lingual Sentiment Classification , 2016, EMNLP.

[18]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[19]  Jeffrey L. Elman,et al.  Finding Structure in Time , 1990, Cogn. Sci..

[20]  Yu Zhang,et al.  End-to-End Adversarial Memory Network for Cross-domain Sentiment Classification , 2017, IJCAI.

[21]  Ting Liu,et al.  Learning Semantic Representations of Users and Products for Document Level Sentiment Classification , 2015, ACL.

[22]  Geoffrey E. Hinton,et al.  Learning representations by back-propagating errors , 1986, Nature.

[23]  Xiaoyan Zhu,et al.  Encoding Syntactic Knowledge in Neural Networks for Sentiment Classification , 2017, ACM Trans. Inf. Syst..

[24]  Wei Zhang,et al.  Collaborative Multi-Level Embedding Learning from Reviews for Rating Prediction , 2016, IJCAI.

[25]  Erik Cambria,et al.  Tensor Fusion Network for Multimodal Sentiment Analysis , 2017, EMNLP.

[26]  Samit Bhattacharya,et al.  Using Deep and Convolutional Neural Networks for Accurate Emotion Classification on DEAP Dataset , 2017, AAAI.

[27]  Yue Zhang,et al.  Improving Twitter Sentiment Classification Using Topic-Enriched Multi-Prototype Word Embeddings , 2016, AAAI.

[28]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[29]  Andrew Y. Ng,et al.  Improving Word Representations via Global Context and Multiple Word Prototypes , 2012, ACL.

[30]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[31]  Yue Zhang,et al.  Context-Sensitive Lexicon Features for Neural Sentiment Analysis , 2016, EMNLP.

[32]  Bing Liu,et al.  Sentiment Lexicon Expansion Based on Neural PU Learning, Double Dictionary Lookup, and Polarity Association , 2017, EMNLP.

[33]  Chu-Ren Huang,et al.  A Cognition Based Attention Model for Sentiment Analysis , 2017, EMNLP.

[34]  Pushpak Bhattacharyya,et al.  Sentiment Intensity Ranking among Adjectives Using Sentiment Bearing Word Embeddings , 2017, EMNLP.

[35]  Yue Zhang,et al.  Tweet Sarcasm Detection Using Deep Neural Network , 2016, COLING.

[36]  Cícero Nogueira dos Santos,et al.  Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts , 2014, COLING.

[37]  Koray Kavukcuoglu,et al.  Learning word embeddings efficiently with noise-contrastive estimation , 2013, NIPS.

[38]  Claire Cardie,et al.  Extracting Opinion Expressions with semi-Markov Conditional Random Fields , 2012, EMNLP.

[39]  Abhishek Kumar,et al.  A Multilayer Perceptron based Ensemble Technique for Fine-grained Financial Sentiment Analysis , 2017, EMNLP.

[40]  John G. Breslin,et al.  A Hierarchical Model of Reviews for Aspect-based Sentiment Analysis , 2016, EMNLP.

[41]  Claire Cardie,et al.  Opinion Mining with Deep Recurrent Neural Networks , 2014, EMNLP.

[42]  Xiaoyan Zhu,et al.  Emotional Chatting Machine: Emotional Conversation Generation with Internal and External Memory , 2017, AAAI.

[43]  Yue Zhang,et al.  Opinion Recommendation Using A Neural Model , 2017, EMNLP.

[44]  Claire Cardie,et al.  Joint Inference for Fine-grained Opinion Extraction , 2013, ACL.

[45]  Shafiq R. Joty,et al.  Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings , 2015, EMNLP.

[46]  Ming Zhou,et al.  Unsupervised Word and Dependency Path Embeddings for Aspect Term Extraction , 2016, IJCAI.

[47]  Andrew G. Clark,et al.  Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics (ACL) , 2002 .

[48]  Pengfei Duan,et al.  Word Embeddings and Convolutional Neural Network for Arabic Sentiment Classification , 2016, COLING.

[49]  Yoshua Bengio,et al.  Deep Sparse Rectifier Neural Networks , 2011, AISTATS.

[50]  Geoffrey E. Hinton,et al.  Reducing the Dimensionality of Data with Neural Networks , 2006, Science.

[51]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[52]  Xiaoyan Zhu,et al.  Linguistically Regularized LSTM for Sentiment Classification , 2016, ACL.

[53]  Diyi Yang,et al.  Hierarchical Attention Networks for Document Classification , 2016, NAACL.

[54]  Iryna Gurevych,et al.  CNN- and LSTM-based Claim Classification in Online User Comments , 2016, COLING.

[55]  Ming Zhou,et al.  Sentiment Embeddings with Applications to Sentiment Analysis , 2016, IEEE Transactions on Knowledge and Data Engineering.

[56]  Yang Liu,et al.  Learning Tag Embeddings and Tag-specific Composition Functions in Recursive Neural Network , 2015, ACL.

[57]  Janyce Wiebe,et al.  Development and Use of a Gold-Standard Data Set for Subjectivity Classifications , 1999, ACL.

[58]  Yue Zhang,et al.  Neural Networks for Open Domain Targeted Sentiment , 2015, EMNLP.

[59]  Yoshua Bengio,et al.  Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling , 2014, ArXiv.

[60]  Daniel Jurafsky,et al.  Do Multi-Sense Embeddings Improve Natural Language Understanding? , 2015, EMNLP.

[61]  Houfeng Wang,et al.  Interactive Attention Networks for Aspect-Level Sentiment Classification , 2017, IJCAI.

[62]  Jürgen Schmidhuber,et al.  Framewise phoneme classification with bidirectional LSTM and other neural network architectures , 2005, Neural Networks.

[63]  Yue Zhang,et al.  Distance Metric Learning for Aspect Phrase Grouping , 2016, COLING.

[64]  Xiaokui Xiao,et al.  Recursive Neural Conditional Random Fields for Aspect-based Sentiment Analysis , 2016, EMNLP.

[65]  Xuanjing Huang,et al.  Cached Long Short-Term Memory Neural Networks for Document-Level Sentiment Classification , 2016, EMNLP.

[66]  Yoshua Bengio,et al.  Neural Machine Translation by Jointly Learning to Align and Translate , 2014, ICLR.

[67]  Qin Lu,et al.  A Question Answering Approach for Emotion Cause Extraction , 2017, EMNLP.

[68]  Tony Veale,et al.  Magnets for Sarcasm: Making Sarcasm Detection Timely, Contextual and Very Personal , 2017, EMNLP.

[69]  Yangqiu Song,et al.  Document-Level Multi-Aspect Sentiment Classification as Machine Comprehension , 2017, EMNLP.

[70]  Lun-Wei Ku,et al.  WordForce: Visualizing Controversial Words in Debates , 2016, COLING.

[71]  Erik Cambria,et al.  Deep Convolutional Neural Network Textual Features and Multiple Kernel Learning for Utterance-level Multimodal Sentiment Analysis , 2015, EMNLP.

[72]  Jufeng Yang,et al.  Learning Visual Sentiment Distributions via Augmented Conditional Probability Neural Network , 2017, AAAI.

[73]  Jianfei Yu,et al.  Learning Sentence Embeddings with Auxiliary Tasks for Cross-Domain Sentiment Classification , 2016, EMNLP.

[74]  Muhammad Abdul-Mageed,et al.  EmoNet: Fine-Grained Emotion Detection with Gated Recurrent Neural Networks , 2017, ACL.

[75]  Xiaojun Wan,et al.  Representation Learning for Aspect Category Detection in Online Reviews , 2015, AAAI.

[76]  Yue Zhang,et al.  Target-Dependent Twitter Sentiment Classification with Rich Automatic Features , 2015, IJCAI.

[77]  Erik Cambria,et al.  A Deeper Look into Sarcastic Tweets Using Deep Convolutional Neural Networks , 2016, COLING.

[78]  Lidong Bing,et al.  Recurrent Attention Network on Memory for Aspect Sentiment Analysis , 2017, EMNLP.

[79]  Pushpak Bhattacharyya,et al.  A Hybrid Deep Learning Architecture for Sentiment Analysis , 2016, COLING.

[80]  Manish Shrivastava,et al.  Towards Sub-Word Level Compositions for Sentiment Analysis of Hindi-English Code Mixed Text , 2016, COLING.

[81]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[82]  Pushpak Bhattacharyya,et al.  Are Word Embedding-based Features Useful for Sarcasm Detection? , 2016, EMNLP.

[83]  Christopher D. Manning,et al.  Improved Semantic Representations From Tree-Structured Long Short-Term Memory Networks , 2015, ACL.

[84]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[85]  Yue Zhang,et al.  Gated Neural Networks for Targeted Sentiment Analysis , 2016, AAAI.

[86]  Pascale Fung,et al.  Real-Time Speech Emotion and Sentiment Recognition for Interactive Dialogue Systems , 2016, EMNLP.

[87]  João Francisco Valiati,et al.  Document-level sentiment classification: An empirical comparison between SVM and ANN , 2013, Expert Syst. Appl..

[88]  Yueting Zhuang,et al.  Microblog Sentiment Classification via Recurrent Random Walk Network Learning , 2017, IJCAI.

[89]  Tao Mei,et al.  Beyond Object Recognition: Visual Sentiment Analysis with Deep Coupled Adjective and Noun Neural Networks , 2016, IJCAI.

[90]  Tong Zhang,et al.  Effective Use of Word Order for Text Categorization with Convolutional Neural Networks , 2014, NAACL.

[91]  Kang Liu,et al.  Book Review: Sentiment Analysis: Mining Opinions, Sentiments, and Emotions by Bing Liu , 2015, CL.

[92]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[93]  Yu Lei,et al.  Content-based Influence Modeling for Opinion Behavior Prediction , 2016, COLING.

[94]  Long Chen,et al.  Learning Bilingual Sentiment Word Embeddings for Cross-language Sentiment Classification , 2015, ACL.

[95]  Jason Weston,et al.  End-To-End Memory Networks , 2015, NIPS.

[96]  Claire Cardie,et al.  Investigating LSTMs for Joint Extraction of Opinion Entities and Relations , 2016, ACL.

[97]  Yi Zheng,et al.  Weakly-Supervised Deep Learning for Customer Review Sentiment Classification , 2016, IJCAI.

[98]  Jiebo Luo,et al.  Visual Sentiment Analysis by Attending on Local Image Regions , 2017, AAAI.

[99]  Zi-Yi Dou,et al.  Capturing User and Product Information for Document Level Sentiment Analysis with Deep Memory Network , 2017, EMNLP.

[100]  Janyce Wiebe,et al.  Recognizing Opinion Sources Based on a New Categorization of Opinion Types , 2016, IJCAI.

[101]  Ming Zhou,et al.  Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification , 2014, ACL.

[102]  Pushpak Bhattacharyya,et al.  Borrow a Little from your Rich Cousin: Using Embeddings and Polarities of English Words for Multilingual Sentiment Classification , 2016, COLING.

[103]  Yann LeCun,et al.  Traffic sign recognition with multi-scale Convolutional Networks , 2011, The 2011 International Joint Conference on Neural Networks.

[104]  Qin Lu,et al.  Intersubjectivity and Sentiment: From Language to Knowledge , 2016, IJCAI.

[105]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[106]  Yoshua Bengio,et al.  A Neural Probabilistic Language Model , 2003, J. Mach. Learn. Res..

[107]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[108]  Yanjun Qi,et al.  Sentiment classification based on supervised latent n-gram analysis , 2011, CIKM '11.

[109]  Alex Graves,et al.  Neural Turing Machines , 2014, ArXiv.

[110]  Yoav Goldberg,et al.  A Primer on Neural Network Models for Natural Language Processing , 2015, J. Artif. Intell. Res..

[111]  Eric P. Xing,et al.  Select-additive learning: Improving generalization in multimodal sentiment analysis , 2016, 2017 IEEE International Conference on Multimedia and Expo (ICME).

[112]  Roi Reichart,et al.  Sarcasm SIGN: Interpreting Sarcasm with Sentiment Based Monolingual Machine Translation , 2017, ACL.

[113]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[114]  Ruifeng Xu,et al.  Stance Classification with Target-specific Neural Attention , 2017, IJCAI.

[115]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[116]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[117]  Zhongfei Zhang,et al.  Semisupervised Autoencoder for Sentiment Analysis , 2015, AAAI.

[118]  Allan Hanbury,et al.  Volatility Prediction using Financial Disclosures Sentiments with Word Embedding-based IR Models , 2017, ACL.

[119]  Xiaocheng Feng,et al.  Effective LSTMs for Target-Dependent Sentiment Classification , 2015, COLING.

[120]  Yoshua Bengio,et al.  Hierarchical Probabilistic Neural Network Language Model , 2005, AISTATS.

[121]  Yejin Choi,et al.  Multilingual Connotation Frames: A Case Study on Social Media for Targeted Sentiment Analysis and Forecast , 2017, ACL.

[122]  Zhiyong Luo,et al.  Combination of Convolutional and Recurrent Neural Network for Sentiment Analysis of Short Texts , 2016, COLING.

[123]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[124]  Mark Steedman,et al.  Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics (ACL) , 2015 .

[125]  Erik Cambria,et al.  Context-Dependent Sentiment Analysis in User-Generated Videos , 2017, ACL.

[126]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[127]  Cheng Li,et al.  Deep Memory Networks for Attitude Identification , 2017, WSDM.

[128]  Véronique Hoste,et al.  Monday mornings are my fave : ) #not Exploring the Automatic Recognition of Irony in English tweets , 2016, COLING.

[129]  Jason Weston,et al.  Memory Networks , 2014, ICLR.

[130]  Qingming Huang,et al.  Dependency Exploitation: A Unified CNN-RNN Approach for Visual Emotion Recognition , 2017, IJCAI.

[131]  Quoc V. Le,et al.  Distributed Representations of Sentences and Documents , 2014, ICML.

[132]  Min Yang,et al.  Attention Based LSTM for Target Dependent Sentiment Classification , 2017, AAAI.

[133]  Ting Liu,et al.  Aspect Level Sentiment Classification with Deep Memory Network , 2016, EMNLP.

[134]  Jianfei Yu,et al.  Recurrent Neural Networks with Auxiliary Labels for Cross-Domain Opinion Target Extraction , 2017, AAAI.

[135]  Yoshua Bengio,et al.  Greedy Layer-Wise Training of Deep Networks , 2006, NIPS.

[136]  Hwee Tou Ng,et al.  An Unsupervised Neural Attention Model for Aspect Extraction , 2017, ACL.

[137]  Pascal Vincent,et al.  Representation Learning: A Review and New Perspectives , 2012, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[138]  Hongyu Guo,et al.  Neural Networks for Integrating Compositional and Non-compositional Sentiment in Sentiment Composition , 2015, *SEMEVAL.

[139]  Yue Zhang,et al.  A Bilingual Attention Network for Code-switched Emotion Prediction , 2016, COLING.

[140]  Le Yi Wang,et al.  Sentiment Lexicon Construction with Representation Learning Based on Hierarchical Sentiment Supervision , 2017, EMNLP.

[141]  Xin Wang,et al.  Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory , 2015, ACL.

[142]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[143]  Pascale Fung,et al.  Zara: A Virtual Interactive Dialogue System Incorporating Emotion, Sentiment and Personality Recognition , 2016, COLING.

[144]  Xiaokui Xiao,et al.  Coupled Multi-Layer Attentions for Co-Extraction of Aspect and Opinion Terms , 2017, AAAI.

[145]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[146]  Kalina Bontcheva,et al.  Stance Detection with Bidirectional Conditional Encoding , 2016, EMNLP.

[147]  Regina Barzilay,et al.  Rationalizing Neural Predictions , 2016, EMNLP.