Bayesian Deep Convolution Belief Networks for Subjectivity Detection

Subjectivity detection aims to distinguish natural language as either opinionated (positive or negative) or neutral. In word vector based convolutional neural network models, a word meaning is simply a signal that helps to classify larger entities such as a document. Previous works do not usually consider prior distribution when using sliding windows to learn word embedding's and, hence, they are unable to capture higher-order and long-range features in text. In this paper, we employ dynamic Gaussian Bayesian networks to learn significant network motifs of words and concepts. These motifs are used to pre-train the convolutional neural network and capture the dynamics of discourse across several sentences.

[1]  Christopher D. Manning,et al.  Fast dropout training , 2013, ICML.

[2]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[3]  Erik Cambria,et al.  Sentiment Data Flow Analysis by Means of Dynamic Linguistic Patterns , 2015, IEEE Computational Intelligence Magazine.

[4]  Yulan He,et al.  Sentence Subjectivity Detection with Weakly-Supervised Learning , 2011, IJCNLP.

[5]  Erik Cambria,et al.  Towards an intelligent framework for multimodal affective data analysis , 2015, Neural Networks.

[6]  Erik Cambria,et al.  Fusing audio, visual and textual clues for sentiment analysis from multimodal content , 2016, Neurocomputing.

[7]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[8]  Erik Cambria,et al.  Affective Computing and Sentiment Analysis , 2016, IEEE Intelligent Systems.

[9]  Björn W. Schuller,et al.  SenticNet 4: A Semantic Resource for Sentiment Analysis Based on Conceptual Primitives , 2016, COLING.

[10]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[11]  Dirk Van den Poel,et al.  Dynamic Bayesian Networks for Acquisition Pattern Analysis: A Financial-Services Cross-Sell Application , 2009, PAKDD Workshops.

[12]  Ruslan Salakhutdinov,et al.  Multimodal Neural Language Models , 2014, ICML.

[13]  Jason Weston,et al.  A unified architecture for natural language processing: deep neural networks with multitask learning , 2008, ICML '08.

[14]  Phil Blunsom,et al.  Recurrent Convolutional Neural Networks for Discourse Compositionality , 2013, CVSM@ACL.

[15]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[16]  B. Schölkopf,et al.  Modeling Human Motion Using Binary Latent Variables , 2007 .

[17]  Geoffrey E. Hinton.  Training Products of Experts by Minimizing Contrastive Divergence , 2002, Neural Computation.

[18]  Janyce Wiebe.  Subjectivity Word Sense Disambiguation , 2009, EMNLP 2009.

[19]  Erik Cambria,et al.  Aspect extraction for opinion mining with a deep convolutional neural network , 2016, Knowl. Based Syst..

[20]  Marco Bonzanini,et al.  Opinion summarisation through sentence extraction: an investigation with movie reviews , 2012, SIGIR '12.

[21]  Jun Suzuki,et al.  Sequence and Tree Kernels with Statistical Feature Mining , 2005, NIPS.

[22]  Kevin P. Murphy,et al.  Learning the Structure of Dynamic Probabilistic Networks , 1998, UAI.

[23]  Ellen Riloff,et al.  Learning Extraction Patterns for Subjective Expressions , 2003, EMNLP.

[24]  Erik Cambria,et al.  Sentic Computing: A Common-Sense-Based Framework for Concept-Level Sentiment Analysis , 2015 .

[25]  Songbo Tan,et al.  A survey on sentiment detection of reviews , 2009, Expert Syst. Appl..

[26]  Phil Blunsom,et al.  A Convolutional Neural Network for Modelling Sentences , 2014, ACL.

[27]  B. Schölkopf,et al.  Isotonic Conditional Random Fields and Local Sentiment Flow , 2007 .

[28]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[29]  Ellen Riloff,et al.  Creating Subjective and Objective Sentence Classifiers from Unannotated Texts , 2005, CICLing.

[30]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[31]  Giuseppe Carenini,et al.  Subjectivity detection in spoken and written conversations , 2010, Natural Language Engineering.

[32]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[33]  Hang Li,et al.  A Deep Architecture for Matching Short Texts , 2013, NIPS.

[34]  Andrés Montoyo,et al.  Improving Subjectivity Detection using Unsupervised Subjectivity Word Sense Disambiguation , 2013, Proces. del Leng. Natural.