SentiVec: Learning Sentiment-Context Vector via Kernel Optimization Function for Sentiment Analysis

Deep learning-based sentiment analysis (SA) methods have drawn more attention in recent years, which calls for more precise word embedding methods. This article proposes SentiVec, a kernel optimization function system for sentiment word embedding, which is based on two phases. The first phase is a supervised learning method, and the second phase consists of two unsupervised updating models, object-word-to-surrounding-words reward model (O2SR) and context-to-object-word reward model (C2OR). SentiVec is aimed at: 1) integrating the statistical information and sentiment orientation into sentiment word vectors and 2) propagating and updating the semantic information to all the word representations in a corpus. Extensive experimental results show that the optimal sentiment vectors successfully extract the features in terms of semantic and sentiment information, which makes it outperform the baseline methods on word similarity, word analogy, and SA tasks.

[1]  Ronan Collobert,et al.  Word Embeddings through Hellinger PCA , 2013, EACL.

[2]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[3]  Navneet Kaur,et al.  Opinion mining and sentiment analysis , 2016, 2016 3rd International Conference on Computing for Sustainable Global Development (INDIACom).

[4]  Jiayin Qi,et al.  Subjective well-being measurement based on Chinese grassroots blog text sentiment analysis , 2015, Inf. Manag..

[5]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[6]  R. Harrison,et al.  Perceptrons in Kernel Feature Spaces , 1998 .

[7]  Quan Pan,et al.  Learning Word Representations for Sentiment Analysis , 2017, Cognitive Computation.

[8]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[9]  Fangzhao Wu,et al.  Towards building a high-quality microblog-specific Chinese sentiment lexicon , 2016, Decis. Support Syst..

[10]  Daniel E. O'Leary,et al.  Blog mining-review and extensions: "From each according to his opinion" , 2011, Decis. Support Syst..

[11]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[12]  Xiaoyan Zhu,et al.  Sentiment Analysis with Global Topics and Local Dependency , 2010, AAAI.

[13]  Kenneth Ward Church,et al.  Very sparse random projections , 2006, KDD '06.

[14]  Richard A. Harshman,et al.  Indexing by Latent Semantic Analysis , 1990, J. Am. Soc. Inf. Sci..

[15]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[16]  Ehud Rivlin,et al.  Placing search in context: the concept revisited , 2002, TOIS.

[17]  Lidong Bing,et al.  Learning Domain-Sensitive and Sentiment-Aware Word Embeddings , 2018, ACL.

[18]  Erik Cambria,et al.  SenticNet 5: Discovering Conceptual Primitives for Sentiment Analysis by Means of Context Embeddings , 2018, AAAI.

[19]  Pedro E. López-de-Teruel,et al.  Nonlinear kernel-based statistical pattern analysis , 2001, IEEE Trans. Neural Networks.

[20]  Harith Alani,et al.  Contextual semantics for sentiment analysis of Twitter , 2016, Inf. Process. Manag..

[21]  Yoshua Bengio,et al.  Word Representations: A Simple and General Method for Semi-Supervised Learning , 2010, ACL.

[22]  Jimmy J. Lin,et al.  Quantitative evaluation of passage retrieval algorithms for question answering , 2003, SIGIR.

[23]  K. Robert Lai,et al.  Refining Word Embeddings for Sentiment Analysis , 2017, EMNLP.

[24]  Thorsten Joachims,et al.  Evaluation methods for unsupervised word embeddings , 2015, EMNLP.

[25]  Michelle L. Gregory,et al.  Quantifying sentiment and influence in blogspaces , 2010, SOMA '10.

[26]  Georgiana Dinu,et al.  Don’t count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors , 2014, ACL.

[27]  Erik Cambria,et al.  Affective Computing and Sentiment Analysis , 2016, IEEE Intelligent Systems.

[28]  Bernhard Schölkopf,et al.  The Kernel Trick for Distances , 2000, NIPS.

[29]  Philip Resnik,et al.  Holistic Sentiment Analysis Across Languages: Multilingual Supervised Latent Dirichlet Allocation , 2010, EMNLP.

[30]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[31]  Yong Shi,et al.  DWWP: Domain-specific new words detection and word propagation system for sentiment analysis in the tourism domain , 2018, Knowl. Based Syst..

[32]  Erik Cambria,et al.  AffectiveSpace 2: Enabling Affective Intuition for Concept-Level Sentiment Analysis , 2015, AAAI.

[33]  Bin Zhou,et al.  Detecting real-time burst topics in microblog streams: how sentiment can help , 2013, WWW '13 Companion.

[34]  Andrew Y. Ng,et al.  Parsing with Compositional Vector Grammars , 2013, ACL.

[35]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[36]  Bo Pang,et al.  A Sentimental Education: Sentiment Analysis Using Subjectivity Summarization Based on Minimum Cuts , 2004, ACL.

[37]  Xue Bai,et al.  Predicting consumer sentiments from online text , 2011, Decis. Support Syst..

[38]  Zheng Lin,et al.  Learning Sentiment-Specific Word Embedding via Global Sentiment Representation , 2018, AAAI.

[39]  Andrea Esuli,et al.  SentiWordNet 3.0: An Enhanced Lexical Resource for Sentiment Analysis and Opinion Mining , 2010, LREC.

[40]  Ming Zhou,et al.  Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification , 2014, ACL.

[41]  Ting Liu,et al.  Document Modeling with Gated Recurrent Neural Network for Sentiment Classification , 2015, EMNLP.

[42]  Khurshid Ahmad,et al.  Sentiment Polarity Identification in Financial News: A Cohesion-based Approach , 2007, ACL.

[43]  Patrick Pantel,et al.  From Frequency to Meaning: Vector Space Models of Semantics , 2010, J. Artif. Intell. Res..

[44]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[45]  Kyoungok Kim,et al.  Sentiment visualization and classification via semi-supervised nonlinear dimensionality reduction , 2014, Pattern Recognit..

[46]  Owen Rambow,et al.  Sentiment Analysis of Twitter Data , 2011 .

[47]  Wei Li,et al.  User reviews: Sentiment analysis using lexicon integrated two-channel CNN-LSTM​ family models , 2020, Appl. Soft Comput..

[48]  Dean P. Foster,et al.  Two Step CCA: A new spectral method for estimating vector models of words , 2012, ICML 2012.

[49]  Yoshua. Bengio,et al.  Learning Deep Architectures for AI , 2007, Found. Trends Mach. Learn..

[50]  Geoffrey E. Hinton,et al.  Three new graphical models for statistical language modelling , 2007, ICML '07.

[51]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[52]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[53]  E. Cambria,et al.  AffectiveSpace: Blending Common Sense and Affective Knowledge to Perform Emotive Reasoning , 2009 .

[54]  Jason Weston,et al.  Natural Language Processing (Almost) from Scratch , 2011, J. Mach. Learn. Res..

[55]  Long-Sheng Chen,et al.  Journal of Informetrics , 2022 .