Stacked denoising autoencoders for sentiment analysis: a review

Deep learning has been shown to outperform numerous conventional machine learning algorithms (e.g., support vector machines) in many fields, such as image processing and text analyses. This is due to its outstanding capability to model complex data distributions. However, as networks become deeper, there is an increased risk of overfitting and higher sensitivity to noise. Stacked denoising autoencoders (SDAs) provide an infrastructure to resolve these issues. In the field of sentiment recognition from textual contents, SDAs have been widely used (especially for domain adaptation), and have been consistently refined and improved through defining new alternate topologies as well as different learning algorithms. A wide selection of these approaches are reviewed and compared in this article. The results coming from the reviewed works indicate the promising capability of SDAs to perform sentiment recognition on a multitude of domains and languages. WIREs Data Mining Knowl Discov 2017, 7:e1212. doi: 10.1002/widm.1212

[1]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[2]  Kilian Q. Weinberger,et al.  Marginalized Denoising Autoencoders for Domain Adaptation , 2012, ICML.

[3]  Dit-Yan Yeung,et al.  Relational Stacked Denoising Autoencoder for Tag Recommendation , 2015, AAAI.

[4]  Ruifang Liu,et al.  Stacked denoising autoencoder and dropout together to prevent overfitting in deep neural network , 2015, 2015 8th International Congress on Image and Signal Processing (CISP).

[5]  Kyunghyun Cho,et al.  Simple Sparsification Improves Sparse Denoising Autoencoders in Denoising Highly Corrupted Images , 2013, ICML.

[6]  Zhongfei Zhang,et al.  Semisupervised Autoencoder for Sentiment Analysis , 2015, AAAI.

[7]  Pascal Vincent,et al.  Stacked Denoising Autoencoders: Learning Useful Representations in a Deep Network with a Local Denoising Criterion , 2010, J. Mach. Learn. Res..

[8]  Long Chen,et al.  Cross-Lingual Sentiment Classification Based on Denoising Autoencoder , 2014, NLPCC.

[9]  Alberto Del Bimbo,et al.  A multimodal feature learning approach for sentiment analysis of social network multimedia , 2016, Multimedia Tools and Applications.

[10]  Pengfei Wei,et al.  Deep Nonlinear Feature Coding for Unsupervised Domain Adaptation , 2016, IJCAI.

[11]  Daumé,et al.  Frustratingly Easy Semi-Supervised Domain Adaptation , 2010 .

[12]  Preslav Nakov,et al.  SemEval-2013 Task 2: Sentiment Analysis in Twitter , 2013, *SEMEVAL.

[13]  Mariusz Kleć,et al.  Sparse Autoencoders in Sentiment Analysis , 2014 .

[14]  Michael I. Jordan,et al.  Latent Dirichlet Allocation , 2001, J. Mach. Learn. Res..

[15]  Björn W. Schuller,et al.  Sentiment analysis and opinion mining: on optimal parameters and performances , 2015, WIREs Data Mining Knowl. Discov..

[16]  Kurt Hornik,et al.  Neural networks and principal component analysis: Learning from examples without local minima , 1989, Neural Networks.

[17]  Guigang Zhang,et al.  Deep Learning , 2016, Int. J. Semantic Comput..

[18]  C. D. Broad,et al.  Emotion and Sentiment , 1954 .

[19]  Erkki Sutinen,et al.  Are They Different? Affect, Feeling, Emotion, Sentiment, and Opinion Detection in Text , 2014, IEEE Transactions on Affective Computing.

[20]  K. Geras,et al.  Composite Denoising Autoencoders , 2016, ECML/PKDD.

[21]  Kenneth Heafield,et al.  Proceedings of the The 53rd Annual Meeting of the Association for Computational Linguistics and The 7th International Joint Conference of the Asian Federation of Natural Language Processing , 2015 .

[22]  François Laviolette,et al.  Domain-Adversarial Training of Neural Networks , 2015, J. Mach. Learn. Res..

[23]  Jiashen Sun,et al.  Incorporating Domain and Sentiment Supervision in Representation Learning for Domain Adaptation , 2015, IJCAI.

[24]  Ivor W. Tsang,et al.  Hybrid Heterogeneous Transfer Learning through Deep Learning , 2014, AAAI.

[25]  Gabriela Csurka,et al.  A Domain Adaptation Regularization for Denoising Autoencoders , 2016, ACL.

[26]  John Blitzer,et al.  Biographies, Bollywood, Boom-boxes and Blenders: Domain Adaptation for Sentiment Classification , 2007, ACL.

[27]  Yong Peng,et al.  Marginalized Denoising Autoencoder via Graph Regularization for Domain Adaptation , 2013, ICONIP.

[28]  Changsheng Xu,et al.  Boosted Multifeature Learning for Cross-Domain Transfer , 2015, ACM Trans. Multim. Comput. Commun. Appl..

[29]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[30]  John Blitzer,et al.  Domain Adaptation with Structural Correspondence Learning , 2006, EMNLP.

[31]  Raymond B. Cattell,et al.  SENTIMENT OR ATTITUDE? THE CORE OF A TERMINOLOGY PROBLEM IN PERSONALITY RESEARCH , 1940 .

[32]  Korris Fu-Lai Chung,et al.  The l2, 1-Norm Stacked Robust Autoencoders for Domain Adaptation , 2016, AAAI.

[33]  Karl-Franzens-Universitaet,et al.  SOCIOLOGY OF EMOTIONS , 2011 .

[34]  K. P. Chow,et al.  LCCT: A Semi-supervised Model for Sentiment Classification , 2015, NAACL.

[35]  Hong Liu,et al.  Cross-domain sentiment classification using deep learning approach , 2014, 2014 IEEE 3rd International Conference on Cloud Computing and Intelligence Systems.

[36]  Long Chen,et al.  Learning Bilingual Sentiment Word Embeddings for Cross-language Sentiment Classification , 2015, ACL.

[37]  Wan-Yu Deng,et al.  Instance-Wise Denoising Autoencoder for High Dimensional Data , 2016 .

[38]  Charles A. Sutton,et al.  Scheduled denoising autoencoders , 2015, ICLR.

[39]  Kilian Q. Weinberger,et al.  Marginalized Stacked Denoising Autoencoders , 2012 .

[40]  Ying Zhang,et al.  Occlusion-Robust Face Recognition Using Iterative Stacked Denoising Autoencoder , 2013, ICONIP.

[41]  Pascal Vincent,et al.  Manifold Parzen Windows , 2002, NIPS.

[42]  Jie Lu,et al.  Transfer Learning in Hierarchical Feature Spaces , 2015, 2015 10th International Conference on Intelligent Systems and Knowledge Engineering (ISKE).

[43]  Mohamad Ivan Fanany,et al.  Online marginalized linear stacked denoising autoencoders for learning from big data stream , 2015, 2015 International Conference on Advanced Computer Science and Information Systems (ICACSIS).

[44]  Yann LeCun,et al.  Modeles connexionnistes de l'apprentissage , 1987 .

[45]  Hinrich Schütze,et al.  Multi-Domain Sentiment Relevance Classification with Automatic Representation Learning , 2014, EACL.

[46]  Yoshua Bengio,et al.  Extracting and composing robust features with denoising autoencoders , 2008, ICML '08.

[47]  Yoshua Bengio,et al.  Domain Adaptation for Large-Scale Sentiment Classification: A Deep Learning Approach , 2011, ICML.

[48]  Wenge Rong,et al.  Auto-encoder based bagging architecture for sentiment analysis , 2014, J. Vis. Lang. Comput..

[49]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[50]  Geoffrey E. Hinton,et al.  Stochastic Neighbor Embedding , 2002, NIPS.