Sentiment Classification Using a Single-Layered BiLSTM Model

This study presents a computationally efficient deep learning model for binary sentiment classification, which aims to decide the sentiment polarity of people’s opinions, attitudes, and emotions expressed in written text. To achieve this, we exploited three widely practiced datasets based on public opinions about movies. We utilized merely one bidirectional long short-term memory (BiLSTM) layer along with a global pooling mechanism and achieved an accuracy of 80.500%, 85.780%, and 90.585% on MR, SST2 and IMDb datasets, respectively. We concluded that the performance metrics of our proposed approach are competitive with the recently published models, having comparatively complex architectures. Also, it is inferred that the proposed single-layered BiLSTM based architecture is computationally efficient and can be recommended for real-time applications in the field of sentiment analysis.

[1]  Christopher D. Manning,et al.  Baselines and Bigrams: Simple, Good Sentiment and Topic Classification , 2012, ACL.

[2]  Wei Zhao,et al.  Investigating the transferring capability of capsule networks for text classification , 2019, Neural Networks.

[3]  Cheng Zhao,et al.  Feature-Based Fusion Adversarial Recurrent Neural Networks for Text Sentiment Classification , 2019, IEEE Access.

[4]  Kuldip K. Paliwal,et al.  Bidirectional recurrent neural networks , 1997, IEEE Trans. Signal Process..

[5]  Jianqiang Li,et al.  Lexicon-Enhanced LSTM With Attention for General Sentiment Analysis , 2018, IEEE Access.

[6]  Yaohui Jin,et al.  A Generalized Recurrent Neural Architecture for Text Classification with Multi-Task Learning , 2017, IJCAI.

[7]  Jiafu Wan,et al.  Deep Learning Based Weighted Feature Fusion Approach for Sentiment Analysis , 2019, IEEE Access.

[8]  Guixian Xu,et al.  Sentiment Analysis of Comment Texts Based on BiLSTM , 2019, IEEE Access.

[9]  Bo Pang,et al.  Seeing Stars: Exploiting Class Relationships for Sentiment Categorization with Respect to Rating Scales , 2005, ACL.

[10]  Santanu Kumar Rath,et al.  Classification of sentiment reviews using n-gram machine learning approach , 2016, Expert Syst. Appl..

[11]  Zhiyong Feng,et al.  LSTM with sentence representations for document-level sentiment classification , 2018, Neurocomputing.

[12]  Jeffrey Dean,et al.  Efficient Estimation of Word Representations in Vector Space , 2013, ICLR.

[13]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[14]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[15]  Li Zhao,et al.  Attention-based LSTM for Aspect-level Sentiment Classification , 2016, EMNLP.

[16]  Nitish Srivastava,et al.  Dropout: a simple way to prevent neural networks from overfitting , 2014, J. Mach. Learn. Res..

[17]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[18]  Long Tian,et al.  Combining Convolution Neural Network and Bidirectional Gated Recurrent Unit for Sentence Semantic Classification , 2018, IEEE Access.

[19]  Yoav Goldberg,et al.  A Primer on Neural Network Models for Natural Language Processing , 2015, J. Artif. Intell. Res..

[20]  Shiliang Sun,et al.  A review of natural language processing techniques for opinion mining systems , 2017, Inf. Fusion.

[21]  Shuai Wang,et al.  Deep learning for sentiment analysis: A survey , 2018, WIREs Data Mining Knowl. Discov..

[22]  Yoav Goldberg,et al.  Neural Network Methods for Natural Language Processing , 2017, Synthesis Lectures on Human Language Technologies.

[23]  Chu-Ren Huang,et al.  A Cognition Based Attention Model for Sentiment Analysis , 2017, EMNLP.

[24]  Ye Zhang,et al.  A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification , 2015, IJCNLP.

[25]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[26]  Jean Oh,et al.  Attention-based Multimodal Neural Machine Translation , 2016, WMT.

[27]  Mike Thelwall,et al.  More than Bag-of-Words: Sentence-based Document Representation for Sentiment Analysis , 2013, RANLP.

[28]  Andrew Y. Ng,et al.  Semantic Compositionality through Recursive Matrix-Vector Spaces , 2012, EMNLP.

[29]  Bing Liu,et al.  Opinion spam and analysis , 2008, WSDM '08.

[30]  José Camacho-Collados,et al.  On the Role of Text Preprocessing in Neural Network Architectures: An Evaluation Study on Text Categorization and Sentiment Analysis , 2017, BlackboxNLP@EMNLP.

[31]  Igor Mozetic,et al.  Multilingual Twitter Sentiment Classification: The Role of Human Annotators , 2016, PloS one.

[32]  João Francisco Valiati,et al.  Document-level sentiment classification: An empirical comparison between SVM and ANN , 2013, Expert Syst. Appl..

[33]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[34]  Siti Mariyam Shamsuddin,et al.  Deep learning-based sentiment classification of evaluative text based on Multi-feature fusion , 2019, Inf. Process. Manag..

[35]  Arun Kumar Sangaiah,et al.  Using convolution control block for Chinese sentiment analysis , 2017, J. Parallel Distributed Comput..

[36]  Marie-Francine Moens,et al.  A machine learning approach to sentiment analysis in multilingual Web texts , 2009, Information Retrieval.

[37]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[38]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[39]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.