Single and Multibranch CNN-Bidirectional LSTM for IMDb Sentiment Analysis

Online users now frequently use the internet to voice opinions, ask for advice, or choose products and services based on the feedback of others. This provides a window into the way users feel about specific topics. The study of natural language processing, a sub-category of sentiment analysis, takes on this task by extracting meaning out of user text through observing the way in which words are grouped and used. Machine learning techniques have made significant advances which allow us to further explore mechanisms for interpreting such data. This research aims to use the internet movie database (IMDb) dataset and the Keras API to compare single and multibranch CNN-Bidirectional LSTMs of various kernel sizes (Maas et al. Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 142–150, 2011; Chollet et al. Keras. 2015. https://keras.io). The results show that while only time to train varies between single and multibranch models, their maximum accuracies are close in range. The highest accuracy model was the single branch with kernel size 9 with an accuracy of 89.54%. While slightly more accurate than the multibranch model with 88.94%, the time savings for the single branch is approximately of 64% (2 h and 20 min).

[1]  Yoon Kim,et al.  Convolutional Neural Networks for Sentence Classification , 2014, EMNLP.

[2]  Jürgen Schmidhuber,et al.  Long Short-Term Memory , 1997, Neural Computation.

[3]  Dumitru Erhan,et al.  Going deeper with convolutions , 2014, 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[4]  Bo Pang,et al.  Thumbs up? Sentiment Classification using Machine Learning Techniques , 2002, EMNLP.

[5]  Ke Zhang,et al.  Video Summarization with Long Short-Term Memory , 2016, ECCV.

[6]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[7]  Mohsen Kahani,et al.  Predicting future personal life events on twitter via recurrent neural networks , 2018, Journal of Intelligent Information Systems.

[8]  Santanu Kumar Rath,et al.  Classification of sentiment reviews using n-gram machine learning approach , 2016, Expert Syst. Appl..

[9]  Jun Zhao,et al.  Recurrent Convolutional Neural Networks for Text Classification , 2015, AAAI.

[10]  Jeffrey Dean,et al.  Distributed Representations of Words and Phrases and their Compositionality , 2013, NIPS.

[11]  Xiang Zhang,et al.  Character-level Convolutional Networks for Text Classification , 2015, NIPS.

[12]  Ludovico Boratto,et al.  Data-driven user behavioral modeling: from real-world behavior to knowledge, algorithms, and systems , 2020, Journal of Intelligent Information Systems.

[13]  Abhishek Verma,et al.  Deep CNN-LSTM with combined kernels from multiple branches for IMDb review sentiment analysis , 2017, 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON).

[14]  Christopher Potts,et al.  Learning Word Vectors for Sentiment Analysis , 2011, ACL.

[15]  Abhishek Verma,et al.  New Deep Neural Nets for Fine-Grained Diabetic Retinopathy Recognition on Hybrid Color Space , 2016, 2016 IEEE International Symposium on Multimedia (ISM).

[16]  Chu-Ren Huang,et al.  Sentiment Classification and Polarity Shifting , 2010, COLING.

[17]  Yoshua Bengio,et al.  Gradient-based learning applied to document recognition , 1998, Proc. IEEE.

[18]  Abhishek Verma,et al.  Novel CNN architecture with residual learning and deep supervision for large-scale scene image categorization , 2016, 2016 IEEE 7th Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON).

[19]  Ronen Feldman,et al.  Techniques and applications for sentiment analysis , 2013, CACM.

[20]  Lamia Rahman,et al.  A new LSTM model by introducing biological cell state , 2016, 2016 3rd International Conference on Electrical Engineering and Information Communication Technology (ICEEICT).

[21]  Yu Liu,et al.  Hybrid deep learning ensemble model for improved large-scale car recognition , 2017, 2017 IEEE SmartWorld, Ubiquitous Intelligence & Computing, Advanced & Trusted Computed, Scalable Computing & Communications, Cloud & Big Data Computing, Internet of People and Smart City Innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI).

[22]  Sergey Ioffe,et al.  Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning , 2016, AAAI.

[23]  Di Cai,et al.  Sentiment Analysis of Polish Texts , 2012 .

[24]  Long-Sheng Chen,et al.  Journal of Informetrics , 2022 .

[25]  Liang Gao,et al.  Deep learning-based personality recognition from text posts of online social networks , 2018, Applied Intelligence.

[26]  Andrew Zisserman,et al.  Very Deep Convolutional Networks for Large-Scale Image Recognition , 2014, ICLR.

[27]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[28]  Muhammad Usman,et al.  An effective model for aspect based opinion mining for social reviews , 2015, 2015 Tenth International Conference on Digital Information Management (ICDIM).

[29]  Hongseok Choi,et al.  Deep recurrent convolutional networks for inferring user interests from social media , 2018, Journal of Intelligent Information Systems.

[30]  Alexander M. Rush,et al.  Character-Aware Neural Language Models , 2015, AAAI.

[31]  Alessandro Moschitti,et al.  UNITN: Training Deep Convolutional Neural Network for Twitter Sentiment Classification , 2015, *SEMEVAL.