Interpretable Movie Review Analysis Using Machine Learning and Transformer Models Leveraging XAI

Text classification has been a common topic of interest for many years. A lot of advanced models has been developed so far in this area. But it is very difficult to understand how the models behave while predicting the class of the text. In our work, we utilized some models to classify the sentiment of movie reviews from text data and observed how the models behaved using Explainable Artificial Intelligence (XAI). At first dataset was collected and pre-processed. Then the processed dataset was separated into different train and test sets. The train set was used to classify using different different machine learning and neural network based models. The test set was used after training to evaluate the trained classifiers. Finally, the performance of the classifiers were compared and evaluated. After different variations of pre-processing and training steps, the best accuracy score of 91% was obtained using Roberta LSTM model. In the trained models, we sent texts that are correctly classified by RoBERTa models but misclassified by other models. Finally we figured out the reasons of misclassification with the help of LIME Algorithm.

[1]  Riyo Hayat Khan,et al.  A Comparative Study on COVID-19 Fake News Detection Using Different Transformer Based Models , 2022, 2022 IEEE Symposium on Industrial Electronics & Applications (ISIEA).

[2]  Mark A. Przybocki,et al.  Four Principles of Explainable Artificial Intelligence , 2020 .

[3]  S. Anbukkarasi,et al.  Analyzing Sentiment in Tamil Tweets using Deep Neural Network , 2020, 2020 Fourth International Conference on Computing Methodologies and Communication (ICCMC).

[4]  Md. Sharif Hossen,et al.  Sentiment Analysis on Movie Review Data Using Machine Learning Approach , 2019, 2019 International Conference on Bangla Speech and Language Processing (ICBSLP).

[5]  William Yang Wang,et al.  Towards Explainable NLP: A Generative Explanation Framework for Text Classification , 2018, ACL.

[6]  Paolo Gastaldo,et al.  Bayesian network based extreme learning machine for subjectivity detection , 2017, J. Frankl. Inst..

[7]  Anuja Arora,et al.  User tweets based genre prediction and movie recommendation using LSI and SVD , 2016, 2016 Ninth International Conference on Contemporary Computing (IC3).

[8]  Carlos Guestrin,et al.  "Why Should I Trust You?": Explaining the Predictions of Any Classifier , 2016, ArXiv.

[9]  Jimmy Ba,et al.  Adam: A Method for Stochastic Optimization , 2014, ICLR.

[10]  Jule Selbo Film Genre for the Screenwriter , 2014 .

[11]  Mehran Memari,et al.  Movie Genre Preference and Culture , 2014 .

[12]  Xiaoyan Zhu,et al.  Movie review mining and summarization , 2006, CIKM '06.

[13]  Lina Zhou,et al.  Movie Review Mining: a Comparison between Supervised and Unsupervised Classification Approaches , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[14]  Bing Liu,et al.  Mining and summarizing customer reviews , 2004, KDD.

[15]  Sepp Hochreiter,et al.  The Vanishing Gradient Problem During Learning Recurrent Neural Nets and Problem Solutions , 1998, Int. J. Uncertain. Fuzziness Knowl. Based Syst..

[16]  S. Hochreiter,et al.  Long Short-Term Memory , 1997, Neural Computation.

[17]  Yoshua Bengio,et al.  Learning long-term dependencies with gradient descent is difficult , 1994, IEEE Trans. Neural Networks.

[18]  Philip H. Swain,et al.  Purdue e-Pubs , 2022 .

[19]  K. Anbananthen,et al.  RoBERTa-LSTM: A Hybrid Model for Sentiment Analysis With Transformer and Recurrent Neural Network , 2022, IEEE Access.

[20]  Harris Papadopoulos,et al.  BERT-based conformal predictor for sentiment analysis , 2020, COPA.

[21]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[22]  Ankit Sharma,et al.  Sentiment Analysis of Movie Review Using Supervised Machine Learning Techniques , 2018 .

[23]  Mihaela Sorostinean,et al.  Sentiment Analysis on Movie Reviews , 2017 .

[24]  Kristin L. Sainani,et al.  Logistic Regression , 2014, PM & R : the journal of injury, function, and rehabilitation.

[25]  Kuat Yessenov,et al.  Sentiment Analysis of Movie Review Comments , 2009 .

[26]  Gerben Bakker,et al.  The economic history of the international film industry , 2005 .

[27]  Wright-Patterson Afb,et al.  Feature Selection Using a Multilayer Perceptron , 1990 .