BERT-Based Sentiment Analysis: A Software Engineering Perspective

Sentiment analysis can provide a suitable lead for the tools used in software engineering along with the API recommendation systems and relevant libraries to be used. In this context, the existing tools like SentiCR, SentiStrength-SE, etc. exhibited low f1-scores that completely defeats the purpose of deployment of such strategies, thereby there is enough scope for performance improvement. Recent advancements show that transformer based pre-trained models (e.g., BERT, RoBERTa, ALBERT, etc.) have displayed better results in the text classification task. Following this context, the present research explores different BERT-based models to analyze the sentences in GitHub comments, Jira comments, and Stack Overflow posts. The paper presents three different strategies to analyse BERT based model for sentiment analysis, where in the first strategy the BERT based pre-trained models are finetuned; in the second strategy an ensemble model is developed from BERT variants, and in the third strategy a compressed model (Distil BERT) is used. The experimental results show that the BERT based ensemble approach and the compressed BERT model attain improvements by 6-12% over prevailing tools for the F1 measure on all three datasets.

[1]  Rouzbeh A. Shirvani,et al.  Natural Language Processing Advancements By Deep Learning: A Survey , 2020, ArXiv.

[2]  Jeffrey Pennington,et al.  GloVe: Global Vectors for Word Representation , 2014, EMNLP.

[3]  Sonali Agarwal,et al.  Web based disease prediction and recommender system , 2021, ArXiv.

[4]  Kevin Gimpel,et al.  ALBERT: A Lite BERT for Self-supervised Learning of Language Representations , 2019, ICLR.

[5]  Steven Bird,et al.  NLTK: The Natural Language Toolkit , 2002, ACL.

[6]  Daniel Jurafsky,et al.  Noising and Denoising Natural Language: Diverse Backtranslation for Grammar Correction , 2018, NAACL.

[7]  Yingying Zhang,et al.  Extracting problematic API features from forum discussions , 2013, 2013 21st International Conference on Program Comprehension (ICPC).

[8]  Omer Levy,et al.  RoBERTa: A Robustly Optimized BERT Pretraining Approach , 2019, ArXiv.

[9]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[10]  Gabriele Bavota,et al.  Two Datasets for Sentiment Analysis in Software Engineering , 2018, 2018 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[11]  Nicole Novielli,et al.  Sentiment Polarity Detection for Software Development , 2017, Empirical Software Engineering.

[12]  Lukasz Kaiser,et al.  Attention is All you Need , 2017, NIPS.

[13]  Sonali Agarwal,et al.  Multi-modality encoded fusion with 3D inception U-net and decoder model for brain tumor segmentation , 2020, Multimedia Tools and Applications.

[14]  Emerson R. Murphy-Hill,et al.  Sentiment and Politeness Analysis Tools on Developer Discussions Are Unreliable, but So Are People , 2018, 2018 IEEE/ACM 3rd International Workshop on Emotion Awareness in Software Engineering (SEmotion).

[15]  Mihai Surdeanu,et al.  The Stanford CoreNLP Natural Language Processing Toolkit , 2014, ACL.

[16]  Ming-Wei Chang,et al.  BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding , 2019, NAACL.

[17]  Tomas Mikolov,et al.  Enriching Word Vectors with Subword Information , 2016, TACL.

[18]  Minhaz Fahim Zibran,et al.  Leveraging Automated Sentiment Analysis in Software Engineering , 2017, 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR).

[19]  Narinder Singh Punn,et al.  CHS-Net: A Deep Learning Approach for Hierarchical Segmentation of COVID-19 via CT Images , 2020, Neural Processing Letters.

[20]  Nicole Novielli,et al.  A Benchmark Study on Sentiment Analysis for Software Engineering Research , 2018, 2018 IEEE/ACM 15th International Conference on Mining Software Repositories (MSR).

[21]  Omer Levy,et al.  word2vec Explained: deriving Mikolov et al.'s negative-sampling word-embedding method , 2014, ArXiv.

[22]  Xuanzhe Liu,et al.  SEntiMoji: an emoji-powered learning approach for sentiment analysis in software engineering , 2019, ESEC/SIGSOFT FSE.

[23]  Matteo Pagliardini,et al.  Unsupervised Learning of Sentence Embeddings Using Compositional n-Gram Features , 2017, NAACL.

[24]  Gabriele Bavota,et al.  Sentiment Analysis for Software Engineering: How Far Can We Go? , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[25]  Anindya Iqbal,et al.  SentiCR: A customized sentiment analysis tool for code review interactions , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[26]  Flavius Frasincar,et al.  Data augmentation in a hybrid approach for aspect-based sentiment analysis , 2021, SAC.

[27]  Flavius Frasincar,et al.  Pattern Learning for Detecting Defect Reports and Improvement Requests in App Reviews , 2020, NLDB.

[28]  ThelwallMike,et al.  Sentiment strength detection in short informal text , 2010 .

[29]  Chanchal Kumar Roy,et al.  Recommending insightful comments for source code using crowdsourced knowledge , 2015, 2015 IEEE 15th International Working Conference on Source Code Analysis and Manipulation (SCAM).

[30]  Minhaz Fahim Zibran,et al.  DEVA: sensing emotions in the valence arousal space in software engineering text , 2018, SAC.

[31]  Nicole Novielli,et al.  EmoTxt: A toolkit for emotion recognition from text , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW).

[32]  Minhaz Fahim Zibran,et al.  SentiStrength-SE: Exploiting domain specificity for improved sentiment analysis in software engineering text , 2018, J. Syst. Softw..