DEVA: sensing emotions in the valence arousal space in software engineering text

Existing tools for automated sentiment analysis in software engineering text suffer from either or both of two limitations. First, they are developed for non-technical domain and perform poorly when operated on software engineering text. Second, those tools attempt to detect valence only, and cannot capture arousal or individual emotional states such as excitement, stress, depression, and relaxation. In this paper, we present the first sentiment analysis tool, DEVA, which is especially designed for software engineering text and also capable of capturing the aforementioned emotional states through the detection of both arousal and valence. We also create a ground-truth dataset containing 1,795 JIRA issue comments. From a quantitative evaluation using this dataset, DEVA is found to have more than 82% precision and more than 78% recall.

[1]  Bernd Brügge,et al.  Towards emotional awareness in software development teams , 2013, ESEC/FSE 2013.

[2]  Christopher Potts,et al.  Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank , 2013, EMNLP.

[3]  George A. Miller,et al.  WordNet: A Lexical Database for English , 1995, HLT.

[4]  Kai Petersen,et al.  Time pressure: a controlled experiment of test case development and requirements review , 2014, ICSE.

[5]  Michal R. Wróbel,et al.  Emotions in the software development process , 2013, 2013 6th International Conference on Human System Interactions (HSI).

[6]  Robert M. Hierons,et al.  Do moods affect programmers’ debug performance? , 2011, Cognition, Technology & Work.

[7]  J. Russell,et al.  Evidence for a three-factor theory of emotions , 1977 .

[8]  Yang Li,et al.  Sentiment analysis of commit comments in GitHub: an empirical study , 2014, MSR 2014.

[9]  Foutse Khomh,et al.  Opiner: An opinion search and summarization engine for APIs , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[10]  Prasun Dewan,et al.  Towards Emotion-Based Collaborative Software Engineering , 2015, 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software Engineering.

[11]  Romain Robbes,et al.  Linking e-mails and source code artifacts , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[12]  Michele Marchesi,et al.  Are Bullies More Productive? Empirical Study of Affectiveness vs. Issue Fixing Time , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[13]  Bram Adams,et al.  The Impact of Human Discussions on Just-in-Time Quality Assurance: An Empirical Study on OpenStack and Eclipse , 2016, 2016 IEEE 23rd International Conference on Software Analysis, Evolution, and Reengineering (SANER).

[14]  Mika Mäntylä,et al.  Mining Valence, Arousal, and Dominance - Possibilities for Detecting Burnout and Productivity? , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[15]  Bram Adams,et al.  Do developers feel emotions? an exploratory analysis of emotions in software artifacts , 2014, MSR 2014.

[16]  Nicole Novielli,et al.  EmoTxt: A toolkit for emotion recognition from text , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW).

[17]  Ahmed E. Hassan,et al.  A Lightweight Approach to Uncover Technical Artifacts in Unstructured Data , 2011, 2011 IEEE 19th International Conference on Program Comprehension.

[18]  Osmar R. Zaïane,et al.  Current State of Text Sentiment Analysis from Opinion to Emotion Mining , 2017, ACM Comput. Surv..

[19]  Minhaz Fahim Zibran,et al.  A Comparison of Dictionary Building Methods for Sentiment Analysis in Software Engineering Text , 2017, 2017 ACM/IEEE International Symposium on Empirical Software Engineering and Measurement (ESEM).

[20]  Foutse Khomh,et al.  Automatic summarization of API reviews , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[21]  Serge Demeyer,et al.  An exploratory qualitative and quantitative analysis of emotions in issue report comments of open source systems , 2017, Empirical Software Engineering.

[22]  Amy Beth Warriner,et al.  Norms of valence, arousal, and dominance for 13,915 English lemmas , 2013, Behavior Research Methods.

[23]  Ahmed E. Hassan,et al.  A Lightweight Approach to Uncover Technical Information in Unstructured Data , 2011 .

[24]  Nicole Novielli,et al.  Bootstrapping a Lexicon for Emotional Arousal in Software Engineering , 2017, 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR).

[25]  Hsin-Hsi Chen,et al.  Building Emotion Lexicon from Weblog Corpora , 2007, ACL.

[26]  J. Fleiss Measuring nominal scale agreement among many raters. , 1971 .

[27]  Bram Adams,et al.  Monitoring sentiment in open source mailing lists: exploratory study on the apache ecosystem , 2014, CASCON.

[28]  Minhaz Fahim Zibran,et al.  Towards understanding and exploiting developers' emotional variations in software engineering , 2016, 2016 IEEE 14th International Conference on Software Engineering Research, Management and Applications (SERA).

[29]  Michele Marchesi,et al.  The Emotional Side of Software Developers in JIRA , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[30]  Hsin-Hsi Chen,et al.  Writer Meets Reader: Emotion Analysis of Social Media from Both the Writer's and Reader's Perspectives , 2009, 2009 IEEE/WIC/ACM International Joint Conference on Web Intelligence and Intelligent Agent Technology.

[31]  Karin Becker,et al.  Sentiment Analysis in Tickets for IT Support , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[32]  Anindya Iqbal,et al.  SentiCR: A customized sentiment analysis tool for code review interactions , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[33]  Mike Thelwall,et al.  TensiStrength: Stress and relaxation magnitude detection for social media texts , 2016, Inf. Process. Manag..

[34]  Walaa Medhat,et al.  Sentiment analysis algorithms and applications: A survey , 2014 .

[35]  Filippo Lanubile,et al.  Affective Trust as a Predictor of Successful Collaboration in Distributed Software Projects , 2016, 2016 IEEE/ACM 1st International Workshop on Emotional Awareness in Software Engineering (SEmotion).

[36]  Alexander Serebrenik,et al.  Choosing your weapons: On sentiment analysis tools for software engineering research , 2015, 2015 IEEE International Conference on Software Maintenance and Evolution (ICSME).

[37]  Elke A. Rundensteiner,et al.  EMOTEX: Detecting Emotions in Twitter Messages , 2014 .

[38]  Daniel McDuff,et al.  AffectAura: an intelligent system for emotional memory , 2012, CHI.

[39]  Thomas Fritz,et al.  Stuck and Frustrated or in Flow and Happy: Sensing Developers' Emotions and Progress , 2015, 2015 IEEE/ACM 37th IEEE International Conference on Software Engineering.

[40]  Marcelo Serrano Zanetti,et al.  The Role of Emotions in Contributors Activity: A Case Study on the GENTOO Community , 2013, 2013 International Conference on Cloud and Green Computing.

[41]  Nicole Novielli,et al.  Towards discovering the role of emotions in stack overflow , 2014, SSE@SIGSOFT FSE.

[42]  Eric Gilbert,et al.  VADER: A Parsimonious Rule-Based Model for Sentiment Analysis of Social Media Text , 2014, ICWSM.

[43]  Ricardo Colomo Palacios,et al.  Using the Affect Grid to Measure Emotions in Software Requirements Engineering , 2011, J. Univers. Comput. Sci..

[44]  Ning Nan,et al.  Impact of Budget and Schedule Pressure on Software Development Cycle Time and Effort , 2009, IEEE Transactions on Software Engineering.

[45]  Nicole Novielli,et al.  The challenges of sentiment detection in the social programmer ecosystem , 2015, SSE@SIGSOFT FSE.

[46]  Minhaz Fahim Zibran,et al.  Exploration and Exploitation of Developers' Sentimental Variations in Software Engineering , 2016, Int. J. Softw. Innov..

[47]  Alexander Serebrenik,et al.  Security and emotion: sentiment analysis of security discussions on GitHub , 2014, MSR 2014.

[48]  Minhaz Fahim Zibran,et al.  Leveraging Automated Sentiment Analysis in Software Engineering , 2017, 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR).

[49]  Nicole Novielli,et al.  Sentiment Polarity Detection for Software Development , 2017, Empirical Software Engineering.

[50]  Chung-Hsien Wu,et al.  Emotion recognition from textual input using an emotional semantic network , 2002, INTERSPEECH.

[51]  Abram Hindle,et al.  Characterizing Energy-Aware Software Projects: Are They Different? , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[52]  Mike Thelwall,et al.  Sentiment strength detection for the social web , 2012, J. Assoc. Inf. Sci. Technol..

[53]  Munmun De Choudhury,et al.  Understanding affect in the workplace via social media , 2013, CSCW.