A Gold Standard for Emotion Annotation in Stack Overflow

Software developers experience and share a wide range of emotions throughout a rich ecosystem of communication channels. A recent trend that has emerged in empirical software engineering studies is leveraging sentiment analysis of developers' communication traces. We release a dataset of 4,800 questions, answers, and comments from Stack Overflow, manually annotated for emotions. Our dataset contributes to the building of a shared corpus of annotated resources to support research on emotion awareness in software development.

[1]  Minhaz Fahim Zibran,et al.  Leveraging Automated Sentiment Analysis in Software Engineering , 2017, 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR).

[2]  Nicole Novielli,et al.  Sentiment Polarity Detection for Software Development , 2017, Empirical Software Engineering.

[3]  Bram Adams,et al.  Do developers feel emotions? an exploratory analysis of emotions in software artifacts , 2014, MSR 2014.

[4]  Bonita Sharif,et al.  Analyzing Developer Sentiment in Commit Logs , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[5]  Nicole Novielli,et al.  The challenges of sentiment detection in the social programmer ecosystem , 2015, SSE@SIGSOFT FSE.

[6]  P. Shaver,et al.  Emotion knowledge: further exploration of a prototype approach. , 1987, Journal of personality and social psychology.

[7]  Alexander Serebrenik,et al.  Security and emotion: sentiment analysis of security discussions on GitHub , 2014, MSR 2014.

[8]  Nicole Novielli,et al.  EmoTxt: A toolkit for emotion recognition from text , 2017, 2017 Seventh International Conference on Affective Computing and Intelligent Interaction Workshops and Demos (ACIIW).

[9]  Nicole Novielli,et al.  Anger and Its Direction in Collaborative Software Development , 2017, 2017 IEEE/ACM 39th International Conference on Software Engineering: New Ideas and Emerging Technologies Results Track (ICSE-NIER).

[10]  Michele Marchesi,et al.  Are Bullies More Productive? Empirical Study of Affectiveness vs. Issue Fixing Time , 2015, 2015 IEEE/ACM 12th Working Conference on Mining Software Repositories.

[11]  Mika Mäntylä,et al.  Mining Valence, Arousal, and Dominance - Possibilities for Detecting Burnout and Productivity? , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[12]  Nicole Novielli,et al.  How to ask for technical help? Evidence-based guidelines for writing questions on Stack Overflow , 2017, Inf. Softw. Technol..

[13]  Foutse Khomh,et al.  Automatic summarization of API reviews , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[14]  Serge Demeyer,et al.  An exploratory qualitative and quantitative analysis of emotions in issue report comments of open source systems , 2017, Empirical Software Engineering.

[15]  Nicole Novielli,et al.  Bootstrapping a Lexicon for Emotional Arousal in Software Engineering , 2017, 2017 IEEE/ACM 14th International Conference on Mining Software Repositories (MSR).

[16]  Mike Thelwall,et al.  Sentiment strength detection for the social web , 2012, J. Assoc. Inf. Sci. Technol..

[17]  Gabriele Bavota,et al.  Sentiment Analysis for Software Engineering: How Far Can We Go? , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[18]  Michele Marchesi,et al.  The Emotional Side of Software Developers in JIRA , 2016, 2016 IEEE/ACM 13th Working Conference on Mining Software Repositories (MSR).

[19]  Alexander Serebrenik,et al.  On negative results when using sentiment analysis tools for software engineering research , 2017, Empirical Software Engineering.

[20]  Anindya Iqbal,et al.  SentiCR: A customized sentiment analysis tool for code review interactions , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).