Frequently Asked Questions in Bug Reports

Bug tracking systems play a central role in software development since they allow users and developers to submit and discuss bugs and new features. To better understand information and communication needs in bug tracking, we analysed what questions are asked in bug reports. We sampled 600 bug reports from the MOZILLA and ECLIPSE projects and located 947 questions in the reports. Next, we used an open card sort and identified eight categories of questions, which can further be broken down into forty groups. We show the value of this catalogue of frequently asked questions with a large quantitative and qualitative study on when questions are asked and how they are answered. A consequence of our results is that constant user involvement is crucial for successful bug reports and that better tools are needed to support this.

[1]  Brad A. Myers,et al.  An Exploratory Study of How Developers Seek, Relate, and Collect Relevant Information during Software Maintenance Tasks , 2006, IEEE Transactions on Software Engineering.

[2]  Gail C. Murphy,et al.  Questions programmers ask during software evolution tasks , 2006, SIGSOFT '06/FSE-14.

[3]  Thomas Zimmermann,et al.  Towards the next generation of bug tracking systems , 2008, 2008 IEEE Symposium on Visual Languages and Human-Centric Computing.

[4]  Øyvind Langsrud,et al.  ANOVA for unbalanced data: Use Type II instead of Type III sums of squares , 2003, Stat. Comput..

[5]  Brad A. Myers,et al.  A Linguistic Analysis of How People Describe Software Problems , 2006, Visual Languages and Human-Centric Computing (VL/HCC'06).

[6]  Forrest Shull,et al.  Building Knowledge through Families of Experiments , 1999, IEEE Trans. Software Eng..

[7]  Tao Xie,et al.  An approach to detecting duplicate bug reports using natural language and execution information , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[8]  Nicholas Jalbert,et al.  Automated duplicate detection for bug tracking systems , 2008, 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN).

[9]  Thomas Zimmermann,et al.  Quality of bug reports in Eclipse , 2007, eclipse '07.

[10]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[11]  Gina Venolia,et al.  The secret life of bugs: Going past the errors and omissions in software repositories , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[12]  Per Runeson,et al.  Detection of Duplicate Defect Reports Using Natural Language Processing , 2007, 29th International Conference on Software Engineering (ICSE'07).

[13]  Christine Halverson,et al.  Designing task visualizations to support the coordination of work in software development , 2006, CSCW '06.

[14]  W. Lewis Johnson,et al.  Interactive Explanation of Software Systems , 2004, Automated Software Engineering.

[15]  Thomas Zimmermann,et al.  Duplicate bug reports considered harmful … really? , 2008, 2008 IEEE International Conference on Software Maintenance.

[16]  Robert DeLine,et al.  Information Needs in Collocated Software Development Teams , 2007, 29th International Conference on Software Engineering (ICSE'07).

[17]  Stacy Marsella,et al.  Task oriented software understanding , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[18]  Gail C. Murphy,et al.  Who should fix this bug? , 2006, ICSE.

[19]  James D. Herbsleb,et al.  Preserving knowledge in design projects: what designers need to know , 1993, INTERCHI.