The Impact of Automatic Crash Reports on Bug Triaging and Development in Mozilla

Free/Open Source Software projects often rely on users submitting bug reports. However, reports submitted by novice users may lack information critical to developers, and the process may be intimidating and difficult. To gather more and better data, projects deploy automatic crash reporting tools, which capture stack traces and memory dumps when a crash occurs. These systems potentially generate large volumes of data, which may overwhelm developers, and their presence may discourage users from submitting traditional bug reports. In this paper, we examine Mozilla's automatic crash reporting system and how it affects their bug triaging process. We find that fewer than 0.00009% of crash reports end up in a bug report, but as many as 2.33% of bug reports have data from crash reports added. Feedback from developers shows that despite some problems, these systems are valuable. We conclude with a discussion of the pros and cons of automatic crash reporting systems.

[1]  Sooyong Park,et al.  Which Crashes Should I Fix First?: Predicting Top Crashes at an Early Stage to Prioritize Debugging Efforts , 2011, IEEE Transactions on Software Engineering.

[2]  Eric S. Raymond,et al.  The cathedral and the bazaar , 1998, First Monday.

[3]  Thomas Zimmermann,et al.  Duplicate bug reports considered harmful … really? , 2008, 2008 IEEE International Conference on Software Maintenance.

[4]  Gail C. Murphy,et al.  Who should fix this bug? , 2006, ICSE.

[5]  Tao Xie,et al.  An approach to detecting duplicate bug reports using natural language and execution information , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[6]  Galen C. Hunt,et al.  Debugging in the (very) large: ten years of implementation and experience , 2009, SOSP '09.

[7]  Foutse Khomh,et al.  An Entropy Evaluation Approach for Triaging Field Crashes: A Case Study of Mozilla Firefox , 2011, 2011 18th Working Conference on Reverse Engineering.

[8]  Daniel Lucrédio,et al.  An Initial Study on the Bug Report Duplication Problem , 2010, 2010 14th European Conference on Software Maintenance and Reengineering.

[9]  Amy J. Ko,et al.  How power users help and hinder open bug reporting , 2010, CHI.

[10]  Gail C. Murphy,et al.  Coping with an open bug repository , 2005, eclipse '05.

[11]  Brad A. Myers,et al.  A Linguistic Analysis of How People Describe Software Problems , 2006, Visual Languages and Human-Centric Computing (VL/HCC'06).

[12]  Thomas Zimmermann,et al.  Information needs in bug reports: improving cooperation between developers and users , 2010, CSCW '10.

[13]  Franz Wotawa,et al.  Automatic Software Bug Triage System (BTS) Based on Latent Semantic Indexing and Support Vector Machine , 2009, 2009 Fourth International Conference on Software Engineering Advances.

[14]  Nicholas Jalbert,et al.  Automated duplicate detection for bug tracking systems , 2008, 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN).

[15]  Thomas Zimmermann,et al.  Improving bug triage with bug tossing graphs , 2009, ESEC/FSE '09.

[16]  Carlos Jensen,et al.  Coping with duplicate bug reports in free/open source software projects , 2011, 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC).

[17]  Ahmed Tamrawi,et al.  Fuzzy set-based automatic bug triaging: NIER track , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[18]  John Anvik,et al.  Automating bug report assignment , 2006, ICSE.

[19]  Oscar Nierstrasz,et al.  Assigning bug reports using a vocabulary-based expertise model of developers , 2009, 2009 6th IEEE International Working Conference on Mining Software Repositories.

[20]  Westley Weimer,et al.  Modeling bug report quality , 2007, ASE '07.

[21]  Thomas Zimmermann,et al.  Automatic Identification of Bug-Introducing Changes , 2006, 21st IEEE/ACM International Conference on Automated Software Engineering (ASE'06).

[22]  Foutse Khomh,et al.  Classifying field crash reports for fixing bugs: A case study of Mozilla Firefox , 2011, 2011 27th IEEE International Conference on Software Maintenance (ICSM).

[23]  Daniel Lucrédio,et al.  One Step More to Understand the Bug Report Duplication Problem , 2010, 2010 Brazilian Symposium on Software Engineering.

[24]  Nachiappan Nagappan,et al.  Crash graphs: An aggregated view of multiple crashes to improve crash triage , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[25]  Thomas Zimmermann,et al.  What Makes a Good Bug Report? , 2008, IEEE Transactions on Software Engineering.

[26]  Martin Pinzger,et al.  "A Bug's Life" Visualizing a Bug Database , 2007, 2007 4th IEEE International Workshop on Visualizing Software for Understanding and Analysis.