The Google FindBugs fixit

In May 2009, Google conducted a company wide FindBugs "fixit". Hundreds of engineers reviewed thousands of FindBugs warnings, and fixed or filed reports against many of them. In this paper, we discuss the lessons learned from this exercise, and analyze the resulting dataset, which contains data about how warnings in each bug pattern were classified. Significantly, we observed that even though most issues were flagged for fixing, few appeared to be causing any serious problems in production. This suggests that most interesting software quality problems were eventually found and fixed without FindBugs, but FindBugs could have found these problems early, when they are cheap to remediate. We compared this observation to bug trends observed in code snapshots from student projects. The full dataset from the Google fixit, with confidential details encrypted, will be published along with this paper.

[1]  Michael D. Ernst,et al.  Prioritizing Warning Categories by Analyzing Software History , 2007, Fourth International Workshop on Mining Software Repositories (MSR'07:ICSE Workshops 2007).

[2]  David Hovemeyer,et al.  Tracking defect warnings across versions , 2006, MSR '06.

[3]  Ciera Jaspan,et al.  Understanding the value of program analysis tools , 2007, OOPSLA '07.

[4]  PughWilliam,et al.  Experiences with marmoset , 2006 .

[5]  William Pugh,et al.  Using checklists to review static analysis warnings , 2009, DEFECTS '09.

[6]  Ivica Crnkovic,et al.  Proceedings of the the 6th joint meeting of the European software engineering conference and the ACM SIGSOFT symposium on The foundations of software engineering , 2007, FSE 2007.

[7]  David Hovemeyer,et al.  Experiences with marmoset: designing and using an advanced submission and testing system for programming courses , 2006, ITICSE '06.

[8]  William Pugh,et al.  A report on a survey and study of static analysis users , 2008, DEFECTS '08.

[9]  T. D. Wilson.  Review of: Boslaugh, Sarah and Watters, Paul Andrew Statistics in a nutshell. Sebastopol, CA: O'Reilly, 2008 , 2008, Inf. Res..

[10]  David Hovemeyer,et al.  Using Static Analysis to Find Bugs , 2008, IEEE Software.

[11]  Dawson R. Engler,et al.  A few billion lines of code later , 2010, Commun. ACM.

[12]  Paul A. Watters,et al.  Statistics in a nutshell , 2008 .

[13]  Raymond P. L. Buse,et al.  A metric for software readability , 2008, ISSTA '08.

[14]  Michael D. Ernst,et al.  Which warnings should I fix first? , 2007, ESEC-FSE '07.

[15]  Thomas Ball,et al.  Static analysis tools as early indicators of pre-release defect density , 2005, ICSE.