Assessing the precision of FindBugs by mining Java projects developed at a university

Software repositories are analyzed to extract useful information on software characteristics. One of them is external quality. A technique used to increase software quality is automatic static analysis, by means of bug finding tools. These tools promise to speed up the verification of source code; anyway, there are still many problems, especially the high number of false positives, that hinder their large adoption in software development industry. We studied the capability of a popular bug-finding tool, FindBugs, for defect prediction purposes, analyzing the issues revealed on a repository of university Java projects. Particularly, we focused on the percentage of them that indicates actual defects with respect to their category and priority, and we ranked them. We found that a very limited set of issues have high precision and therefore have a positive impact on code external quality.

[1]  Jan Jürjens,et al.  Comparing Bug Finding Tools with Reviews and Tests , 2005, TestCom.

[2]  E. Liski An Introduction to Categorical Data Analysis, 2nd Edition by Alan Agresti , 2007 .

[3]  Stefan Wagner,et al.  An Evaluation of Two Bug Pattern Tools for Java , 2008, 2008 1st International Conference on Software Testing, Verification, and Validation.

[4]  Leon Moonen,et al.  Evaluating the relation between coding standard violations and faultswithin and across software versions , 2009, 2009 6th IEEE International Working Conference on Mining Software Repositories.

[5]  Michael D. Ernst,et al.  Which warnings should I fix first? , 2007, ESEC-FSE '07.

[6]  David Hovemeyer,et al.  Evaluating and tuning a static analysis to find null pointer bugs , 2005, PASTE '05.

[7]  Leon Moonen,et al.  Assessing the value of coding standards: An empirical study , 2008, 2008 IEEE International Conference on Software Maintenance.

[8]  Barry W. Boehm,et al.  Software Defect Reduction Top 10 List , 2001, Computer.

[9]  Yuanyuan Zhou,et al.  Have things changed now?: an empirical study of bug characteristics in modern open source software , 2006, ASID '06.

[10]  Edward N. Adams,et al.  Optimizing Preventive Service of Software Products , 1984, IBM J. Res. Dev..

[11]  J. David Morgenthaler,et al.  Evaluating static analysis defect warnings on production software , 2007, PASTE '07.

[12]  Barry W. Boehm Software process management: lessons learned from history , 1987, ICSE '87.

[13]  Barry Boehm,et al.  Top 10 list [software development] , 2001 .

[14]  David Hovemeyer,et al.  Improving your software using static analysis to find bugs , 2006, OOPSLA '06.

[15]  David Hovemeyer,et al.  Using Static Analysis to Find Bugs , 2008, IEEE Software.

[16]  Marco Torchiano,et al.  A Fully Automatic Approach to the Assessment of Programming Assignments , 2009 .

[17]  Laurie A. Williams,et al.  On the value of static analysis for fault detection in software , 2006, IEEE Transactions on Software Engineering.