Efficient Allocation of Verification Resources using Revision History Information

Verifying large industrial designs is getting harder each day. The current verification methodologies are not able to guarantee bug free designs. Some recurrent questions during a design verification are: Which modules are most likely to contain undetected bugs? In which modules the verification team should concentrate their effort? This information is very useful, because it is better to start verifying the most bug-prone modules. In this work we present a novel approach to answer these questions. In order to identify these bug-prone modules, the revision history of the design is used. Using information of an academic experiment, we demonstrate that there is a close relationship between bugs/changes history and future bugs. Our results show that allocating modules for verification based on bugs/changes leaded to the coverage of 91.67% of future bugs, while random based strategy covered only 37.5% of the future bugs.

[1]  Elaine J. Weyuker,et al.  Predicting the location and number of faults in large software systems , 2005, IEEE Transactions on Software Engineering.

[2]  Francine Bacchini,et al.  Verification: what works and what doesn't , 2004, DAC '04.

[3]  Miroslav N. Velev,et al.  Collection of high-level microprocessor bugs from formal verification of pipelined and superscalar designs , 2003, International Test Conference, 2003. Proceedings. ITC 2003..

[4]  Jerry L. Trahan,et al.  Neural-network techniques for software-quality evaluation , 1998, Annual Reliability and Maintainability Symposium. 1998 Proceedings. International Symposium on Product Quality and Integrity.

[5]  Andreas Zeller,et al.  Where do bugs come from? , 2006, SOEN.

[6]  Elaine J. Weyuker,et al.  Where the bugs are , 2004, ISSTA '04.

[7]  Richard C. Holt,et al.  The top ten list: dynamic fault prediction , 2005, 21st IEEE International Conference on Software Maintenance (ICSM'05).

[8]  Avinoam Kolodny.  Networks on chips: keeping up with rent's rule and moore's law , 2007, SLIP '07.

[9]  Michael Pilato.  Version Control with Subversion , 2004 .

[10]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[11]  Harvey P. Siy,et al.  Predicting Fault Incidence Using Software Change History , 2000, IEEE Trans. Software Eng..

[12]  Taghi M. Khoshgoftaar,et al.  Predicting the order of fault-prone modules in legacy software , 1998, Proceedings Ninth International Symposium on Software Reliability Engineering (Cat. No.98TB100257).

[13]  Taghi M. Khoshgoftaar,et al.  Ordering Fault-Prone Software Modules , 2003, Software Quality Journal.

[14]  N. Nagappan,et al.  Use of relative code churn measures to predict system defect density , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[15]  Paolo Toth,et al.  Knapsack Problems: Algorithms and Computer Implementations , 1990 .

[16]  John P. Hayes,et al.  Collection and Analysis of Microprocessor Design Errors , 2000, IEEE Des. Test Comput..

[17]  Andreas Zeller.  Where Do Bugs Come From? , 2007, Electron. Notes Theor. Comput. Sci..