Pre-Silicon Bug Forecast

The ever-intensifying time-to-market pressure imposes great challenges on the pre-silicon design phase of hardware. Before the tape-out, a pre-silicon design has to be thoroughly inspected by time-consuming functional verification and code review to exclude bugs. For functional verification and code review, a critical issue determining their efficiency is the allocation of resources (e.g., computational resources and manpower) to different modules of a design, which is conventionally guided by designers' experiences. Such practices, though simple and straightforward, may take high risks of wasting resources on bug-free modules or missing bugs in buggy modules, and thus could affect the success and timeline of the tape-out. In this paper, we propose a novel framework called pre-silicon bug forecast to predict the bug information of hardware designs. In this framework, bug models are built via machine learning techniques to characterize the relationship between design characteristics and the bug information, which can be leveraged to predict how bugs distribute in different modules of the current design. Such predicted bug information is adequate to regulate the resources among different modules to achieve efficient functional verification and code review. To evaluate the effectiveness of the proposed pre-silicon bug forecast framework, we conducted detailed experiments on several open-source hardware projects. Moreover, we also investigate the impacts of different learning techniques and different sets of characteristic on the performance of bug models. Experimental results show that with appropriate learning techniques and characteristics, about 90% modules could be correctly predicted as buggy or clean and the number of bugs of each module could also be accurately predicted.

[1]  Frank Elberzhager,et al.  Transparent combination of expert and measurement data for defect prediction: an industrial case study , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[2]  C. Bazeghi,et al.  /spl mu/Complexity: estimating processor design effort , 2005, 38th Annual IEEE/ACM International Symposium on Microarchitecture (MICRO'05).

[3]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[4]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[5]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[6]  Witold Pedrycz,et al.  A comparative analysis of the efficiency of change metrics and static code attributes for defect prediction , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[7]  Hinrich Schütze,et al.  Book Reviews: Foundations of Statistical Natural Language Processing , 1999, CL.

[8]  Jose Renau,et al.  Understanding bug fix patterns in verilog , 2008, MSR '08.

[9]  Onur Mutlu,et al.  Online design bug detection: RTL analysis, flexible mechanisms, and evaluation , 2008, 2008 41st IEEE/ACM International Symposium on Microarchitecture.

[10]  Andreas Zeller,et al.  When do changes induce fixes? , 2005, ACM SIGSOFT Softw. Eng. Notes.

[11]  Alon Gluska.  Coverage-oriented verification of Banias , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[12]  Bob Bentley,et al.  Validating the Intel(R) Pentium(R) 4 microprocessor , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[13]  Fabrizio Sebastiani,et al.  Machine learning in automated text categorization , 2001, CSUR.

[14]  อนิรุธ สืบสิงห์,et al.  Data Mining Practical Machine Learning Tools and Techniques , 2014 .

[15]  Ahmed E. Hassan,et al.  Predicting faults using the complexity of code changes , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[16]  John D. Provence,et al.  Measures of Syntactic Complexity for Modeling Behavioral VHDL , 1995, 32nd Design Automation Conference.

[17]  Michele Banko,et al.  Scaling to Very Very Large Corpora for Natural Language Disambiguation , 2001, ACL.

[18]  J. R. Quinlan.  Learning With Continuous Classes , 1992 .

[19]  Tianshi Chen,et al.  Estimating design quality of digital systems via machine learning , 2010, 2010 17th IEEE International Conference on Electronics, Circuits and Systems.

[20]  Todd M. Austin,et al.  Microprocessor Verification via Feedback-Adjusted Markov Models , 2007, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[21]  San Cristóbal Mateo,et al.  The Lack of A Priori Distinctions Between Learning Algorithms , 1996 .

[22]  Ieee Circuits,et al.  IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems information for authors , 2018, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[23]  Ariel Fuxman,et al.  Improving classification accuracy using automatically extracted training data , 2009, KDD.

[24]  Jacek Czerwonka,et al.  CRANE: Failure Prediction, Change Analysis and Test Prioritization in Practice -- Experiences from Windows , 2011, 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation.

[25]  N. Nagappan,et al.  Use of relative code churn measures to predict system defect density , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[26]  Vladimir N. Vapnik,et al.  The Nature of Statistical Learning Theory , 2000, Statistics for Engineering and Information Science.

[27]  Tianshi Chen,et al.  Empirical design bugs prediction for verification , 2011, 2011 Design, Automation & Test in Europe.

[28]  Alon Gluska.  Practical methods in coverage-oriented verification of the Merom microprocessor , 2006, 2006 43rd ACM/IEEE Design Automation Conference.

[29]  Osamu Mizuno,et al.  Bug prediction based on fine-grained module histories , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[30]  Rajender Singh Chhillar,et al.  Empirical analysis of object-oriented design metrics for predicting high, medium and low severity faults using mallows Cp , 2011, SOEN.

[31]  Taghi M. Khoshgoftaar,et al.  Experimental perspectives on learning from imbalanced data , 2007, ICML '07.

[32]  Premkumar T. Devanbu,et al.  Ownership, experience and defects: a fine-grained study of authorship , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[33]  Jose Renau,et al.  uComplexity: Estimating Processor Design Effort , 2005, MICRO.

[34]  Francesco Pessolano,et al.  An objective measure of digital system design quality , 2000, Proceedings IEEE 2000 First International Symposium on Quality Electronic Design (Cat. No. PR00525).

[35]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[36]  Avi Ziv,et al.  Coverage directed test generation for functional verification using Bayesian networks , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[37]  Andreas Zeller,et al.  Predicting faults from cached history , 2008, ISEC '08.

[38]  Harald C. Gall,et al.  Don't touch my code!: examining the effects of ownership on software quality , 2011, ESEC/FSE '11.

[39]  Mark A. Hall,et al.  Correlation-based Feature Selection for Discrete and Numeric Class Machine Learning , 1999, ICML.

[40]  Matthew Taylor,et al.  An effective and flexible approach to functional verification of processor families , 2002, Seventh IEEE International High-Level Design Validation and Test Workshop, 2002..

[41]  Yuming Zhou,et al.  Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults , 2006, IEEE Transactions on Software Engineering.