Diagnostic Test Generation for Statistical Bug Localization Using Evolutionary Computation

Verification is increasingly becoming a bottleneck in the process of designing electronic circuits. While there exists several verification tools that assist in detecting occurrences of design errors, or bugs, there is a lack of solutions for accurately pin-pointing the root causes of these errors. Statistical bug localization has proven to be an approach that scales up to large designs and is widely utilized both in debugging hardware and software. However, the accuracy of localization is highly dependent on the quality of the stimuli. In this paper we formulate diagnostic test set generation as a task for an evolutionary algorithm, and propose dedicated fitness functions that closely correlate with the bug localization capabilities. We perform experiments on the register-transfer level design of the Plasma microprocessor coupling an evolutionary test-pattern generator and a simulator for fitness evaluation. As a result, the diagnostic resolution of the tests is significantly improved.

[1]  G. Squillero,et al.  Automatic test generation for verifying microprocessors , 2005, IEEE Potentials.

[2]  Giovanni Squillero,et al.  Artificial evolution in computer aided design: from the optimization of parameters to the creation of assembly programs , 2011, Computing.

[3]  H. Cleve,et al.  Locating causes of program failures , 2005, Proceedings. 27th International Conference on Software Engineering, 2005. ICSE 2005..

[4]  Igor L. Markov,et al.  Automatic error diagnosis and correction for RTL designs , 2007, 2007 IEEE International High Level Design Validation and Test Workshop.

[5]  Giovanni Squillero,et al.  MicroGP—An Evolutionary Assembly Program Generator , 2005, Genetic Programming and Evolvable Machines.

[6]  Jaan Raik,et al.  A scalable model based RTL framework zamiaCAD for static analysis , 2012, 2012 IEEE/IFIP 20th International Conference on VLSI and System-on-Chip (VLSI-SoC).

[7]  Raimund Ubar,et al.  Assessment of diagnostic test for automated bug localization , 2013, 2013 14th Latin American Test Workshop - LATW.

[8]  Byoungju Choi,et al.  A family of code coverage-based heuristics for effective fault localization , 2010, J. Syst. Softw..

[9]  Kwang-Ting Cheng,et al.  Coverage discounting: A generalized approach for testbench qualification , 2011, 2011 IEEE International High Level Design Validation and Test Workshop.

[10]  A.J.C. van Gemund,et al.  On the Accuracy of Spectrum-based Fault Localization , 2007, Testing: Academic and Industrial Conference Practice and Research Techniques - MUTATION (TAICPART-MUTATION 2007).

[11]  Jaan Raik,et al.  Localization of Bugs in Processor Designs Using zamiaCAD Framework , 2012, 2012 13th International Workshop on Microprocessor Test and Verification (MTV).

[12]  Michael I. Jordan,et al.  Scalable statistical bug isolation , 2005, PLDI '05.

[13]  Andreas Veneris,et al.  Design diagnosis using Boolean satisfiability , 2004 .

[14]  Kwang-Ting Cheng,et al.  Mutation-based diagnostic test generation for hardware design error diagnosis , 2010, 2010 IEEE International Test Conference.

[15]  Giovanni Squillero,et al.  RT-Level ITC'99 Benchmarks and First ATPG Results , 2000, IEEE Des. Test Comput..

[16]  Jaan Raik,et al.  Comparison of Model-Based Error Localization algorithms for C designs , 2013, East-West Design & Test Symposium (EWDTS 2013).

[17]  Franz Wotawa,et al.  Automated source-level error localization in hardware designs , 2006, IEEE Design & Test of Computers.

[18]  Giovanni Squillero,et al.  An Effective Technique for the Automatic Generation of Diagnosis-Oriented Programs for Processor Cores , 2008, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[19]  Mary Jean Harrold,et al.  Empirical evaluation of the tarantula automatic fault-localization technique , 2005, ASE.

[20]  Rolf Drechsler.  Evolutionary algorithms for VLSI CAD , 1998 .

[21]  Chao Liu,et al.  Statistical Debugging: A Hypothesis Testing-Based Approach , 2006, IEEE Transactions on Software Engineering.

[22]  Giovanni Squillero,et al.  Evolutionary Optimization: the µGP toolkit , 2011 .