Mutation analysis with coverage discounting

Mutation testing is an established technique for evaluating validation thoroughness, but its adoption has been limited by the manual effort required to analyze the results. This paper describes the use of coverage discounting for mutation analysis, where undetected mutants are explained in terms of functional coverpoints, simplifying their analysis and saving effort. Two benchmarks are shown to compare this improved flow against regular mutation analysis. We also propose a confidence metric and simulation ordering algorithm optimized for coverage discounting, potentially reducing overall simulation time.

[1]  Kwang-Ting Cheng,et al.  Coverage discounting: A generalized approach for testbench qualification , 2011, 2011 IEEE International High Level Design Validation and Test Workshop.

[2]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[3]  Andreas Zeller,et al.  (Un-)Covering Equivalent Mutants , 2010, 2010 Third International Conference on Software Testing, Verification and Validation.

[4]  Kurt Keutzer,et al.  OCCOM: efficient computation of observability-based code coverage metrics for functional verification , 1998, Proceedings 1998 Design and Automation Conference. 35th DAC. (Cat. No.98CH36175).

[5]  Magdy S. Abadir,et al.  Coverage metrics for verification of concurrent SystemC designs using mutation testing , 2010, 2010 IEEE International High Level Design Validation and Test Workshop (HLDVT).

[6]  Franco Fummi,et al.  Functional qualification of TLM verification , 2009, 2009 Design, Automation & Test in Europe Conference & Exhibition.