Improving validation coverage metrics to account for limited observability

In both pre-silicon and post-silicon validation, the detection of design errors requires both stimulus capable of activating the errors and checkers capable of detecting the behavior as erroneous. Most functional and code coverage metrics evaluate only the activation component of the testbench and ignore propagation and detection. In this paper, we summarize our recent work in developing improved metrics that account for propagation and/or detection of design errors. These works include tools for observability-enhanced code coverage and mutation analysis of high-level designs as well as an analytical method, Coverage Discounting, which adds checker sensitivity to arbitrary functional coverage metrics.

[1]  Tao Lv,et al.  An Observability Branch Coverage Metric Based on Dynamic Factored Use-Define Chains , 2006, 2006 15th Asian Test Symposium.

[2]  Kwang-Ting Cheng,et al.  SCEMIT: A SystemC error and mutation injection tool , 2010, Design Automation Conference.

[3]  Kwang-Ting Cheng,et al.  An instrumented observability coverage method for system validation , 2009, 2009 IEEE International High Level Design Validation and Test Workshop.

[4]  Richard J. Lipton,et al.  Theoretical and empirical studies on using program mutation to test the functional correctness of programs , 1980, POPL '80.

[5]  Chien-Nan Jimmy Liu,et al.  An observability measure to enhance statement coverage metric for proper evaluation of verification completeness , 2005, Proceedings of the ASP-DAC 2005. Asia and South Pacific Design Automation Conference, 2005..

[6]  Kurt Keutzer,et al.  OCCOM: efficient computation of observability-based code coverage metrics for functional verification , 1998, Proceedings 1998 Design and Automation Conference. 35th DAC. (Cat. No.98CH36175).

[7]  Kwang-Ting Cheng,et al.  Coverage discounting: A generalized approach for testbench qualification , 2011, 2011 IEEE International High Level Design Validation and Test Workshop.

[8]  Avi Ziv,et al.  User defined coverage—a tool supported methodology for design verification , 1998, DAC.

[9]  Ian G. Harris,et al.  A data flow fault coverage metric for validation of behavioral HDL descriptions , 2000, IEEE/ACM International Conference on Computer Aided Design. ICCAD - 2000. IEEE/ACM Digest of Technical Papers (Cat. No.00CH37140).