An observability-based code coverage metric for functional simulation

Functional simulation is the most widely used method for design verification. At various levels of abstraction, e.g., behavioral, register-transfer level and gate level, the designer simulates the design using a large number of vectors attempting to debug and verify the design. A major problem with functional simulation is the lack of good metrics and tools to evaluate the quality of a set of functional vectors. Metrics used currently are based on instruction counts and are quite simplistic. Designers are forced to use ad-hoc methods to terminate functional simulation, e.g., CPU time limitations, We propose a new metric for measuring the extent of design verification provided by a set of functional simulation vectors. This metric is universal, and can be used uniformly for all designs. Our metric computes observability information to determine whether effects of errors that are activated by the program stimuli can be observed at the circuit outputs. We provide preliminary experimental evidence that supports the validity of the proposed metric. We believe that using this metric in design verification will result in higher-quality functional tests and improved correctness checking.

[1]  Boris Beizer,et al.  Software testing techniques (2. ed.) , 1990 .

[2]  Melvin A. Breuer,et al.  Digital systems testing and testable design , 1990 .

[3]  Kwang-Ting Cheng,et al.  Transition fault testing for sequential circuits , 1993, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[4]  Krishan K. Sabnani,et al.  An algorithmic procedure for checking safety properties of protocols , 1989, IEEE Trans. Commun..

[5]  Orna Grumberg,et al.  Research on Automatic Verification of Finite-State Concurrent Systems , 1987 .

[6]  Michael Kantrowitz,et al.  I'm done simulating; now what? Verification coverage analysis and correctness checking of the DEC chip 21164 Alpha microprocessor , 1996, DAC '96.

[7]  Mike Gordon,et al.  Hardware verification by formal proof , 1985 .

[8]  Kenneth A. Foster Error Sensitive Test Cases Analysis (ESTCA) , 1980, IEEE Transactions on Software Engineering.

[9]  Jacob A. Abraham,et al.  Test Generation for Microprocessors , 1980, IEEE Transactions on Computers.

[10]  William E. Howden,et al.  Reliability of the Path Analysis Testing Strategy , 1976, IEEE Transactions on Software Engineering.

[11]  Jacob A. Abraham,et al.  Functional Testing of Microprocessors , 1984, IEEE Transactions on Computers.

[12]  Donald E. Thomas,et al.  The Verilog® Hardware Description Language , 1990 .

[13]  Sungho Kang,et al.  Modelling and simulation of design errors , 1992, Proceedings 1992 IEEE International Conference on Computer Design: VLSI in Computers & Processors.

[14]  J. Paul Roth,et al.  Diagnosis of automata failures: a calculus and a method , 1966 .

[15]  Brian Marick,et al.  The craft of software testing , 1994 .

[16]  Boris Beizer,et al.  Software Testing Techniques , 1983 .