State Coverage: An Empirical Analysis Based on a User Study

State coverage is a relatively new metric to evaluate the quality of test suites. While most existing test adequacy criteria measure the degree of exploration of the code under test, state coverage estimates the strength of the assertions in the test suite. Initial experiments have shown that there is a correlation between state coverage and mutation adequacy, and that expert users can discover new faults by modifying the test suite to increase state coverage. Since the faults injected by mutation testing are relatively simple, it is not clear whether these experiment are valid in a broader setting. Furthermore, these results may not be reproducible by average users, since they usually lack full understanding of the internals of the tool.