The Effectiveness of Statistical Testing when Applied to Logic Systems

In this paper we demonstrate the effectiveness of statistical testing for error detection on the example of a Programmable Logic System (PLS). The introduction of statistical testing arose from the wish to quantify the PLS's reliability. An appropriate statistical testing algorithm was devised and implemented, which is described in detail in this paper. We compare the results of statistical testing with those of a variety of other testing methods employed on the PLS. In terms of differences detected per number of tests, statistical testing showed an outstanding effectiveness. Furthermore, it detected a problem, which was missed by all other testing techniques. This together with its potential for reliability quantification illustrates its importance for system validation as part of a risk-based safety-case.