Testing of Safety-Critical Systems - a Structural Approach to Test Case Design

In the development of many safety-critical systems, test cases are still created on the basis of experience rather than systematic methods. As a consequence, many redundant test cases are created and many aspects remain untested. One of the most important questions in testing dependable systems is: which are the right test techniques to obtain a test set that will detect critical errors in a complex system? In this paper, we provide an overview of the state-of-practice in designing test cases for dependable event-based systems regulated by the IEC 61508 and DO-178B standards. For example, the IEC 61508 standard stipulates modelbased testing and systematic test-case design and generation techniques such as transition-based testing and equivalence-class partitioning for software verification. However, it often remains unclear in which situation these techniques should be applied and what information is needed to select the right technique to obtain the best set of test cases. We propose an approach that selects appropriate test techniques by considering issues such as specification techniques, failure taxonomies and quality risks. We illustrate our findings with a case study for an interlocking system for Siemens transportation systems.