Specification and Testing Using Generalised Machines: A Presentation and a Case Study

Although testing is a major part of software development, it rarely gets the attention it deserves from researchers, partly because its foundations are weak and ill-understood. The principal purpose of testing is to detect (and then remove) faults in a software system. However, very few of the existing methods allow the tester to make any precise statement about the type or number of faults that remain undetected after testing is completed. In particular, none of the main techniques used by the software industry can give serious guarantees that a system is fault-free after testing has been completed. This paper advocates the use of a formal method both as a specification language and as a basis of a test data selection strategy. It presents a new method for generating test cases from this type of formal specification that provides a more convincing answer to the problem of detecting all faults in a software system. The method is reductionist in the sense that it guarantees that a system is fault-free provided that its components are fault-free; in turn, the same method could be used to test the resulting sub-systems, so the reduction will continue until the components considered are either known to be correct or are fairly simple pieces of code that can be successfully tested using traditional methods. The formal method used, X-machines, is a blend of finite state machines, data structures and processing functions and provides a simple and intuitive way of specifying computer systems. The use of X-machines as a specification tool and the testing method are illustrated with a case study. Index terms : Functional testing, test set, correctness, formal specification, finite state machines, X-machines.