Three Approaches to Testing Theory.

Abstract : The relationship between program and specification is the core of programming theory. Since both program and specification are finite representations of infinite representations of infinite objects (the input-output behavior implemented and desired), it is not surprising that the relationship is not an effective one. Testing theory is concerned with the implications of limited program executions, with the ultimate goal of deciding a program's correctness. The difficulty at the heart of testing for correctness is that the class of programs computing each function is not algorithmically recognizable. With a functional specification, no test can distinguish correct programs from all others. However, this problem can be circumvented: by requiring human assistance in selecting tests; by strengthening specifications; or by treating tests as statistical samples. Each approach has also contributed useful insight to practical testing. The primary goal of testing theory is explanation. Many testing methods seem to work in practice, but there is no theoretical reason for their good performance. Experiments to validate testing methods are of doubtful validity because it is difficult to distinguish the contributions of people and the programming/testing process from those of the method. Too often success in practice can be traced to an accident of human intuition, in which the testing scheme is incidental.