The theory and practice of specification based software testing

In this thesis my aim is to examine the common ground between formal methods and testing, and the benefits the two fields bring to one another. All too often they are regarded as mutually exclusive approaches in the development of software systems. The thesis begins with an examination of the motivation behind software testing, a summary of its development over the past few decades, and a survey of existing techniques. This involves a detailed discussion of some of those techniques, and leads on to an extensive case study. The case study shows how the use of a formal specification enables an existing “partition” based testing method to be used with far greater precision, but also highlights some of the limitations of the partition based techniques. The thesis continues with a comprehensive look at the development of theoretical models of testing since the mid 1970’s, and the way they have used successively more complex software models in order to be able to adequately describe suitable test cases. The remainder of the thesis is concerned with the introduction and use of Eilenberg’s X -machines as a formal model for the description of software specifications. The goal is to develop the X -machine model to the point where it is both useful and use-able as a tool for system specification, and at the same time the basis for a model of software testing so that test cases can be derived directly from the specification. To this end some of the theoretical properties of X -machines are examined, and some simple but very relevant results proved. The work is grounded on further case studies. Acknowledgements This work was supported by SERC Quota award number 89315080. I would like to thank my supervisor, Mike Holcombe, for all of his help, advice and enthusiasm towards my work over the past three years. At various stages, my work has benefited from discussions with Mike Stannett, Belkacem Mezhoud, Robert Plant, Mark Dunn and the rest of the Formal Methods and Software Engineering Group of the Computer Science Department and with Ian MConnell of the Applied and Computational Maths Department. Thanks also to the many emacs and TEX users who have selflessly placed their work in the public domain; in particular both J. M. Spivey and Paul Taylor’s LTEX styles have eased the writing and improved the appearance of this thesis. The case study of chapter 2 would have made far less sense without the helpful comments of Martin Woodward (Department of Computer Science, Liverpool University) and the anonymous referees of of the Journal of Software Testing, Verification and Reliability. Finally, thanks to Mary Rhoads who is the main reason this thesis is finished on time.