Search{Based Testing

This thesis is concerned with the problem of automatic test data generation for structural testing criteria, in particular the branch coverage adequacy criterion, using search–based techniques. The primary objective of this thesis is to advance the current state–of–the–art in automated search– based structural testing. Despite the large body of work within the field of search–based testing, the accompanying literature remains without convincing solutions for several important problems, including: support for pointers, dynamic data structures, and loop–assigned flag variables. Furthermore, relatively little work has been done to extend search–based testing to multi–objective problem formulations. One of the obstacles for the wider uptake of search–based testing has been the lack of publicly available tools, which may have contributed to the lack of empirical studies carried out on real–world systems. This thesis presents AUSTIN, a prototype structural test data generation tool for the C language. The tool is built on top of the CIL framework and combines a hill climber with a custom constraint solver for pointer type inputs. AUSTIN has been applied to five large open source applications, as well as eight non– trivial, machine generated C functions drawn from three real–world embedded software modules from the automotive sector. Furthermore, AUSTIN has been compared to a state–of–the–art Evolutionary Testing Framework and a dynamic symbolic execution tool, CUTE. In all cases AUSTIN was shown to be competitive, both in terms of branch coverage and efficiency. To address the problem of loop–assigned flags, this thesis presents a testability transformation along with a tool that transforms programs with loop– assigned flags into flag–free equivalents, so that existing search–based test data generation approaches can successfully be applied. The thesis concludes by introducing multi–objective branch coverage. It presents results from a case study of the twin objectives of branch coverage and dynamic memory consumption for both real and synthetic programs.