A search-based framework for automatic test-set generation for MATLAB/Simulink models

Modern testing requires faults to be discovered at the earliest possible stage, i.e. specification or architecture design stage rather than at the coding stage, because the cost of fixing an error increases with the time between its introduction and detection. Testing is required to exercise high-level models. Simulink is a software package for modelling, simulating, and analysing system-level designs of dynamic systems. Simulink has been widely used in designing large-scale embedded systems, including engine controllers. A prominent feature of such systems is that they are of high complexity and with a need for high integrity. Thus, the verification and validation processes for such systems need to be stringent. Search-based test-data generation techniques have been widely used in code level testing. Testing of various functional and non-functional properties has been targeted, as has fulfilling certain structural coverage criteria. This work takes a step further to attempt test-data generation at the higher levels of abstraction offered by Simulink models. The thesis shows how search-based test-data generation techniques, taken largely from automated code-level testing, can be adapted for Simulink models and applied to generate test data that satisfy identified test aims. In this thesis, these aims are mainly concerned with analogues of code-level structural and fault-based coverage criteria. The presence of persistent state has been shown to be problematic at the code level and such difficulties remain when Simulink models are to be tested. In order to address this problem, a technique called tracing and deducing is introduced, which helps provide better guidance to the search. Test-set extraction techniques have been used for regression testing test selection. The work in this thesis extends the existing body of work to address a wider range of problems. The use of the test-set extraction techniques enables efficient test-sets to be provided. All the above techniques are finally encapsulated into a comprehensive testing framework, which enables test-sets that are both effective and efficient to be generated. A number of experiments show that the proposed techniques are suited to their tasks and that the framework effectively and efficiently generates test-sets for Simulink models.