Reliability-Oriented Verification of Mission-Critical Software Systems

With software systems increasingly being employed in critical contexts, assuring high reliability levels for large, complex systems can incur huge verification costs. Critical system developers often encounter serious difficulties in satisfying reliability requirements at competitive and acceptable cost and time. Currently, it is not clear how engineers should plan an effective verification strategy oriented to improve the final reliability, since it is not trivial to figure out what activities mainly impact the reliability-cost trade-off and how much they affect reliability. Most often, crucial choices in the verification activity are left to the engineers� intuition, which base their decisions on personal expertise and on past experience, due to the lack of convincing approaches coping with them. However, when dealing with high reliability targets and tight time/cost constraints, engineers responsible for verification should have quantitative evidences of the consequences of their choices, and base their decision on them. One fundamental aspect in a reliability-oriented verification process concerns the identification of the most critical parts of the system, i.e., the major contributors to its unreliability. This is crucial to conveniently distribute efforts for verification. However, even suitably allocating efforts, engineers should know what verification techniques most impact the final reliability, and what techniques are most suited for the features of the system under test. Hence, the proper selection of verification techniques that best adapt to the specific system being developed is another critical challenge to be addressed. Coping with these issues, engineers could tune a verification process for their systems simply following a quantitative reasoning able to highlight cost/benefits of each choice. Based on these considerations, the thesis proposes a solution to carrying out an effective verification specifically oriented to improve reliability. It intends to provide engineers with quantitative means that should be adopted and embedded in their process, to allow them conveniently allocating efforts and selecting techniques for the system under test. The thesis first identifies the major open challenges to be faced, by trying to figure out what are the most crucial steps that engineers need to take for an effective planning. Then, to cope with them, it proposes: i) an optimization model to allocate verification effort to different system components in order to achieve a required reliability level at minimum verification costs; ii) an approach, based on empirical analyses, to quantitatively support the selection of the best verification techniques; iii) a procedure to improve verification processes in the considered class of systems, able to iteratively refine results across the developed projects.