The three dimensions of formal validation and verification of reactive system behaviors

Abstract : In-spite of three decades of software formal verification and validation (FV&V) research, there exists no ideal FV&V technique that works well for all FV&V concerns. That is, there is no one technique that enables the following: (1) easy and correct construction of requirement specification of complex real-life properties, and (2) complete verification coverage of complete real-life complex software with respect to those requirements. Moreover, many of the FV&V techniques are ineffective in handling temporal behavior of reactive systems. In this article, the authors present a visual tradeoff space, called the FV&V "tradeoff cuboid," for software engineers to discuss the various tradeoffs (e.g. cost, coverage, etc.) between different FV&V approaches in order to select the appropriate techniques for V&V. They illustrate the use of the tradeoff space with a discussion of cost and coverage tradeoffs among three categories of FV&V techniques: theorem proving, non-execution-based model checking, and execution-based model checking via the combination of runtime verification and automatic test generation. They show, using the cuboid, the pros and cons of the three categories of techniques. The authors advocate the assertion-based over the model-based approach to V&V for requirements specifications because the former allows system developers to modularize their thinking and focus on each property (or sets of properties) in isolation. In addition, it is much easier to verify the behavior of the actual system against each assertion (or sets of assertions) than comparing the equivalence of two monolithic formal models.