Evidence-based verification

The ability of generating diagnostic information and performing other “post-verification” analyses is an important feature of verification tools. Traditionally these analyses rely on the proof computed during verification and hence are tightly geared to the infrastructures of these checkers. We present a framework for certifying verification results and efficiently generating diagnostic information in the domain of equivalence checking, preorder checking, and model checking of finite-state system. The central idea is to use a generic data structure called “abstract proof structure” (APS) to abstractly encode the proof structure by which a verification engine (checker) reaches its result. APS carries checker-independent evidence for justifying verification result and provides the basis for a variety of “post-verification” analyses. In this framework, APS serves as interface data structure between checkers and analysis routines. Checkers are enhanced with bookkeeping code to produce APSs as their results instead of simple “yes/no” answer. A wide range of existing checkers can be modified to produce APS without compromising their time and space complexities. We then show how APSs may be used to perform an array of “post-verification” analyses, including certification of verification result and generation of different forms of diagnostic information. An immediate usage of APS is to certify verification result by checking the internal consistency of APS submitted by checker. We provide efficient algorithms to preform such checking. The primary goal of this framework is to improve the efficiency and flexibility of diagnostic generation routines. We show that many diagnostic analyses can be preformed using APSs, including some traditional (e.g., counterexamples [7] in model checking and HML formulas [22] for bisimulation) and novel (e.g., vacuity detection [2]) diagnostic schema. We also show how winning strategies for property-checking games [43] can be built from APS. The analysis routines based on APS enjoy independence from checkers; hence they can be easily modified and migrated from one checker to another. In essence APSs help standardize the output of checkers, and analysis routines therefore can be created and executed independently, which has other interesting applications such as a client-server architectures for verification.