Branch Sequence Coverage Criterion for Testing-Based Formal Verification with Symbolic Execution

In specification-based testing, a desirable goal is to generate adequate test data to test every defined functional scenario in which all of its implementation paths are covered. However, experience suggests that this is a rather ambitious criterion that can hardly be met in practice due to the potentially great number of test data required. To address this challenge, we propose a new coverage criterion called Branch Sequence Coverage (BSC), aiming to strike a good balance between the assurance of program quality and the number of test data required by automatically finding all the necessary symbolic paths. In this paper, we discuss the features of BSC and compare it with partition testing (PT). We also illustrate how BSC can be well incorporated into a recently developed testing method known as Testing-Based Formal Verification with Symbolic Execution (TBFV-SE) for verification of the correctness of paths. We mainly focus on the exploration of the theoretical aspect of BSC and finally present a small case study to display the efficiency of BSC testing.

[1]  Shaoying Liu Testing-Based Formal Verification for Theorems and Its Application in Software Specification Verification , 2016, TAP@STAF.

[2]  Xavier Rival,et al.  Trace Partitioning in Abstract Interpretation Based Static Analyzers , 2005, ESOP.

[3]  Paul Ammann,et al.  Using formal methods to derive test frames in category-partition testing , 1994, Proceedings of COMPASS'94 - 1994 IEEE 9th Annual Conference on Computer Assurance.

[4]  Koushik Sen,et al.  Symbolic execution for software testing: three decades later , 2013, CACM.

[5]  Koushik Sen,et al.  CUTE: a concolic unit testing engine for C , 2005, ESEC/FSE-13.

[6]  Paul Ammann,et al.  Using Z specifications in category partition testing , 1992, COMPASS `92 Proceedings of the Seventh Annual Conference on Computer Assurance.

[7]  Rupak Majumdar,et al.  Cause clue clauses: error localization using maximum satisfiability , 2010, PLDI '11.

[8]  Shaoying Liu A Tool Supported Testing Method for Reducing Cost and Improving Quality , 2016, 2016 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[9]  Shaoying Liu,et al.  Generating test data from SOFL specifications , 1999, J. Syst. Softw..

[10]  Koushik Sen,et al.  CUTE and jCUTE: Concolic Unit Testing and Explicit Path Model-Checking Tools , 2006, CAV.

[11]  Michael R. Lowry,et al.  Combining unit-level symbolic execution and system-level concrete execution for testing nasa software , 2008, ISSTA '08.

[12]  Rong Wang,et al.  TBFV-SE: Testing-Based Formal Verification with Symbolic Execution , 2018, 2018 IEEE International Conference on Software Quality, Reliability and Security (QRS).

[13]  Corina S. Pasareanu,et al.  Symbolic PathFinder: symbolic execution of Java bytecode , 2010, ASE.

[14]  Shaoying Liu Formal Engineering for Industrial Software Development: Using the SOFL Method , 2004 .

[15]  Shaoying Liu,et al.  Formal Specification-Based Inspection for Verification of Programs , 2012, IEEE Transactions on Software Engineering.