Assisted Coverage Closure

Malfunction of safety-critical systems may cause damage to people and the environment. Software within those systems is rigorously designed and verified according to domain specific guidance, such as ISO26262 for automotive safety. This paper describes academic and industrial co-operation ini¾?tool development to support one of the most stringent of the requirements -- achieving full code coverage in requirements-driven testing. We present a verification workflow supported by a tool that integrates the coverage measurement tool RapiCover with the test-vector generator FShell. The tool assists closing the coverage gap by providing the engineer with test vectors that help in debugging coverage-related code quality issues and creating new test cases, as well as justifying the presence of unreachable parts of the code in order to finally achieve full effective coverage according to the required criteria. We illustrate the tool's practical utility on automotive industry benchmarks. It generates 8$$\times $$ more MC/DC coverage than random search.

[1]  Daniel Kroening,et al.  A Tool for Checking ANSI-C Programs , 2004, TACAS.

[2]  Nancy G. Leveson,et al.  An empirical evaluation of the MC/DC coverage criterion on the HETE-2 satellite software , 2000, 19th DASC. 19th Digital Avionics Systems Conference. Proceedings (Cat. No.00CH37126).

[3]  Mary Jean Harrold,et al.  Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage , 2003, IEEE Trans. Software Eng..

[4]  Neelam Gupta,et al.  A concept analysis inspired greedy algorithm for test suite minimization , 2005, PASTE '05.

[5]  John A. Clark,et al.  Automatic Test Data Generation for Multiple Condition and MCDC Coverage , 2009, 2009 Fourth International Conference on Software Engineering Advances.

[6]  Leanna Rierson,et al.  Developing Safety-Critical Software: A Practical Guide for Aviation Software and DO-178C Compliance , 2013 .

[7]  Raimund Kirner,et al.  Error Detection Rate of MC/DC for a Case Study from the Automotive Domain , 2010, SEUS.

[8]  Daniel Kroening,et al.  Chaining Test Cases for Reactive System Testing , 2013, ICTSS.

[9]  Daniel Kroening,et al.  Efficient Computation of Recurrence Diameters , 2003, VMCAI.

[10]  I. Toyn,et al.  Proof vs testing in the context of safety standards , 2005, 24th Digital Avionics Systems Conference.

[11]  Mary Sheeran,et al.  Checking Safety Properties Using Induction and a SAT-Solver , 2000, FMCAD.

[12]  Gordon Fraser,et al.  Testing with model checkers: a survey , 2009, Softw. Test. Verification Reliab..

[13]  Helmut Veith,et al.  FShell: Systematic Test Case Generation for Dynamic Analysis and Measurement , 2008, CAV.

[14]  Mark Harman,et al.  An Analysis and Survey of the Development of Mutation Testing , 2011, IEEE Transactions on Software Engineering.

[15]  Armin Biere,et al.  Symbolic Model Checking without BDDs , 1999, TACAS.

[16]  Z. ZamliKamal,et al.  On Test Case Generation Satisfying the MC/DC Criterion , 2013 .