Combining testing and proof to gain high assurance in software: A case study

Dynamic software test methods are generally easy to use, but the results only apply to the specific input values tested. Static analysis produces results which are more general, but can require more effort to perform. There are potential benefits in combining both types of techniques because the results obtained can be more general than standalone dynamic testing but less resource-intensive than standalone static analysis. This paper presents a specific example of this approach applied to the verification of continuous monotonic functions. This approach combines a monotonicity analysis with a defined set of tests to demonstrate the accuracy of a software function over its entire input range. Unlike “standalone” dynamic methods, our approach provides full coverage, and guarantees a maximal error. We present a case study of the application of our approach to the analysis and testing of the software-implemented transfer function in a smart sensor. This demonstrated that relatively low levels of effort were needed to apply the approach. We conclude by discussing future developments of this approach.

[1]  M. R. Woodward,et al.  Quantifying the test effectiveness of Algol 68 programs , 1977, Strathclyde ALGOL 68 Conference.

[2]  Bernard Botella,et al.  A CLP Framework for Computing Structural Test Data , 2000, Computational Logic.

[3]  Toh Ne Win,et al.  Verifying Distributed Algorithms via Dynamic Analysis and Theorem Proving , 2002 .

[4]  Santiago Escobar,et al.  Automated Certification of Non-Interference in Rewriting Logic , 2008, FMICS.

[5]  Jan Jürjens,et al.  Modelling and Verification of Layered Security Protocols: A Bank Application , 2003, SAFECOMP.

[6]  Claude Marché,et al.  The Why/Krakatoa/Caduceus Platform for Deductive Program Verification , 2007, CAV.

[7]  Alain Denise,et al.  A new way of automating statistical testing methods , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[8]  Lukasz Garstecki,et al.  Generating Reliable Conformance Test Suites for Parallel and Distributed Languages, Libraries, and APIs , 2004, International Conference on Computational Science.

[9]  Richard M. Murray,et al.  Verification of Periodically Controlled Hybrid Systems: Application to an Autonomous Vehicle , 2012, TECS.

[10]  K. Rustan M. Leino,et al.  Annotation inference for modular checkers , 2001, Inf. Process. Lett..

[11]  K. Rustan M. Leino,et al.  Houdini, an Annotation Assistant for ESC/Java , 2001, FME.

[12]  J. Filliâtre,et al.  ACSL: ANSI/ISO C Specification Language , 2008 .

[13]  Armin Biere,et al.  Combined Static and Dynamic Analysis , 2005, AIOOL@VMCAI.

[14]  Michael D. Ernst Static and dynamic analysis: synergy and duality , 2003 .

[15]  Michael D. Ernst,et al.  Static verification of dynamically detected program invariants: Integrating Daikon and ESC/Java , 2001, RV@CAV.

[16]  Carroll Morgan,et al.  Programming from specifications , 1990, Prentice Hall International Series in computer science.

[17]  Bruno Marre,et al.  PathCrawler: Automatic Generation of Path Tests by Combining Static and Dynamic Analysis , 2005, EDCC.

[18]  Yves Deville,et al.  Consistency techniques for interprocedural test data generation , 2003, ESEC/FSE-11.

[19]  Manuel V. Hermenegildo,et al.  Combining Static Analysis and Profiling for Estimating Execution Times , 2006, PADL.