Constraint-based test generation for automotive operating systems

This work suggests a method for systematically constructing a software-level environment model for safety checking automotive operating systems by introducing a constraint specification language, OSEK_CSL. OSEK_CSL is designed to specify the usage constraints of automotive operating systems using a pre-defined set of constraint types identified from the international standard OSEK/VDX. Each constraint specified in OSEK_CSL is interpreted as either a regular language or a context-free language that can be checked by a finite automaton or a pushdown automaton. The set of usage constraints is used to systematically classify the universal usage model of OSEK-/VDX-based operating systems and to generate test sequences with varying degrees of constraint satisfaction using LTL model checking. With pre-defined constraint patterns and the full support of automation, test engineers can choose the degree of constraint satisfaction and generate test cases using combinatorial intersections of selected constraints that cover all corner cases classified by constraints. A series of experiments on an open-source automotive operating system show that our approach finds safety issues more effectively than conventional specification-based testing, scenario-based testing, and conformance testing.

[1]  Michael Norrish,et al.  seL4: formal verification of an OS kernel , 2009, SOSP '09.

[2]  Sarfraz Khurshid,et al.  Incremental Test Generation for Software Product Lines , 2010, IEEE Transactions on Software Engineering.

[3]  Yunja Choi.  Constraint Specification and Test Generation for OSEK/VDX-Based Operating Systems , 2013, SEFM.

[4]  Jianqi Shi,et al.  ORIENTAIS: Formal Verified OSEK/VDX Real-Time Operating System , 2012, 2012 IEEE 17th International Conference on Engineering of Complex Computer Systems.

[5]  D. John.  OSEK/VDX conformance testing-MODISTARC , 1998 .

[6]  Elaine J. Weyuker,et al.  Automatically Generating Test Data from a Boolean Specification , 1994, IEEE Trans. Software Eng..

[7]  Sanjai Rayadurgam,et al.  Coverage based test-case generation using model checkers , 2001, Proceedings. Eighth Annual IEEE International Conference and Workshop On the Engineering of Computer-Based Systems-ECBS 2001.

[8]  Christian A. Müller,et al.  Towards the Formal Verification of a Distributed Real-Time Automotive System , 2010, NASA Formal Methods.

[9]  Qin Li,et al.  Modeling and Verifying the Code-Level OSEK/VDX Operating System with CSP , 2011, 2011 Fifth International Conference on Theoretical Aspects of Software Engineering.

[10]  Jean-Luc Béchennec,et al.  Trampoline An Open Source Implementation of the OSEK/VDX RTOS Specification , 2006, 2006 IEEE Conference on Emerging Technologies and Factory Automation.

[11]  Kenneth L. McMillan,et al.  Automated assumption generation for compositional verification , 2008, Formal Methods Syst. Des..

[12]  Toshiaki Aoki,et al.  Conformance Testing for OSEK/VDX Operating System Using Model Checking , 2011, 2011 18th Asia-Pacific Software Engineering Conference.

[13]  Yunja Choi,et al.  Model checking Trampoline OS: a case study on safety analysis for automotive software , 2014, Softw. Test. Verification Reliab..

[14]  Angelo Gargantini,et al.  An Evaluation of Specification Based Test Generation Techniques Using Model Checkers , 2009, 2009 Testing: Academic and Industrial Conference - Practice and Research Techniques.

[15]  Toshiaki Aoki,et al.  Automatic Generation of Model Checking Scripts Based on Environment Modeling , 2010, SPIN.

[16]  Wolfgang Rosenstiel,et al.  Semiformal verification of temporal properties in automotive hardware dependent software , 2009, 2009 Design, Automation & Test in Europe Conference & Exhibition.

[17]  Edmund M. Clarke,et al.  Counterexample-guided abstraction refinement , 2003, 10th International Symposium on Temporal Representation and Reasoning, 2003 and Fourth International Conference on Temporal Logic. Proceedings..

[18]  Leonardo Mendonça de Moura,et al.  A Model-Constructing Satisfiability Calculus , 2013, VMCAI.

[19]  Richard H. Carver,et al.  Use of sequencing constraints for specification-based testing of concurrent programs , 1998 .

[20]  Yongjun Li,et al.  A New Specification-based Test Data Generation Strategy for OSEK OS , 2010, 2010 10th IEEE International Conference on Computer and Information Technology.

[21]  Marta Z. Kwiatkowska,et al.  Abstraction Refinement for Probabilistic Software , 2008, VMCAI.

[22]  Gregor von Bochmann,et al.  FSM-based incremental conformance testing methods , 2004, IEEE Transactions on Software Engineering.

[23]  Natasha Sharygina,et al.  Leveraging Interpolant Strength in Model Checking , 2012, CAV.

[24]  Kim G. Larsen,et al.  Testing Real-Time Systems Using UPPAAL , 2008, Formal Methods and Testing.

[25]  Christel Baier,et al.  Modeling component connectors in Reo by constraint automata , 2004, Sci. Comput. Program..

[26]  Clémentine Nebut,et al.  Automatic test generation: a use case driven approach , 2006, IEEE Transactions on Software Engineering.

[27]  Takashi Kitamura,et al.  Formal Model-Based Test for AUTOSAR Multicore RTOS , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[28]  Manfred Broy,et al.  On the correctness of upper layers of automotive systems , 2008, Formal Aspects of Computing.

[29]  Bruno Dutertre,et al.  A Fast Linear-Arithmetic Solver for DPLL(T) , 2006, CAV.

[30]  Stavros Tripakis,et al.  Conformance testing for real-time systems , 2009, Formal Methods Syst. Des..

[31]  Thomas In der Rieden,et al.  An approach to the pervasive formal specification and verification of an automotive system: status report , 2005, FMICS '05.

[32]  Matthew B. Dwyer,et al.  Automated environment generation for software model checking , 2003, 18th IEEE International Conference on Automated Software Engineering, 2003. Proceedings..

[33]  Javier Tuya,et al.  Automatic generation of assumptions for modular verification of software specifications , 2006, J. Syst. Softw..

[34]  Angelo Gargantini,et al.  An Evaluation of Model Checkers for Specification Based Test Case Generation , 2009, 2009 International Conference on Software Testing Verification and Validation.

[35]  Michael Norrish,et al.  seL4: formal verification of an operating-system kernel , 2010, Commun. ACM.

[36]  Yunja Choi,et al.  Property-based Code Slicing for Efficient Verification of OSEK/VDX Operating Systems , 2012, FTSCS.

[37]  Thomas A. Henzinger,et al.  Interface automata , 2001, ESEC/FSE-9.

[38]  Thomas A. Henzinger,et al.  Lazy abstraction , 2002, POPL '02.

[39]  Jana Maria Heinsohn,et al.  Einführung in die ISO 26262 "Functional Safety - Road Vehicles" , 2011 .

[40]  Sarfraz Khurshid,et al.  Improving the effectiveness of spectra-based fault localization using specifications , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.