Contributions of model checking and CoFI methodology to the development of space embedded software

The role of embedded software in the last space accidents highlights the importance of verification and validation techniques for the development of space embedded software. In this context, this work analyses the contribution of two verification techniques applied to the onboard data handling software of space products. The first technique is model checking. The system is modeled by a set of timed automata and the verification of safety and liveness properties is performed using UPPAAL model checker. The verified model is then used to generate the embedded software. The second technique analyzed in this work is model based approach for the generation of test cases. The Conformance and Fault Injection (CoFI) testing methodology is used to guide the development of a set of Finite State Machine (FSM) models from the software specification. The test suite is automatically generated from the FSM models. The contributions of the two methodologies are analyzed based on the results provided by an experiment. Two software products are used as case study, each one implementing two services of the Packet Utilization Standard (PUS). These services represent the functionalities offered by a satellite onboard data handling computer. One of the products is developed with the aid of model checking, while the other is developed according to the practices currently used at the Instituto Nacional de Pesquisas Espaciais (INPE). Both software products are tested by the CoFI methodology. The experiment highlights the advantages and vulnerable points of model checking. It also demonstrates that the main contribution of CoFI testing methodology is to highlight problems related to situations that have not been considered in the software specification, such as the occurrence of inopportune events. This analysis helps to understand how different techniques can be integrated in the design of critical embedded software.

[1]  A. Agresti.  An introduction to categorical data analysis , 1997 .

[2]  Mirko Conrad,et al.  Systematic Model-Based Testing of Embedded Automotive Software , 2004, MBT.

[3]  Emilia Villani,et al.  A Comparative Analysis of two Verification Techniques for DEDS: Model Checking versus Model-based Testing , 2009 .

[4]  Claes Wohlin,et al.  Experimentation in software engineering: an introduction , 2000 .

[5]  Kim G. Larsen,et al.  A Tutorial on Uppaal , 2004, SFM.

[6]  Rodrigo Pastl Pontes,et al.  Contribuições do model checking e da metodologia CoFi para o software embarcado espacial , 2011 .

[7]  Marcel Verhoef,et al.  Timed automata based analysis of embedded system architectures , 2006, IPDPS.

[8]  David Notkin,et al.  Model checking large software specifications , 1996, SIGSOFT '96.

[9]  Ana R. Cavalli,et al.  FSM-based conformance testing methods: A survey annotated with experimental evaluation , 2010, Inf. Softw. Technol..

[10]  G.J. Holzmann,et al.  Using SPIN model checking for flight software verification , 2002, Proceedings, IEEE Aerospace Conference.

[11]  Shlomo Greenberg,et al.  Evaluating and comparing simulation verification vs. formal verification approach on block level design , 2004, Proceedings of the 2004 11th IEEE International Conference on Electronics, Circuits and Systems, 2004. ICECS 2004..

[12]  Kwang-Ting Cheng,et al.  A comparison of BDDs, BMC, and sequential SAT for model checking , 2003, Eighth IEEE International High-Level Design Validation and Test Workshop.

[13]  Marco Vieira,et al.  Benchmarking Software Requirements Documentation for Space Application , 2010, SAFECOMP.

[14]  Shinichi Honiden,et al.  Model Checking Process with Goal Oriented Requirements Analysis , 2008, 2008 15th Asia-Pacific Software Engineering Conference.

[15]  Jiang Chau Wang,et al.  Comparing two testbench methods for hierarchical functional verification of a bluetooth baseband adaptor , 2005, 2005 Third IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS'05).

[16]  Mak Tafazoli,et al.  A study of on-orbit spacecraft failures , 2009 .

[17]  Eliane Martins,et al.  A Conformance Testing Process for Space Applications Software Services , 2006, J. Aerosp. Comput. Inf. Commun..

[18]  Alexander Pretschner,et al.  Abstractions for Model-Based Testing , 2005, Electron. Notes Theor. Comput. Sci..

[19]  Eliane Martins,et al.  ConData: A Tool for Automating Specification-Based Test Case Generation for Communication Systems , 2004, Software Quality Journal.

[20]  Ashish Tiwari,et al.  On the run-time verification of autonomy software , 2003, 28th Annual NASA Goddard Software Engineering Workshop, 2003. Proceedings..

[21]  M. Whalen,et al.  Software model checking for avionics systems , 2008, 2008 IEEE/AIAA 27th Digital Avionics Systems Conference.

[22]  Rodrigo Pastl Pontes,et al.  MODEL-BASED REFINEMENT OF REQUIREMENT SPECIFICATION : A COMPARISON OF TWO V & V APPROACHES , 2009 .

[23]  Rajeev Alur,et al.  A Theory of Timed Automata , 1994, Theor. Comput. Sci..

[24]  Marcelo Henrique,et al.  A New Model-Based Approach for Specification Analysis and Refinement of Space Operations , 2010 .

[25]  Ronaldo Arias,et al.  Lessons Learned from an Onboard ECSS PUS Object-Oriented Implementation , 2008 .

[26]  Olivier Notebaert.  Benefits of the Standardization Efforts for On -Board Data Interfaces and Services , 2006 .

[27]  Kristin Yvonne Rozier,et al.  Linear Temporal Logic Symbolic Model Checking , 2011, Comput. Sci. Rev..

[28]  Amit M. Paradkar.  Towards model-based generation of self-priming and self-checking conformance tests for interactive systems , 2003, SAC '03.

[29]  W. Eric Wong,et al.  The Role of Software in Recent Catastrophic Accidents , 2009 .

[30]  F. Garcia,et al.  Formal Verification of Safety and Liveness Properties for Logic Controllers. A Tool Comparison , 2006, 2006 3rd International Conference on Electrical and Electronics Engineering.

[31]  Tobias Schüle,et al.  Global vs. local model checking: a comparison of verification techniques for infinite state systems , 2004, Proceedings of the Second International Conference on Software Engineering and Formal Methods, 2004. SEFM 2004..

[32]  Eliane Martins,et al.  Embedded Critical Software Testing for Aerospace Applications based on PUS , 2010 .

[33]  Gerard J. Holzmann,et al.  Validation of mission critical software design and implementation using model checking [spacecraft] , 2002, Proceedings. The 21st Digital Avionics Systems Conference.

[34]  Guilherme Horta Travassos,et al.  Model-based testing approaches selection for software projects , 2009, Inf. Softw. Technol..

[35]  Nancy G. Leveson,et al.  Role of Software in Spacecraft Accidents , 2004 .

[36]  Eliane Martins,et al.  An Independent Software Verification and Validation Process for Space Applications , 2008 .

[37]  Lodewijk Bergmans,et al.  Experimental evaluation of a tool for the verification and transformation of source code in event-driven systems , 2009, Empirical Software Engineering.

[38]  Joanne M. Atlee,et al.  Feasibility of model checking software requirements: a case study , 1996, Proceedings of 11th Annual Conference on Computer Assurance. COMPASS '96.