An analysis of unit tests of a flight software product line

This paper presents an analysis of the unit testing approach developed and used by the Core Flight Software System (CFS) product line team at the NASA Goddard Space Flight Center (GSFC). The goal of the analysis is to understand, review, and recommend strategies for improving the CFS' existing unit testing infrastructure as well as to capture lessons learned and best practices that can be used by other software product line (SPL) teams for their unit testing. The results of the analysis show that the core and application modules of the CFS are unit tested in isolation using a stub framework developed by the CFS team. The application developers can unit test their code without waiting for the core modules to be completed, and vice versa. The analysis found that this unit testing approach incorporates many practical and useful solutions such as allowing for unit testing without requiring hardware and special OS features in-the-loop by defining stub implementations of dependent modules. These solutions are worth considering when deciding how to design the testing architecture for a SPL.

[1]  René L. Krikhaar,et al.  A two-phase process for software architecture improvement , 1999, Proceedings IEEE International Conference on Software Maintenance - 1999 (ICSM'99). 'Software Maintenance for Business Change' (Cat. No.99CB36360).

[2]  Eliane Martins,et al.  Constructing self-testable software components , 2001, 2001 International Conference on Dependable Systems and Networks.

[3]  Stuart I. Feldman,et al.  Make — a program for maintaining computer programs , 1979, Softw. Pract. Exp..

[4]  George T. Heineman Unit Testing of Software Components with Inter-component Dependencies , 2009, CBSE.

[5]  Sarfraz Khurshid,et al.  Incremental Test Generation for Software Product Lines , 2010, IEEE Transactions on Software Engineering.

[6]  J Hayhurst Kelly,et al.  A Practical Tutorial on Modified Condition/Decision Coverage , 2001 .

[7]  Frank J. van der Linden,et al.  Creating Architectures with Building Blocks , 1995, IEEE Softw..

[8]  David McComas,et al.  Connecting research and practice: an experience report on research infusion with software architecture visualization and evaluation , 2010, Innovations in Systems and Software Engineering.

[9]  Vuong Ly,et al.  Architectural Analysis of Systems Based on the Publisher-Subscriber Style , 2010, 2010 17th Working Conference on Reverse Engineering.

[10]  David McComas,et al.  Verifying architectural design rules of the flight software product line , 2009, SPLC.

[11]  Adam A. Porter,et al.  Using symbolic evaluation to understand behavior in configurable software systems , 2010, 2010 ACM/IEEE 32nd International Conference on Software Engineering.

[12]  Daniel Hoffman,et al.  Software Fundamentals: Collected Papers by David L.Parnas , 2001 .

[13]  David McComas,et al.  Architecture-Based Unit Testing of the Flight Software Product Line , 2010, SPLC.

[14]  Juha Taina,et al.  Product family testing: a survey , 2004, SOEN.

[15]  Mehdi Jazayeri,et al.  Software Architecture for Product Families: Principles and Practice , 2000 .

[16]  Klaus Pohl,et al.  Software product line testing , 2006, CACM.

[17]  Victor R. Basili Building an Experience Base for Software Engineering: A Report on the First CeBASE eWorkshop , 2001, PROFES.

[18]  Loe Feijs,et al.  A relational approach to support software architecture analysis , 1998 .

[19]  Tim Trew Enabling the Smooth Integration of Core Assets: Defining and Packaging Architectural Rules for a Family of Embedded Products , 2005, SPLC.

[20]  Frank van der Linden,et al.  Software Architectures for Product Families , 2000, Lecture Notes in Computer Science.

[21]  D. Ganesan,et al.  Software Architecture Discovery for Testability, Performance, and Maintainability of Industrial Systems , 2012 .

[22]  Mario Piattini,et al.  Software Product Line Testing - A Systematic Review , 2009, ICSOFT.