Model-Based Testing of Highly Configurable Embedded Systems

This chapter reports the results of a cycle computer case study and a previously conducted industrial case study from the automation domain. The key result is a model-based testing process for highly configurable embedded systems. The initial version of the testing process was built upon parameterizeable systems. The cycle computer case study adds the configuration using the product line concept and a feature model to store the parameterizable data. Thus, parameters and their constraints can be managed in a very structured way. Escalating demand for flexibility has made modern embedded software systems highly adjustable. This configurability is often realized through parameters and a highly configurable system possesses a handful of those. Small changes in parameter values can often account for significant changes in the system’s behavior, whereas in some other cases, changed parameters may not result in any perceivable reaction. The case studies address the challenge of applying model-based testing to configurable embedded software systems in order to reduce development effort. As a result of the case studies, a model-based testing process was developed. This process integrates existing model-based testing methods and tools such as combinatorial design and constraint processing as well as the product line engineering approach. The testing process was applied as part of the case studies and analyzed in terms of its actual saving potentials, which turned out to reduce the testing effort by more than a third. Detlef Streitferdt Technische Universität Ilmenau, Germany Florian Kantz ABB Corporate Research, Germany Philipp Nenninger ABB Automation Products, Germany Thomas Ruschival Datacom Telematica, Brazil Holger Kaul ABB Corporate Research, Germany Thomas Bauer Fraunhofer IESE, Germany Tanvir Hussain The Mathworks GmbH, Germany Robert Eschbach ITK Engineering AG, Germany DOI: 10.4018/978-1-4666-6034-2.ch010

[1]  Jeffrey M. Voas,et al.  Estimating the Probability of Failure When Testing Reveals No Failures , 1992, IEEE Trans. Software Eng..

[2]  Patrick Donohoe,et al.  Feature-Oriented Project Line Engineering , 2002, IEEE Softw..

[3]  Ina Schieferdecker,et al.  Model-Based Testing of Embedded Systems Exemplified for the Automotive Domain , 2010 .

[4]  Bruno Legeard,et al.  A taxonomy of model‐based testing approaches , 2012, Softw. Test. Verification Reliab..

[5]  Thomas Bauer,et al.  Risikobasierte Ableitung und Priorisierung von Testfällen für den modellbasierten Systemtest , 2008, Software Engineering.

[6]  Detlef Streitferdt,et al.  Test Case Mutation in Hybrid State Space for Reduction of No-Fault-Found Test Results in the Industrial Automation Domain , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[7]  Stacy J. Prowell TML: a description language for Markov chain usage models , 2000, Inf. Softw. Technol..

[8]  Stacy J. Prowell Using Markov Chain Usage Models to Test Complex Systems , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[9]  Detlef Streitferdt,et al.  Testing with Large Parameter Sets for the Development of Embedded Systems in the Automation Domain , 2009, 2009 33rd Annual IEEE International Computer Software and Applications Conference.

[10]  P. S. Thiagarajan,et al.  Message Sequence Charts , 2003, UML for Real.

[11]  Hassan Gomaa,et al.  Model-based testing for applications derived from software product lines , 2005, ACM SIGSOFT Softw. Eng. Notes.

[12]  Klaus Pohl,et al.  Software Product Line Engineering , 2005 .

[13]  Stacy J. Prowell,et al.  Foundations of Sequence-Based Software Specification , 2003, IEEE Trans. Software Eng..

[14]  Klaus Pohl,et al.  Testing Variabilities in Use Case Models , 2003, PFE.

[15]  Myra B. Cohen,et al.  Coverage and adequacy in software product line testing , 2006, ROSATEA '06.

[16]  D. Richard Kuhn,et al.  Software fault interactions and implications for software testing , 2004, IEEE Transactions on Software Engineering.

[17]  R. Eschbach,et al.  From Requirements to Statistical Testing of Embedded Systems , 2007, Fourth International Workshop on Software Engineering for Automotive Systems (SEAS '07).

[18]  John D. Musa,et al.  Operational profiles in software-reliability engineering , 1993, IEEE Software.

[19]  Robert Eschbach,et al.  Generating System Models for a Highly Configurable Train Control System Using a Domain-Specific Language: A Case Study , 2009, 2009 International Conference on Software Testing, Verification, and Validation Workshops.

[20]  Klaus Schmid,et al.  Product Line Engineering , 2013, IEEE Software.

[21]  Stacy J. Prowell,et al.  Cleanroom software engineering: technology and process , 1999 .

[22]  Thomas Bauer,et al.  Risk-based Statistical Testing: A Refinement- based Approach to the Reliability Analysis of Safety-Critical Systems , 2009 .