Evolutionary Testing of Embedded Systems

The development of embedded systems is an essential industrial activity. More than 90 % of all produced electronic components are used in embedded systems. Testing of embedded systems is considerably more complex than testing of conventional software systems. This is due on the one hand to the technical features of embedded systems, and on the other hand to the special requirements made on these kinds of systems: embedded systems usually have to fulfill functional as well as temporal requirements. Very often embedded systems are safety-relevant. In addition due to high costs resulting from errors occurring during the operation of embedded systems, high quality requirements apply. Dynamic testing is the most important method for testing such quality requirements. However, test case design is difficult to automate, therefore, most test cases have to be defined manually. A promising approach to automate test case design is the Evolutionary Test. It can be applied to testing the temporal behavior of systems, to structural testing as well as to safety testing. Effectiveness and efficiency of the test process can be clearly improved by Evolutionary Tests. This has been successfully proved in several case studies. Evolutionary Tests thus contribute to quality improvement as well as to the reduction of development costs.

[1]  Matthias Grochtmann,et al.  Systematic Testing of Real-Time Systems , 2000 .

[2]  Bryan F. Jones,et al.  A Strategy for Using Genetic Algorithms to Automate Branch and Fault-Based Testing , 1998, Comput. J..

[3]  John J. Grefenstette,et al.  Test and evaluation by genetic algorithms , 1993, IEEE Expert.

[4]  Martin Törngren,et al.  Fundamentals of Implementing Real-Time Control Applications in Distributed Computer Systems , 1998, Real-Time Systems.

[5]  Peter P. Puschner,et al.  Problems in Static Worst-Case Execution Time Analysis , 1997, MMB.

[6]  Edsger W. Dijkstra,et al.  Notes on structured programming , 1970 .

[7]  Harmen-Hinrich Sthamer,et al.  The automatic generation of software test data using genetic algorithms , 1995 .

[8]  H. Waeselynck,et al.  An Experimentation with Statistical Testing , 1994 .

[9]  J. Wegener,et al.  Test Case Design by Means of the CTE XL , 2000 .

[10]  Bryan F. Jones,et al.  Structural performance measure of evolutionary testing applied to worst-case timing of real-time systems , 2000, IEE Proc. Softw..

[11]  Peter P. Puschner,et al.  Testing the results of static worst-case execution-time analysis , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[12]  Joachim Wegener,et al.  Evolutionary test environment for automatic structural testing , 2001, Inf. Softw. Technol..

[13]  Fevzi Belli,et al.  Erprobte Modelle zur Quantifizierung der Software-Zuverlässigkeit , 1998, Informatik-Spektrum.

[14]  Ron Patton,et al.  Software Testing , 2000 .

[15]  Joachim Wegener,et al.  Testing the temporal behavior of real-time tasks using extended evolutionary algorithms , 1999, Proceedings 20th IEEE Real-Time Systems Symposium (Cat. No.99CB37054).

[16]  Matthias Grochtmann,et al.  Verifying Timing Constraints of Real-Time Systems by Means of Evolutionary Testing , 1998, Real-Time Systems.

[17]  Joachim Wegener,et al.  A Comparison of Static Analysis and Evolutionary Testing for the Verification of Timing Constraints , 1998, Proceedings. Fourth IEEE Real-Time Technology and Applications Symposium (Cat. No.98TB100245).

[18]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[19]  Richard G. Hamlet,et al.  Foundations of software testing: dependability theory , 1994, SIGSOFT '94.

[20]  Mirko Conrad,et al.  Model-based Generation and Structured Representation of Test Scenarios , 1999 .

[21]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[22]  Boris Beizer,et al.  Black Box Testing: Techniques for Functional Testing of Software and Systems , 1996, IEEE Software.

[23]  Simeon C. Ntafos,et al.  ON TESTING WITH REQUIRED ELEMENTS. , 1981 .

[24]  Richard G. Hamlet,et al.  Partition Testing Does Not Inspire Confidence , 1990, IEEE Trans. Software Eng..

[25]  Matthias Grochtmann,et al.  Testing Temporal Correctness of Real-Time Systems by Means of Genetic Algorithms , 1997 .

[26]  B. F. Jones,et al.  The Automatic Generation Of Software Test Data Sets Using Adaptive Search Techniques , 1970 .

[27]  Matthias Grochtmann,et al.  Classification trees for partition testing , 1993, Softw. Test. Verification Reliab..

[28]  Matthias Grochtmann,et al.  - Yet Another Computer-Aided Software Testing Tool? , 1994 .

[29]  Matthias Grochtmann.  Evolutionary Testing of Temporal Correctness , 1998 .

[30]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[31]  Frank Müller,et al.  Generalizing timing predictions to set-associative caches , 1997, Proceedings Ninth Euromicro Workshop on Real Time Systems.

[32]  John A. Clark,et al.  An automated framework for structural test-data generation , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[33]  Edsger W. Dijkstra,et al.  Structured programming , 1972, A.P.I.C. Studies in data processing.

[34]  William E. Howden,et al.  Functional program testing and analysis , 1986 .