A Comparison of Static Analysis and Evolutionary Testing for the Verification of Timing Constraints

This paper contrasts two methods to verify timing constraints of real-time applications. The method of static analysis predicts the worst-case and best-case execution times of a task's code by analyzing execution paths and simulating processor characteristics without ever executing the program or requiring the program's input. Evolutionary testing is an iterative testing procedure, which approximates the extreme execution times within several generations. By executing the test object dynamically and measuring the execution times the inputs are guided yielding gradually tighter predictions of the extreme execution times. We examined both approaches on a number of real world examples. The results show that static analysis and evolutionary testing are complementary methods, which together provide upper and lower bounds for both worst-case and best-case execution times.

[1]  Per Stenström,et al.  An Integrated Path and Timing Analysis Method based on Cycle-Level Symbolic Execution , 1999, Real-Time Systems.

[2]  David B. Whalley,et al.  Bounding worst-case instruction cache performance , 1994, 1994 Proceedings Real-Time Systems Symposium.

[3]  Matthias Grochtmann,et al.  Verifying Timing Constraints of Real-Time Systems by Means of Evolutionary Testing , 1998, Real-Time Systems.

[4]  DasarathyB..  Timing Constraints of Real-Time Systems , 1985 .

[5]  Andy J. Wellings,et al.  Hybrid instruction cache partitioning for preemptive real-time systems , 1997, Proceedings Ninth Euromicro Workshop on Real Time Systems.

[6]  Duncan Clarke,et al.  Testing Real-Time Constraints in a Process Algebraic Setting , 1995, 1995 17th International Conference on Software Engineering.

[7]  Sharad Malik,et al.  Efficient microarchitecture modeling and path analysis for real-time software , 1995, Proceedings 16th IEEE Real-Time Systems Symposium.

[8]  Andrew Wolfe,et al.  Software-based cache partitioning for real-time applications , 1994 .

[9]  Thomas Bäck,et al.  Evolutionary algorithms in theory and practice - evolution strategies, evolutionary programming, genetic algorithms , 1996 .

[10]  Sharad Malik,et al.  Cache modeling for real-time software: beyond direct mapped instruction caches , 1996, 17th IEEE Real-Time Systems Symposium.

[11]  D. B. Kirk,et al.  SMART (strategic memory allocation for real-time) cache design , 1989, [1989] Proceedings. Real-Time Systems Symposium.

[12]  Miguel Felder,et al.  Testing Timing Behavior of Real-Time Software , 1997 .

[13]  Matthias Grochtmann.  Evolutionary Testing of Temporal Correctness , 1998 .

[14]  Bryan F. Jones,et al.  A Strategy for Using Genetic Algorithms to Automate Branch and Fault-Based Testing , 1998, Comput. J..

[15]  L. Darrell Whitley,et al.  The GENITOR Algorithm and Selection Pressure: Why Rank-Based Allocation of Reproductive Trials is Best , 1989, ICGA.

[16]  Frank Mueller,et al.  Timing Predictions for Multi-Level Caches , 1997 .

[17]  Reinhard Wilhelm,et al.  Efficient and Precise Cache Behavior Prediction for Real-Time Systems , 1999, Real-Time Systems.

[18]  Peter P. Puschner,et al.  Testing the results of static worst-case execution-time analysis , 1998, Proceedings 19th IEEE Real-Time Systems Symposium (Cat. No.98CB36279).

[19]  Joachim Wegener,et al.  Testing the temporal behavior of real-time tasks using extended evolutionary algorithms , 1999, Proceedings 20th IEEE Real-Time Systems Symposium (Cat. No.99CB37054).

[20]  Peter P. Puschner,et al.  Calculating the maximum execution time of real-time programs , 1989, Real-Time Systems.

[21]  Walter S. Heath.  Real-Time Software Techniques , 1990 .

[22]  Sang Lyul Min,et al.  An Accurate Worst Case Timing Analysis for RISC Processors , 1995, IEEE Trans. Software Eng..

[23]  James E. Baker,et al.  Reducing Bias and Inefficienry in the Selection Algorithm , 1987, ICGA.

[24]  David B. Whalley,et al.  Bounding Pipeline and Instruction Cache Performance , 1999, IEEE Trans. Computers.

[25]  Heinz Mühlenbein,et al.  Predictive Models for the Breeder Genetic Algorithm I. Continuous Parameter Optimization , 1993, Evolutionary Computation.

[26]  David B. Whalley,et al.  Timing Analysis for Data and Wrap-Around Fill Caches , 1999, Real-Time Systems.

[27]  Manuel E. Benitez,et al.  A portable global optimizer and linker , 1988, PLDI '88.

[28]  Joachim Wegener,et al.  A Comparison of Static Analysis and Evolutionary Testing for the Verification of Timing Constraints , 1998, Proceedings. Fourth IEEE Real-Time Technology and Applications Symposium (Cat. No.98TB100245).

[29]  Sang Lyul Min,et al.  Analysis of cache-related preemption delay in fixed-priority preemptive scheduling , 1998, 17th IEEE Real-Time Systems Symposium.

[30]  Jochen Liedtke,et al.  OS-controlled cache predictability for real-time systems , 1997, Proceedings Third IEEE Real-Time Technology and Applications Symposium.

[31]  B. Dasarathy,et al.  Timing Constraints of Real-Time Systems: Constructs for Expressing Them, Methods of Validating Them , 1989, IEEE Transactions on Software Engineering.

[32]  Frank Mueller,et al.  Static cache simulation and its applications , 1995 .

[33]  Frank Mueller.  Compiler support for software-based cache partitioning , 1995 .

[34]  Sandro Morasca,et al.  FUNCTIONAL TEST CASE GENERATION FOR REAL-TIME SYSTEMS , 1993 .

[35]  Frank Müller,et al.  Timing Analysis for Instruction Caches , 2000, Real-Time Systems.

[36]  Joachim Wegener,et al.  Testing real-time systems using genetic algorithms , 1997, Software Quality Journal.

[37]  Sang Lyul Min,et al.  Worst case timing analysis of RISC processors: R3000/R3010 case study , 1995, Proceedings 16th IEEE Real-Time Systems Symposium.

[38]  Sang Lyul Min,et al.  Efficient worst case timing analysis of data caching , 1996, Proceedings Real-Time Technology and Applications.

[39]  Matthias Grochtmann,et al.  Classification trees for partition testing , 1993, Softw. Test. Verification Reliab..

[40]  Hartmut Pohlheim,et al.  Genetic and evolutionary algorithm toolbox for use with matlab , 1994 .

[41]  Richard M. Stallman,et al.  Using and Porting GNU CC , 1998 .

[42]  Henrik Theiling,et al.  Fast and Precise WCET Prediction by Separated Cache and Path Analyses , 2000, Real-Time Systems.

[43]  Rajiv Gupta,et al.  Interprocedural conditional branch elimination , 1997, PLDI '97.

[44]  Harmen-Hinrich Sthamer,et al.  The automatic generation of software test data using genetic algorithms , 1995 .

[45]  David B. Whalley,et al.  Timing analysis for data caches and set-associative caches , 1997, Proceedings Third IEEE Real-Time Technology and Applications Symposium.

[46]  Andy J. Wellings,et al.  Adding instruction cache effect to schedulability analysis of preemptive real-time systems , 1996, Proceedings Real-Time Technology and Applications.

[47]  David B. Whalley,et al.  Integrating the timing analysis of pipelining and instruction caching , 1995, Proceedings 16th IEEE Real-Time Systems Symposium.

[48]  David B. Whalley,et al.  Avoiding conditional branches by code replication , 1995, PLDI '95.

[49]  Glenford J. Myers,et al.  Art of Software Testing , 1979 .

[50]  Chang Yun Park,et al.  Predicting program execution times by analyzing static and dynamic program paths , 1993, Real-Time Systems.