Worst-Case Scheduling of Software Tasks - A Constraint Optimization Model to Support Performance Testing

Real-Time Embedded Systems (RTES) in safety-critical domains, such as maritime and energy, must satisfy strict performance requirements to be deemed safe. Therefore, such systems have to be thoroughly tested to ensure their correct behavior even under the worst operating conditions. In this paper, we address the need of deriving worst case scenarios with respect to three com- mon performance requirements, namely task deadlines, response time, and CPU usage. Specifically, we investigate whether this worst-case analysis can be ef- fectively re-expressed as a Constrained Optimization Problem (COP) over the space of possible inputs to the system. Solving this problem means finding the sets of inputs that maximize the chance to violate performance requirements at runtime. Such inputs can in turn be used to test if the target RTES meets the expected performance even in the worst case. We develop an OPL model for IBM ILOG CP OPTIMIZER that implements a task priority-based preemptive scheduling, and apply it to a case study from the maritime and energy domain. Our validation shows that (1) the input to our model can be provided with rea- sonable effort in an industrial setting, and (2) the COP effectively identifies test cases that maximize deadline misses, response time, and CPU usage.

[1]  John A. Clark,et al.  Holistic schedulability analysis for distributed hard real-time systems , 1994, Microprocess. Microprogramming.

[2]  Laurence A. Wolsey,et al.  Integration of AI and OR Techniques in Constraint Programming for Combinatorial Optimization Problems, 4th International Conference, CPAIOR 2007, Brussels, Belgium, May 23-26, 2007, Proceedings , 2007, CPAIOR.

[3]  Wasif Afzal,et al.  A systematic review of search-based testing for non-functional system properties , 2009, Inf. Softw. Technol..

[4]  Mark Wallace,et al.  Principles and Practice of Constraint Programming – CP 2004 , 2004, Lecture Notes in Computer Science.

[5]  Hassan Gomaa,et al.  Designing concurrent, distributed, and real-time applications with UML , 2000, ICSE.

[6]  Mitsuo Gen,et al.  Advanced scheduling problem using constraint programming techniques in SCM environment , 2002 .

[7]  Mehrdad Sabetzadeh,et al.  Modeling and analysis of CPU usage in safety-critical embedded systems to support stress testing , 2012, MODELS'12.

[8]  D. J. Berndt,et al.  High Volume Software Testing using Genetic Algorithms , 2005, Proceedings of the 38th Annual Hawaii International Conference on System Sciences.

[9]  Boris Beizer,et al.  Software Testing Techniques , 1983 .

[10]  Abhishek Singh,et al.  Identifying Malicious Code Through Reverse Engineering , 2009, Advances in Information Security.

[11]  Edward A. Lee,et al.  Introduction to Embedded Systems - A Cyber-Physical Systems Approach , 2013 .

[12]  Hadrien Cambazard,et al.  Decomposition and Learning for a Hard Real Time Task Allocation Problem , 2004, CP.

[13]  Arnaud Gotlieb,et al.  Stress testing of task deadlines: A constraint programming approach , 2013, 2013 IEEE 24th International Symposium on Software Reliability Engineering (ISSRE).

[14]  Hadrien Cambazard,et al.  Solving a real-time allocation problem with constraint programming , 2008, J. Syst. Softw..

[15]  Tobias Nipkow,et al.  FM 2006: Formal Methods, 14th International Symposium on Formal Methods, Hamilton, Canada, August 21-27, 2006, Proceedings , 2006, FM.

[16]  Edward Tsang,et al.  Constraint Based Scheduling: Applying Constraint Programming to Scheduling Problems , 2003, J. Sched..

[17]  Hermann Kopetz,et al.  Real-time systems , 2018, CSC '73.

[18]  Martin W. P. Savelsbergh,et al.  Integer-Programming Software Systems , 2005, Ann. Oper. Res..

[19]  Walter F. Tichy,et al.  Proceedings 25th International Conference on Software Engineering , 2003, 25th International Conference on Software Engineering, 2003. Proceedings..

[20]  Lionel C. Briand,et al.  Using genetic algorithms for early schedulability analysis and stress testing in real-time systems , 2006, Genetic Programming and Evolvable Machines.

[21]  Alan Bundy,et al.  Constructing Induction Rules for Deductive Synthesis Proofs , 2006, CLASE.

[22]  Gert Smolka Principles and Practice of Constraint Programming-CP97 , 1997, Lecture Notes in Computer Science.

[23]  André Langevin,et al.  An Optimal Constraint Programming Approach to the Open-Shop Problem , 2012, INFORMS J. Comput..

[24]  Philippe Baptiste,et al.  Constraint - based scheduling : applying constraint programming to scheduling problems , 2001 .

[25]  Philippe Baptiste,et al.  An Experimental Comparison of Constraint-Based Algorithms for the Preemptive Job-Shop Scheduling Problem , 1997 .

[26]  Arnaud Gotlieb,et al.  Testing Deadline Misses for Real-Time Systems Using Constraint Optimization Techniques , 2012, 2012 IEEE Fifth International Conference on Software Testing, Verification and Validation.

[27]  Jonas Mellin,et al.  Test Case Generation for Mutation-based Testing of Timeliness , 2006, Electron. Notes Theor. Comput. Sci..

[28]  Thomas A. Henzinger,et al.  The Embedded Systems Design Challenge , 2006, FM.

[29]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[30]  Marina Schmid,et al.  Software For Dependable Systems Sufficient Evidence , 2016 .

[31]  Kevin Lano,et al.  Slicing of UML models using model transformations , 2010, MODELS'10.

[32]  Philippe Laborie,et al.  IBM ILOG CP Optimizer for Detailed Scheduling Illustrated on Three Problems , 2009, CPAIOR.

[33]  Diwakar Krishnamurthy,et al.  A model-based approach for testing the performance of web applications , 2006, SOQUA '06.

[34]  Elaine J. Weyuker,et al.  Experience with Performance Testing of Software Systems: Issues, an Approach, and Case Study , 2000, IEEE Trans. Software Eng..

[35]  Ron Bell,et al.  Introduction to IEC 61508 , 2006 .