TESS: Automated Performance Evaluation of Self-Healing and Self-Adaptive Distributed Software Systems

This paper deals with the problem of evaluating and testing recovery and adaptation frameworks (RAF) for distributed software systems. We present TESS, a testbed for automatically generating distributed software architectures and their corresponding runtime applications, deploying them to the nodes of a cluster, running many different types of experiments involving failures and adaptation, and collecting in a database the values of a variety of failure recovery and adaptation metrics. Using the collected data, TESS automatically performs a thorough and scientific analysis of the efficiency and/or effectiveness of a RAF.This paper presents a case study on the use of TESS to evaluate DARE, a RAF developed by our group.

[1]  Jason Porter,et al.  Design and Experimentation of an Automated Performance Evaluation Testbed for Self-Healing and Self-Adaptive Distributed Software Systems , 2017 .

[2]  Claudia Szabo,et al.  Identifying Self-Organization and Adaptability in Complex Adaptive Systems , 2017, 2017 IEEE 11th International Conference on Self-Adaptive and Self-Organizing Systems (SASO).

[3]  Samuel Kounev,et al.  Performance evaluation of message-oriented middleware using the SPECjms2007 benchmark , 2009, Perform. Evaluation.

[4]  Daniel A. Menascé,et al.  DARE: A Distributed Adaptation and Failure Recovery Framework for Software Systems , 2017, 2017 IEEE International Conference on Autonomic Computing (ICAC).

[5]  Anas N. Al-Rabadi,et al.  A comparison of modified reconstructability analysis and Ashenhurst‐Curtis decomposition of Boolean functions , 2004 .

[6]  A. Gilles,et al.  The Art of Computer Systems Performance Analysis (Techniques for Experimental Design, Measurement, Simulation, and Modeling) , 1992 .

[7]  Jeff Magee,et al.  The Evolving Philosophers Problem: Dynamic Change Management , 1990, IEEE Trans. Software Eng..

[8]  A. Taleb-Bendiab,et al.  Performance evaluation for self-healing distributed services and fault detection mechanisms , 2006, J. Comput. Syst. Sci..

[9]  Steffen Becker,et al.  Model-driven performance engineering of self-adaptive systems: a survey , 2012, QoSA '12.

[10]  Sam Malek,et al.  SASSY: A Framework for Self-Architecting Service-Oriented Systems , 2011, IEEE Software.

[11]  Daniel A. Menascé,et al.  DeSARM: A Decentralized Mechanism for Discovering Software Architecture Models at Runtime in Distributed Systems , 2016, MoDELS@Run.time.

[12]  Jeffrey O. Kephart,et al.  The Vision of Autonomic Computing , 2003, Computer.

[13]  Daniel A. Menascé,et al.  Model-based Recovery Connectors for Self-adaptation and Self-healing , 2016, ICSOFT-EA.

[14]  Andrew Dinn,et al.  Performance and Dependability Evaluation of Distributed Event-based Systems: A Dynamic Code-injection Approach , 2017, ICPE.

[15]  Daniel A. Menascé,et al.  Model-Based Recovery and Adaptation Connectors: Design and Experimentation , 2016, ICSOFT.

[16]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.