Criteria for the evaluation of self-* systems

In the last years, the growing complexity of the current applications has led to the design of self-adapting systems presenting self-* properties. These systems are composed of several autonomous interactive entities. They behave autonomously and present enhanced characteristics allowing them to handle dynamics coming from exogenous and endogenous changes. In this paper, we propose a set of criteria for the description and evaluation of the adaptive properties of such systems. They aim to provide a concrete mechanism to analyze the quality of the design of adaptive systems, to evaluate the effect of self-* properties on the performances and to compare the adaptive features of different systems. The criteria are grouped into different categories: methodological, architectural, intrinsic, and runtime evaluation. They have been identified and specified by analyzing several case studies, which address self-adaptivity issues through different approaches with different objectives in various application contexts.

[1]  Claudia Raibulet,et al.  Towards Evaluation Mechanisms for Runtime Adaptivity: From Case Studies to Metrics , 2009, 2009 Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns.

[2]  Rolf P. Würtz,et al.  Introduction: Organic Computing , 2008, Organic Computing.

[3]  Paul Clements,et al.  Software architecture in practice , 1999, SEI series in software engineering.

[4]  Rogério de Lemos,et al.  Software Engineering for Self-Adaptive Systems [outcome of a Dagstuhl Seminar] , 2009, Software Engineering for Self-Adaptive Systems.

[5]  Claudia Raibulet.  Facets of Adaptivity , 2008, ECSA.

[6]  Seif Haridi,et al.  Distributed Algorithms , 1992, Lecture Notes in Computer Science.

[7]  Gauthier Picard,et al.  The ADELFE Methodology - Designing Adaptive Cooperative Multi-Agent Systems , 2004 .

[8]  Stephan Weibelzahl,et al.  Evaluation of Adaptive Systems , 2001, User Modeling.

[9]  Weishan Zhang,et al.  Towards Self-Managed Executable Petri Nets , 2008, 2008 Second IEEE International Conference on Self-Adaptive and Self-Organizing Systems.

[10]  Gauthier Picard,et al.  The ADELFE Methodology , 2004 .

[11]  Gauthier Picard,et al.  Characterizing and Evaluating Problem Solving Self-* Systems , 2009, 2009 Computation World: Future Computing, Service Computation, Cognitive, Adaptive, Content, Patterns.

[12]  Marie-Pierre Gleizes,et al.  Engineering Systems Which Generate Emergent Functionalities , 2008, EEMMAS.

[13]  Farokh B. Bastani,et al.  A Flexible Content Adaptation System Using a Rule-Based Approach , 2007 .

[14]  Amnon Meisels,et al.  Comparing performance of distributed constraints process ing algorithms , 2002 .

[15]  Claudia Raibulet,et al.  Evaluation of dynamic adaptivity through metrics: an achievable target? , 2009, 2009 Joint Working IEEE/IFIP Conference on Software Architecture & European Conference on Software Architecture.

[16]  Mary Shaw,et al.  An Introduction to Software Architecture , 1993, Advances in Software Engineering and Knowledge Engineering.

[17]  Philippe Mathieu,et al.  A Reverse Engineering Form for Multi Agent Systems , 2008, ESAW.

[18]  David Meignan,et al.  A Coalition-Based Metaheuristic for the vehicle routing problem , 2008, 2008 IEEE Congress on Evolutionary Computation (IEEE World Congress on Computational Intelligence).

[19]  Mauro Pezzè,et al.  Automatic Generation of Runtime Failure Detectors from Property Templates , 2009, Software Engineering for Self-Adaptive Systems.

[20]  Robert Laddaga,et al.  Introduction: The First International Workshop on Self-Adaptive Software , 2000, IWSAS.

[21]  Seyed Masoud Sadjadi,et al.  Composing adaptive software , 2004, Computer.

[22]  Ali A. Ghorbani,et al.  On the Evaluation of Adaptive Web Systems , 2004, Workshop on Web-based Support Systems.

[23]  Lawrence Chung,et al.  1 Metrics for Software Adaptability , 1999 .

[24]  P. Meseguer,et al.  On the Evaluation of DisCSP Algorithms ? , 2004 .

[25]  Marie-Pierre Gleizes,et al.  Self-Organisation and Emergence in MAS: An Overview , 2006, Informatica.

[26]  Gabor Karsai,et al.  An Approach to Self-adaptive Software Based on Supervisory Control , 2001, IWSAS.

[27]  David Garlan,et al.  Rainbow: architecture-based self-adaptation with reusable infrastructure , 2004 .

[28]  Farokh B. Bastani,et al.  A Flexible Content Adaptation System Using a Rule-Based Approach , 2007, IEEE Transactions on Knowledge and Data Engineering.

[29]  Mihalis Yannakakis,et al.  On the complexity of local search , 1990, STOC '90.

[30]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[31]  Yan Liu,et al.  An extensible and lightweight architecture for adaptive server applications , 2008 .

[32]  Natalia Stash,et al.  AHA! The adaptive hypermedia architecture , 2003, HYPERTEXT '03.