Evaluating the adaptivity of computing systems

Although adaptivity, the ability to adapt, is an important property of complex computing systems, so far little thought has been given to its evaluation. In this paper we propose a framework and methodology for the definition of benefit-based adaptivity metrics. The metrics thus defined allow an informed choice between systems based on their adaptivity to be made. We demonstrate application of the framework in a case study of restart strategies for Web Services Reliable Messaging. Additionally, we provide a broad survey of related approaches that may be used in the study of adaptivity (comprising, among others, robustness, performability, and control analysis), and evaluate their respective merits in relation to the proposed adaptivity metric.

[1]  Horst Zuse,et al.  Software metrics: using measurement theory to describe the properties and scales of static software complexity metrics , 1989, SIGP.

[2]  Philipp Reinecke,et al.  The Fast and the Fair: A Fault-Injection-Driven Comparison of Restart Oracles for Reliable Web Services , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[3]  William H. Sanders,et al.  The Möbius Framework and Its Implementation , 2002, IEEE Trans. Software Eng..

[4]  Donald F. Towsley,et al.  Measurement and modelling of the temporal dependence in packet loss , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).

[5]  Tyson R. Browning,et al.  Designing systems for adaptability by means of architecture options , 2008, Syst. Eng..

[6]  Aaron B. Brown,et al.  Measuring the Effectiveness of Self-Healing Autonomic Systems , 2005, Second International Conference on Autonomic Computing (ICAC'05).

[7]  Tyson R. Browning,et al.  2.1.2 Designing Systems for Adaptability by Means of Architecture Options , 2006 .

[8]  William H. Sanders,et al.  Extended Abstracts: PMCCS3: Third International Workshop on Performability Modeling of Computer and Communication Systems , 1997 .

[9]  Yixin Diao,et al.  Feedback Control of Computing Systems , 2004 .

[10]  Aaron B. Brown,et al.  Benchmarking autonomic capabilities: promises and pitfalls , 2004, International Conference on Autonomic Computing, 2004. Proceedings..

[11]  Themistoklis Charalambous,et al.  On Dynamic Resource Provisioning for Consolidated Servers in Virtual Data Centers , 2007 .

[12]  Ben Jann,et al.  Einführung in die Statistik , 2005 .

[13]  Andrew Jaquith.  Security Metrics: Replacing Fear, Uncertainty, and Doubt , 2007 .

[14]  C. Ghezzi,et al.  Towards Self-healing Service Compositions , 2004 .

[15]  S. Sastry,et al.  Adaptive Control: Stability, Convergence and Robustness , 1989 .

[16]  Kishor S. Trivedi,et al.  Techniques and Tools for Reliability and Performance Evaluation: Problems and Perspectives , 1994, Computer Performance Evaluation.

[17]  Ali Akoglu,et al.  Hierarchical Built-in Self-testing and FPGA Based Healing Methodology for System-on-a-Chip , 2007, Second NASA/ESA Conference on Adaptive Hardware and Systems (AHS 2007).

[18]  Katinka Wolter,et al.  Analysis of Restart Mechanisms in Software Systems , 2006, IEEE Transactions on Software Engineering.

[19]  Shari Lawrence Pfleeger,et al.  Towards a Framework for Software Measurement Validation , 1995, IEEE Trans. Software Eng..

[20]  Darrel C. Ince,et al.  Algebraic Validation of software Metrics , 1991, ESEC.

[21]  Victor M. Becerra,et al.  Optimal control , 2008, Scholarpedia.

[22]  Gerhard Haßlinger,et al.  The Gilbert-Elliott Model for Packet Loss in Real Time Services on the Internet , 2011, MMB.

[23]  Vern Paxson,et al.  End-to-end Internet packet dynamics , 1997, SIGCOMM '97.

[24]  Katinka Wolter,et al.  Analysis and algorithms for restart , 2004 .

[25]  Philipp Reinecke,et al.  A Measurement Study of the Interplay Between Application Level Restart and Transport Protocol , 2004, ISAS.

[26]  Hausi A. Müller,et al.  Quality Criteria and an Analysis Framework for Self-Healing Systems , 2007, International Workshop on Software Engineering for Adaptive and Self-Managing Systems (SEAMS '07).

[27]  Elaine J. Weyuker,et al.  Evaluating Software Complexity Measures , 2010, IEEE Trans. Software Eng..

[28]  John H. Holland,et al.  Adaptation in Natural and Artificial Systems: An Introductory Analysis with Applications to Biology, Control, and Artificial Intelligence , 1992 .

[29]  Rajarshi Das,et al.  Achieving Self-Management via Utility Functions , 2007, IEEE Internet Computing.

[30]  Philipp Reinecke,et al.  Adaptivity metric and performance for restart strategies in web services reliable messaging , 2008, WOSP '08.

[31]  Yin Zhang,et al.  On the constancy of internet path properties , 2001, IMW '01.

[32]  Wilhelm Hasselbring,et al.  A Classification Scheme for Self-adaptation Research , 2006 .

[33]  Karama Kanoun,et al.  Dependability benchmarking for computer systems , 2008 .

[34]  Lars Skyttner,et al.  General Systems Theory: Perspectives, Problems, Practice , 2006 .

[35]  Self-Managing Systems,et al.  Proceedings of the 2007 International Workshop on Software Engineering for Adaptive and Self-Managing Systems , 2007 .

[36]  Michael Dahlin,et al.  End-to-end WAN service availability , 2001, TNET.

[37]  Mehdi Hashemian.  Design for adaptability , 2005 .

[38]  Balachander Krishnamurthy,et al.  Web protocols and practice , 2001 .

[39]  Karama Kanoun,et al.  The Autonomic Computing Benchmark , 2008 .

[40]  Rainer Schlittgen.  Einführung in die Statistik: Analyse und Modellierung von Daten , 2008 .

[41]  Frank L. Lewis,et al.  Optimal Control , 1986 .

[42]  Philipp Reinecke,et al.  Experimental Analysis of the Correlation of HTTP GET Invocations , 2006, EPEW.

[43]  Mauricio Marín,et al.  Collecting data about moving objects in a distributed environment , 2004, XXIV International Conference of the Chilean Computer Science Society.

[44]  Philipp Reinecke,et al.  Phase-Type Approximations for Message Transmission Times in Web Services Reliable Messaging , 2008, SIPEW.

[45]  Stephen Gilmore,et al.  Automatic extraction of PEPA performance models from UML activity diagrams annotated with the MARTE profile , 2008, WOSP '08.

[46]  Lonnie R. Welch,et al.  Characterizing robustness in dynamic real-time systems , 2007, J. Syst. Softw..

[47]  PartridgeCraig,et al.  Improving round-trip time estimates in reliable transport protocols , 1991 .

[48]  John F. Meyer,et al.  On Evaluating the Performability of Degradable Computing Systems , 1980, IEEE Transactions on Computers.

[49]  Gerhard Haßlinger,et al.  Packet Loss in Real-Time Services: Markovian Models Generating QoE Impairments , 2008, 2008 16th Interntional Workshop on Quality of Service.