Observation-Enhanced QoS Analysis of Component-Based Systems

We present a new method for the accurate analysis of the quality-of-service (QoS) properties of component-based systems. Our method takes as input a QoS property of interest and a high-level continuous-time Markov chain (CTMC) model of the analysed system, and refines this CTMC based on observations of the execution times of the system components. The refined CTMC can then be analysed with existing probabilistic model checkers to accurately predict the value of the QoS property. The paper describes the theoretical foundation underlying this model refinement, the tool we developed to automate it, and two case studies that apply our QoS analysis method to a service-based system implemented using public web services and to an IT support system at a large university, respectively. Our experiments show that traditional CTMC-based QoS analysis can produce highly inaccurate results and may lead to invalid engineering and business decisions. In contrast, our new method reduced QoS analysis errors by 84.4–89.6 percent for the service-based system and by 94.7–97 percent for the IT support system, significantly lowering the risk of such invalid decisions.

[1]  Steffen Becker,et al.  Performance Prediction of Component-Based Systems - A Survey from an Engineering Perspective , 2004, Architecting Systems with Trustworthy Components.

[2]  Lars Grunske,et al.  Evaluating probabilistic models with uncertain model parameters , 2012, Software & Systems Modeling.

[3]  Tadashi Dohi,et al.  A Variational Bayesian Approach for Estimating Parameters of a Mixture of Erlang Distribution , 2010 .

[4]  Tadashi Dohi,et al.  Communications in Statistics-Simulation and Computation , 2014 .

[5]  Heiko Koziolek,et al.  Performance evaluation of component-based software systems: A survey , 2010, Perform. Evaluation.

[6]  K. Gopinath,et al.  Are Markov Models Effective for Storage Reliability Modelling? , 2015, ArXiv.

[7]  Philipp Reinecke,et al.  Phase-Type Distributions , 2012, Resilience Assessment and Evaluation of Computing Systems.

[8]  Steffen Becker,et al.  The Palladio component model for model-driven performance prediction , 2009, J. Syst. Softw..

[9]  Mark Fackrell,et al.  Modelling healthcare systems with phase-type distributions , 2009, Health care management science.

[10]  Bradley R. Schmerl,et al.  Improving self-adaptation planning through software architecture-based stochastic modeling , 2016, J. Syst. Softw..

[11]  Ren Asmussen,et al.  Fitting Phase-type Distributions via the EM Algorithm , 1996 .

[12]  C. O'Cinneide.  Phase-type distributions: open problems and a few properties , 1999 .

[13]  Juan A. Carrasco.  Computationally Efficient and Numerically Stable Reliability Bounds for Repairable Fault-Tolerant Systems , 2002, IEEE Trans. Computers.

[14]  Souheib Baarir,et al.  The GreatSPN tool: recent enhancements , 2009, PERV.

[15]  Hiroyuki Okamura,et al.  Fitting Phase-Type Distributions and Markovian Arrival Processes: Algorithms and Tools , 2016 .

[16]  FranksGreg,et al.  Enhanced Modeling and Solution of Layered Queueing Networks , 2009 .

[17]  David S. Rosenblum,et al.  Asymptotic Bounds for Quantitative Verification of Perturbed Probabilistic Systems , 2013, ICFEM.

[18]  Radu Calinescu,et al.  Developing self-verifying service-based systems , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[19]  Petr Novotný,et al.  Optimizing Performance of Continuous-Time Stochastic Systems using Timeout Synthesis , 2014, QEST.

[20]  Radu Calinescu,et al.  Search-Based Synthesis of Probabilistic Models for Quality-of-Service Software Engineering (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[21]  Marta Z. Kwiatkowska,et al.  Stochastic Model Checking , 2007, SFM.

[22]  Danna Zhou,et al.  d. , 1934, Microbial pathogenesis.

[23]  Nils Jansen,et al.  Synthesis and Verification of Self-aware Computing Systems , 2017, Self-Aware Computing Systems.

[24]  Dorina C. Petriu,et al.  The Future of Software Performance Engineering , 2007, Future of Software Engineering (FOSE '07).

[25]  Carlo Ghezzi,et al.  Formal Verification With Confidence Intervals to Establish Quality of Service Properties of Software Systems , 2016, IEEE Transactions on Reliability.

[26]  Radu Calinescu,et al.  Self-Adaptive Role-Based Access Control for Business Processes , 2017, 2017 IEEE/ACM 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[27]  Carlo Ghezzi,et al.  A formal approach to adaptive software: continuous assurance of non-functional requirements , 2011, Formal Aspects of Computing.

[28]  Miklós Telek,et al.  Matching More Than Three Moments with Acyclic Phase Type Distributions , 2007 .

[29]  William J. Stewart,et al.  Probability, Markov Chains, Queues, and Simulation: The Mathematical Basis of Performance Modeling , 2009 .

[30]  Philipp Reinecke,et al.  HyperStar: Phase-Type Fitting Made Easy , 2012, 2012 Ninth International Conference on Quantitative Evaluation of Systems.

[31]  Christel Baier,et al.  Model checking for performability , 2013, Mathematical Structures in Computer Science.

[32]  Gethin Norman,et al.  Quantitative Verification: Formal Guarantees for Timeliness, Reliability and Performance , 2014 .

[33]  Kishor S. Trivedi,et al.  Stochastic Modeling of Composite Web Services for Closed-Form Analysis of Their Performance and Reliability Bottlenecks , 2007, ICSOC.

[34]  Wang Yi,et al.  UPPAAL 4.0 , 2006, Third International Conference on the Quantitative Evaluation of Systems - (QEST'06).

[35]  Robert K. Brayton,et al.  Verifying Continuous Time Markov Chains , 1996, CAV.

[36]  A. Horváth,et al.  Matching Three Moments with Minimal Acyclic Phase Type Distributions , 2005 .

[37]  Gabriel Ciobanu,et al.  PHASE: A Stochastic Formalism for Phase-Type Distributions , 2014, ICFEM.

[38]  Christel Baier,et al.  Model-Checking Algorithms for Continuous-Time Markov Chains , 2002, IEEE Trans. Software Eng..

[39]  Lars Grunske,et al.  Software Architecture Optimization Methods: A Systematic Literature Review , 2013, IEEE Transactions on Software Engineering.

[40]  Kim G. Larsen,et al.  Testing Real-Time Systems Using UPPAAL , 2008, Formal Methods and Testing.

[41]  Raffaela Mirandola,et al.  Hybrid multi-attribute QoS optimization in component based software systems , 2013, J. Syst. Softw..

[42]  Radu Calinescu,et al.  Using Intelligent Proxies to Develop Self-Adaptive Service-Based Systems , 2013, 2013 International Symposium on Theoretical Aspects of Software Engineering.

[43]  Radu Calinescu,et al.  Efficient synthesis of robust models for stochastic systems , 2018, J. Syst. Softw..

[44]  C. Murray Woodside,et al.  Enhanced Modeling and Solution of Layered Queueing Networks , 2009, IEEE Transactions on Software Engineering.

[45]  Qing Zhang,et al.  Continuous-Time Markov Chains and Applications: A Two-Time-Scale Approach , 2012 .

[46]  Adele H. Marshall,et al.  Simulating Coxian phase-type distributions for patient survival , 2009, Int. Trans. Oper. Res..

[47]  Philipp Reinecke,et al.  Phase-Type Fitting Using HyperStar , 2013, EPEW.

[48]  A. Horváth,et al.  Approximating heavy tailed behaviour with Phase type distributions , 2000 .

[49]  Radu Calinescu,et al.  Self-adaptive Software with Decentralised Control Loops , 2015, FASE.

[50]  Carlo Ghezzi,et al.  Supporting Self-Adaptation via Quantitative Verification and Sensitivity Analysis at Run Time , 2016, IEEE Transactions on Software Engineering.

[51]  Philipp Reinecke,et al.  Cluster-based fitting of phase-type distributions to empirical data , 2012, Comput. Math. Appl..

[52]  Daniel A. Menascé.  Software, performance, or engineering? , 2002, WOSP '02.

[53]  Radu Calinescu,et al.  Accurate Analysis of Quality Properties of Software with Observation-Based Markov Chain Refinement , 2017, 2017 IEEE International Conference on Software Architecture (ICSA).

[54]  Connie U. Smith,et al.  Introduction to Software Performance Engineering: Origins and Outstanding Problems , 2007, SFM.

[55]  Marta Z. Kwiatkowska.  From software verification to ‘everyware’ verification , 2013, Computer Science - Research and Development.

[56]  Hypererlang approximation of probability distributions on (0, ∞) and its application , 1983 .

[57]  Joost-Pieter Katoen,et al.  How Fast and Fat Is Your Probabilistic Model Checker? An Experimental Performance Comparison , 2007, Haifa Verification Conference.

[58]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[59]  Joost-Pieter Katoen,et al.  Beyond Memoryless Distributions: Model Checking Semi-Markov Chains , 2001, PAPM-PROBMIV.

[60]  Murali Sitaraman,et al.  Foundations of component-based systems , 2000 .

[61]  Peter Buchholz,et al.  A Novel Approach for Phase-Type Fitting with the EM Algorithm , 2006, IEEE Transactions on Dependable and Secure Computing.

[62]  Hausi A. Müller,et al.  Runtime Evolution of Highly Dynamic Software , 2014, Evolving Software Systems.

[63]  Danny Weyns,et al.  Engineering Trustworthy Self-Adaptive Software with Dynamic Assurance Cases , 2017, IEEE Transactions on Software Engineering.

[64]  Jin Liu,et al.  Segment-based adaptive hyper-Erlang model for long-tailed network traffic approximation , 2008, The Journal of Supercomputing.

[65]  Marco Ajmone Marsan,et al.  Modelling with Generalized Stochastic Petri Nets , 1995, PERV.

[66]  Vojtech Rehák,et al.  Dealing with Zero Density Using Piecewise Phase-Type Approximation , 2014, EPEW.

[67]  Eva Ishay,et al.  Fitting Phase-Type Distributions to Data from a Telephone Call Center , 2002 .

[68]  Carlo Ghezzi,et al.  Quality Prediction of Service Compositions through Probabilistic Model Checking , 2008, QoSA.

[69]  Radu Calinescu,et al.  Synthesis of probabilistic models for quality-of-service software engineering , 2018, Automated Software Engineering.

[70]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[71]  Carlo Ghezzi,et al.  Mining behavior models from user-intensive web applications , 2014, ICSE.

[72]  Aldo Cumani,et al.  ML estimation of the parameters of a PH distribution in triangular canonical form , 2014 .

[73]  Joost-Pieter Katoen,et al.  The Ins and Outs of the Probabilistic Model Checker MRMC , 2009, 2009 Sixth International Conference on the Quantitative Evaluation of Systems.

[74]  Aarnout Brombacher,et al.  Probability... , 2009, Qual. Reliab. Eng. Int..

[75]  Sebastian Junges,et al.  A Storm is Coming: A Modern Probabilistic Model Checker , 2017, CAV.

[76]  Raffaela Mirandola,et al.  QoS and energy management with Petri nets: A self-adaptive framework , 2012, J. Syst. Softw..

[77]  Tsuyoshi Murata,et al.  {m , 1934, ACML.