Using Runtime Quantitative Verification to Provide Assurance Evidence for Self-Adaptive Software - Advances, Applications and Research Challenges

Providing assurance that self-adaptive software meets its dependability, performance and other quality-of-service (QoS) requirements is a great challenge. Recent approaches to addressing it use formal methods at runtime, to drive the reconfiguration of self-adaptive software in provably correct ways. One approach that shows promise is runtime quantitative verification (RQV), which uses quantitative model checking to reverify the QoS properties of self-adaptive software after environmental, requirement and system changes. This reverification identifies QoS requirement violations and supports the dynamic reconfiguration of the software for recovery from such violations. More importantly, it provides irrefutable assurance evidence that adaptation decisions are correct. In this paper, we survey recent advances in the development of efficient RQV techniques, the application of these techniques within multiple domains and the remaining research challenges.

[1]  Radu Calinescu,et al.  Adaptive model learning for continual verification of non-functional properties , 2014, ICPE.

[2]  Sebastián Uchitel,et al.  Synthesis of live behaviour models , 2010, FSE '10.

[3]  Anne H. H. Ngu,et al.  QoS-aware middleware for Web services composition , 2004, IEEE Transactions on Software Engineering.

[4]  Carlo Ghezzi,et al.  Synthesizing dynamically updating controllers from changes in scenario-based specifications , 2012, 2012 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[5]  Amir Pnueli,et al.  In Transition From Global to Modular Temporal Reasoning about Programs , 1989, Logics and Models of Concurrent Systems.

[6]  Rami Bahsoon,et al.  A decentralized self-adaptation mechanism for service-based applications in the cloud , 2013, IEEE Transactions on Software Engineering.

[7]  Carlo Ghezzi,et al.  Model evolution by run-time parameter adaptation , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[8]  Bradley R. Schmerl,et al.  Software Engineering for Self-Adaptive Systems: A Second Research Roadmap , 2010, Software Engineering for Self-Adaptive Systems.

[9]  Sarfraz Khurshid,et al.  Software assurance by bounded exhaustive testing , 2004, IEEE Transactions on Software Engineering.

[10]  Lijun Zhang,et al.  Probabilistic Reachability for Parametric Markov Models , 2009, SPIN.

[11]  Vincenzo Grassi,et al.  MOSES: A Framework for QoS Driven Runtime Adaptation of Service-Oriented Systems , 2012, IEEE Transactions on Software Engineering.

[12]  C. A. R. HOARE,et al.  An axiomatic basis for computer programming , 1969, CACM.

[13]  Radu Calinescu,et al.  An incremental verification framework for component-based software systems , 2013, CBSE '13.

[14]  Jesper Andersson,et al.  On decentralized self-adaptation: lessons from the trenches and challenges for the future , 2010, SEAMS '10.

[15]  Jeffrey O. Kephart,et al.  The Vision of Autonomic Computing , 2003, Computer.

[16]  Sebastián Uchitel,et al.  Hope for the best, prepare for the worst: multi-tier control for adaptive systems , 2014, ICSE.

[17]  Carlo Ghezzi,et al.  A Syntactic-Semantic Approach to Incremental Verification , 2013, ArXiv.

[18]  Carlo Ghezzi,et al.  Mining behavior models from user-intensive web applications , 2014, ICSE.

[19]  Danilo Ardagna,et al.  Adaptive Service Composition in Flexible Processes , 2007, IEEE Transactions on Software Engineering.

[20]  Radu Calinescu,et al.  Self-adaptive Software with Decentralised Control Loops , 2015, FASE.

[21]  Lijun Zhang,et al.  PARAM: A Model Checker for Parametric Markov Models , 2010, CAV.

[22]  Christel Baier,et al.  Probabilistic model checking for energy analysis in software product lines , 2013, MODULARITY.

[23]  Jeff Magee,et al.  FlashMob: distributed adaptive self-assembly , 2011, SEAMS '11.

[24]  Robert K. Brayton,et al.  Model-checking continuous-time Markov chains , 2000, TOCL.

[25]  Radu Calinescu,et al.  Using Continual Verication to Automate Service Selection in Service-Based Systems , 2013 .

[26]  Hongyang Qu,et al.  Incremental Runtime Verification of Probabilistic Systems , 2012, RV.

[27]  Bradley R. Schmerl,et al.  Optimal planning for architecture-based self-adaptation via model checking of stochastic games , 2015, SAC.

[28]  David Garlan,et al.  Stochastic game analysis and latency awareness for proactive self-adaptation , 2014, SEAMS 2014.

[29]  Radu Calinescu,et al.  Formal Methods @ Runtime , 2010, Monterey Workshop.

[30]  Nancy A. Lynch,et al.  Probabilistic Simulations for Probabilistic Processes , 1994, Nord. J. Comput..

[31]  Calin Belta,et al.  Incremental synthesis of control policies for heterogeneous multi-agent systems with linear temporal logic specifications , 2013, 2013 IEEE International Conference on Robotics and Automation.

[32]  Carlo Ghezzi,et al.  Run-time efficient probabilistic model checking , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[33]  Hongyang Qu,et al.  Assume-Guarantee Verification for Probabilistic Systems , 2010, TACAS.

[34]  Marta Z. Kwiatkowska,et al.  PRISM 4.0: Verification of Probabilistic Real-Time Systems , 2011, CAV.

[35]  Sebastián Uchitel,et al.  Synthesis of live behaviour models for fallible domains , 2011, 2011 33rd International Conference on Software Engineering (ICSE).

[36]  Thomas A. Henzinger,et al.  Theory in practice for system design and verification , 2015, SIGL.

[37]  Radu Calinescu,et al.  Efficient runtime quantitative verification using caching, lookahead, and nearly-optimal reconfiguration , 2014, SEAMS 2014.

[38]  Håkan L. S. Younes Ymer: A Statistical Model Checker , 2005, CAV.

[39]  Danny Weyns,et al.  Tele Assistance: A Self-Adaptive Service-Based System Exemplar , 2015, 2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems.

[40]  Radu Calinescu,et al.  Developing self-verifying service-based systems , 2013, 2013 28th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[41]  Radu Calinescu,et al.  Emerging Techniques for the Engineering of Self-Adaptive High-Integrity Software , 2013, Assurances for Self-Adaptive Systems.

[42]  Giordano Tamburrelli,et al.  Probabilistic Verification at Runtime for Self-Adaptive Systems , 2013, Assurances for Self-Adaptive Systems.

[43]  Hongyang Qu,et al.  Incremental quantitative verification for Markov decision processes , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[44]  Conrado Daws Symbolic and Parametric Model Checking of Discrete-Time Markov Chains , 2004, ICTAC.

[45]  Joost-Pieter Katoen,et al.  The Ins and Outs of the Probabilistic Model Checker MRMC , 2009, 2009 Sixth International Conference on the Quantitative Evaluation of Systems.

[46]  Roger C. Cheung,et al.  A User-Oriented Software Reliability Model , 1978, IEEE Transactions on Software Engineering.

[47]  Carlo Ghezzi,et al.  A formal approach to adaptive software: continuous assurance of non-functional requirements , 2011, Formal Aspects of Computing.

[48]  Radu Calinescu,et al.  Dynamic QoS Management and Optimization in Service-Based Systems , 2011, IEEE Transactions on Software Engineering.

[49]  Bradley R. Schmerl,et al.  On Patterns for Decentralized Control in Self-Adaptive Systems , 2010, Software Engineering for Self-Adaptive Systems.

[50]  Lars Grunske,et al.  An Efficient Method for Architecture-Based Reliability Evaluation for Evolving Systems with Changing Parameters , 2010, 2010 IEEE 21st International Symposium on Software Reliability Engineering.

[51]  Carlo Ghezzi,et al.  Self-adaptive software needs quantitative verification at runtime , 2012, CACM.

[52]  Wouter Joosen,et al.  The MACODO middleware for context-driven dynamic agent organizations , 2010, TAAS.

[53]  Carlo Ghezzi,et al.  Further steps towards efficient runtime verification: Handling probabilistic cost models , 2012, 2012 First International Workshop on Formal Methods in Software Engineering: Rigorous and Agile Approaches (FormSERA).

[54]  Carlo Ghezzi Evolution, Adaptation, and the Quest for Incrementality , 2012, Monterey Workshop.

[55]  Lijun Zhang,et al.  Synthesis for PCTL in Parametric Markov Decision Processes , 2011, NASA Formal Methods.

[56]  Radu Calinescu,et al.  Search-Based Synthesis of Probabilistic Models for Quality-of-Service Software Engineering (T) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[57]  Marta Z. Kwiatkowska Quantitative verification: models techniques and tools , 2007, ESEC-FSE '07.

[58]  Ioannis Konstantinou,et al.  Dependable Horizontal Scaling Based on Probabilistic Model Checking , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[59]  Rogério de Lemos,et al.  Evaluation of resilience in self-adaptive systems using probabilistic model-checking , 2012, 2012 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[60]  Marta Z. Kwiatkowska,et al.  Permissive Controller Synthesis for Probabilistic Systems , 2014, TACAS.

[61]  Marta Z. Kwiatkowska,et al.  Performance analysis of probabilistic timed automata using digital clocks , 2003, Formal Methods Syst. Des..

[62]  Edmund M. Clarke,et al.  Learning Probabilistic Systems from Tree Samples , 2012, 2012 27th Annual IEEE Symposium on Logic in Computer Science.

[63]  Carlo Ghezzi,et al.  Syntactic-semantic incrementality for agile verification , 2015, Sci. Comput. Program..

[64]  Radu Calinescu,et al.  Using observation ageing to improve markovian model learning in QoS engineering , 2011, ICPE '11.

[65]  Radu Calinescu,et al.  Using quantitative analysis to implement autonomic IT systems , 2009, 2009 IEEE 31st International Conference on Software Engineering.

[66]  Yuanyuan Zhang,et al.  Search-based software engineering: Trends, techniques and applications , 2012, CSUR.

[67]  Radu Calinescu,et al.  Compositional Reverification of Probabilistic Safety Properties for Large-Scale Complex IT Systems , 2012, Monterey Workshop.

[68]  Mahesh Viswanathan,et al.  Learning continuous time Markov chains from sample executions , 2004, First International Conference on the Quantitative Evaluation of Systems, 2004. QEST 2004. Proceedings..

[69]  Christel Baier,et al.  Principles of model checking , 2008 .