Timely and accurate detection of model deviation in self-adaptive software-intensive systems

Control-based approaches to self-adaptive software-intensive systems (SASs) are hailed for their optimal performance and theoretical guarantees on the reliability of adaptation behavior. However, in practice the guarantees are often threatened by model deviations occurred at runtime. In this paper, we propose a Model-guided Deviation Detector (MoD2) for timely and accurate detection of model deviations. To ensure reliability, a SAS can switch a control-based optimal controller for a mandatory controller once an unsafe model deviation is detected. MoD2 achieves both high timeliness and high accuracy through a deliberate fusion of parameter deviation estimation, uncertainty compensation, and safe region quantification. Empirical evaluation with three exemplar systems validated the efficacy of MoD2 (93.3% shorter detection delay, 39.4% lower FN rate, and 25.2% lower FP rate), as well as the benefits of the adaptation-switching mechanism (abnormal rate dropped by 29.2%).

[1]  Marin Litoiu,et al.  Observability and controllability of autonomic computing systems for composed Web services , 2011, 2011 6th IEEE International Symposium on Applied Computational Intelligence and Informatics (SACI).

[2]  Mary Shaw,et al.  What Can Control Theory Teach Us About Assurances in Self-Adaptive Software Systems? , 2017, Software Engineering for Self-Adaptive Systems.

[3]  Kang G. Shin,et al.  What does control theory bring to systems research? , 2009, OPSR.

[4]  Lecture Notes,et al.  Introduction to Bayesian Analysis , 2002 .

[5]  Martina Maggio,et al.  Testing self-adaptive software with probabilistic guarantees on performance metrics , 2020, ESEC/SIGSOFT FSE.

[6]  C. Knospe,et al.  PID control , 2006, IEEE Control Systems.

[7]  Rogério de Lemos Software Engineering for Self-Adaptive Systems III. Assurances , 2017, Lecture Notes in Computer Science.

[8]  Necmiye Ozay,et al.  Guaranteed model-based fault detection in cyber-physical systems: A model invalidation approach , 2016, Autom..

[9]  Danny Weyns,et al.  SimCA*: A Control-theoretic Approach to Handle Uncertainty in Self-adaptive Systems with Guarantees , 2019, ACM Trans. Auton. Adapt. Syst..

[10]  Bradley R. Schmerl,et al.  Proactive self-adaptation under uncertainty: a probabilistic model checking approach , 2015, ESEC/SIGSOFT FSE.

[11]  Mark Harman,et al.  Automated web application testing using search based software engineering , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[12]  Qiang Li,et al.  Kalman Filter and Its Application , 2015, 2015 8th International Conference on Intelligent Networks and Intelligent Systems (ICINIS).

[13]  Qi Tian,et al.  Appending Adversarial Frames for Universal Video Attack , 2019, 2021 IEEE Winter Conference on Applications of Computer Vision (WACV).

[14]  Lionel C. Briand,et al.  Approximation-Refinement Testing of Compute-Intensive Cyber-Physical Models: An Approach Based on System Identification , 2019, 2020 IEEE/ACM 42nd International Conference on Software Engineering (ICSE).

[15]  Chin-Shung Hsu Digital control systems , 1982, Proceedings of the IEEE.

[16]  Mary Shaw,et al.  Engineering Self-Adaptive Systems through Feedback Loops , 2009, Software Engineering for Self-Adaptive Systems.

[17]  Luciano Baresi,et al.  A discrete-time feedback controller for containerized cloud applications , 2016, SIGSOFT FSE.

[18]  Cristina Alcaraz,et al.  Recovery of Structural Controllability for Control Systems , 2014, Critical Infrastructure Protection.

[19]  Klaus Pohl,et al.  Online Reinforcement Learning for Self-adaptive Information Systems , 2020, CAiSE.

[20]  Mihaly Berekmeri,et al.  Control-Theoretical Software Adaptation: A Systematic Literature Review , 2017, IEEE Transactions on Software Engineering.

[21]  Thomas Vogel,et al.  Control Strategies for Self-Adaptive Software Systems , 2017, ACM Trans. Auton. Adapt. Syst..

[22]  Weidong Yang,et al.  Detection of model-plant mismatch in closed-loop control system , 2017 .

[23]  Naim A. Kheir,et al.  Control system design , 2001, Autom..

[24]  Catia Trubiani,et al.  Software performance self-adaptation through efficient model predictive control , 2017, 2017 32nd IEEE/ACM International Conference on Automated Software Engineering (ASE).

[25]  Bradley R. Schmerl,et al.  Rainbow: architecture-based self-adaptation with reusable infrastructure , 2004, International Conference on Autonomic Computing, 2004. Proceedings..

[26]  Carlo Ghezzi,et al.  Managing non-functional uncertainty via model-driven adaptivity , 2013, 2013 35th International Conference on Software Engineering (ICSE).

[27]  Carlo Ghezzi,et al.  Self-adaptive software meets control theory: A preliminary approach supporting reliability requirements , 2011, 2011 26th IEEE/ACM International Conference on Automated Software Engineering (ASE 2011).

[28]  Agathe Girard,et al.  Dynamic systems identification with Gaussian processes , 2005 .

[29]  S. Sivaganesan,et al.  Objective Priors for Parameters in a Normal Linear Regression with Measurement Error , 2013 .

[30]  O. Fecarotta,et al.  Wastewater Pump Control under Mechanical Wear , 2019, Water.

[31]  John Mylopoulos,et al.  Model Predictive Control for Software Systems with CobRA , 2016, 2016 IEEE/ACM 11th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[32]  Eduardo F. Camacho,et al.  Min–Max MPC based on an upper bound of the worst case cost with guaranteed stability. Application to a pilot plant , 2011 .

[33]  P. Kiessler,et al.  An Introduction to Bayesian Analysis: Theory and Methods , 2008 .

[34]  Sebastian G. Elbaum,et al.  Reducing failure rates of robotic systems though inferred invariants monitoring , 2013, 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems.

[35]  Henry Hoffmann,et al.  Automated design of self-adaptive software with control-theoretical formal guarantees , 2014, Software Engineering & Management.

[36]  Ladan Tahvildari,et al.  Towards a goal‐driven approach to action selection in self‐adaptive software , 2012, Softw. Pract. Exp..

[37]  Thomas Vogel,et al.  Software Engineering Meets Control Theory , 2015, 2015 IEEE/ACM 10th International Symposium on Software Engineering for Adaptive and Self-Managing Systems.

[38]  Jun Sun,et al.  Learning from Mutants: Using Code Mutation to Learn and Monitor Invariants of a Cyber-Physical System , 2018, 2018 IEEE Symposium on Security and Privacy (SP).

[39]  Qixin Wang,et al.  A System Identification Based Oracle for Control-CPS Software Fault Localization , 2019, 2019 IEEE/ACM 41st International Conference on Software Engineering (ICSE).

[40]  Swarnendu Biswas,et al.  Proteus: Language and Runtime Support for Self-Adaptive Software Development , 2019, IEEE Software.

[41]  M. Arlitt,et al.  Web Server Workload Characterization: The Search for Invariants , 1996, SIGMETRICS.

[42]  Fred Spiring,et al.  Introduction to Statistical Quality Control , 2007, Technometrics.

[43]  Martin Arlitt,et al.  A workload characterization study of the 1998 World Cup Web site , 2000, IEEE Netw..

[44]  Zheng Luo,et al.  Model (in)validation and fault detection for systems with polynomial state-space models , 2016, 2016 American Control Conference (ACC).

[45]  Henry Hoffmann,et al.  Automated control of multiple software goals using multiple actuators , 2017, ESEC/SIGSOFT FSE.

[46]  Yixin Diao,et al.  Feedback Control of Computing Systems , 2004 .

[47]  Aamir Saghir,et al.  Introduction to Statistical Process Control , 2020 .

[48]  Henry Hoffmann,et al.  Automated multi-objective control for self-adaptive software design , 2015, ESEC/SIGSOFT FSE.

[49]  Gang Huang,et al.  An adaptive offloading framework for Android applications in mobile edge computing , 2019, Science China Information Sciences.

[50]  Danny Weyns,et al.  Keep it SIMPLEX: satisfying multiple goals with guarantees in control-based self-adaptive systems , 2016, SIGSOFT FSE.

[51]  Silvio Simani,et al.  Model-Based Fault Diagnosis Techniques , 2003 .

[52]  Michèle Basseville,et al.  Detection of abrupt changes: theory and application , 1993 .

[53]  Sujeet Shenoi,et al.  Critical Infrastructure Protection VIII , 2014, IFIP Advances in Information and Communication Technology.

[54]  Hyochoong Bang,et al.  Introduction to Kalman Filter and Its Applications , 2018, Introduction and Implementations of the Kalman Filter.

[55]  Ignacio Vidal,et al.  Bayesian inference in measurement error models from objective priors for the bivariate normal distribution , 2019 .

[56]  W. P. M. H. Heemels,et al.  Observability and Controllability Analysis of Linear Systems Subject to Data Losses , 2018, IEEE Transactions on Automatic Control.

[57]  Henry Hoffmann,et al.  Self-Adaptive Video Encoder: Comparison of Multiple Adaptation Strategies Made Simple , 2017, 2017 IEEE/ACM 12th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[58]  Michel Kinnaert,et al.  Diagnosis and Fault-Tolerant Control , 2006 .

[59]  Siobhán Clarke,et al.  Model-based cross-layer monitoring and adaptation of multilayer systems , 2013, Science China Information Sciences.

[60]  Sam Malek,et al.  Taming uncertainty in self-adaptive software , 2011, ESEC/FSE '11.

[61]  Karl-Erik Årzén,et al.  Brownout: building more robust cloud applications , 2014, ICSE.

[62]  Yi Qin,et al.  CoMID: Context-Based Multiinvariant Detection for Monitoring Cyber-Physical Software , 2018, IEEE Transactions on Reliability.

[63]  Sridhar Adepu,et al.  An Investigation into the Response of a Water Treatment System to Cyber Attacks , 2016, 2016 IEEE 17th International Symposium on High Assurance Systems Engineering (HASE).

[64]  Martin F. Arlitt,et al.  Web server workload characterization: the search for invariants , 1996, SIGMETRICS '96.

[65]  Sirish L. Shah,et al.  Detection of model-plant mismatch in MPC applications ☆ , 2009 .

[66]  Fredrik Gustafsson,et al.  Adaptive Filtering and Change Detection: Gustafsson: Adaptive , 2001 .

[67]  Jun Han,et al.  A systematic survey on the design of self-adaptive software systems using control engineering approaches , 2012, 2012 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS).

[68]  Biao Huang,et al.  System Identification , 2000, Control Theory for Physicists.

[69]  John Mylopoulos,et al.  Engineering Self-Adaptive Software Systems , 2018, ACM Trans. Auton. Adapt. Syst..

[70]  M. V. Iordache,et al.  Diagnosis and Fault-Tolerant Control , 2007, IEEE Transactions on Automatic Control.

[71]  Daniel Jackson,et al.  Model-Based Security Analysis of a Water Treatment System , 2016, 2016 IEEE/ACM 2nd International Workshop on Software Engineering for Smart Cyber-Physical Systems (SEsCPS).