A comparison framework for runtime monitoring approaches

We present a comparison framework for runtime monitoring approaches.We demonstrate the usefulness of our framework.We discuss perspectives for researchers working in the area of monitoring. The full behavior of complex software systems often only emerges during operation. They thus need to be monitored at run time to check that they adhere to their requirements. Diverse runtime monitoring approaches have been developed in various domains and for different purposes. Their sheer number and heterogeneity, however, make it hard to find the right approach for a specific application or purpose. The aim of our research therefore was to develop a comparison framework for runtime monitoring approaches. Our framework is based on an analysis of the literature and existing taxonomies for monitoring languages and patterns. We use examples from existing monitoring approaches to explain the framework. We demonstrate its usefulness by applying it to 32 existing approaches and by comparing 3 selected approaches in the light of different monitoring scenarios. We also discuss perspectives for researchers.

[1]  Neil A. M. Maiden.  Monitoring Our Requirements , 2013, IEEE Software.

[2]  Paul Grünbacher,et al.  ReMinds : A flexible runtime monitoring framework for systems of systems , 2016, J. Syst. Softw..

[3]  Luciano Baresi,et al.  A Comprehensive Resource Management Solution for Web-based Systems , 2014, ICAC.

[4]  Luciano Baresi,et al.  Event-Based Multi-level Service Monitoring , 2013, 2013 IEEE 20th International Conference on Web Services.

[5]  Manas Ranjan Patra,et al.  An event based, non-intrusive monitoring framework for Web Service Based Systems , 2010, 2010 International Conference on Computer Information Systems and Industrial Management Applications (CISIM).

[6]  Jan Recker,et al.  Real-time risk monitoring in business processes: A sensor-based approach , 2013, J. Syst. Softw..

[7]  Carlo Ghezzi,et al.  Runtime monitoring of component changes with Spy@Runtime , 2012, 2012 34th International Conference on Software Engineering (ICSE).

[8]  Paola Mello,et al.  Monitoring business constraints with the event calculus , 2013, ACM Trans. Intell. Syst. Technol..

[9]  Elke A. Rundensteiner,et al.  The HIT Model: Workflow-Aware Event Stream Monitoring , 2013, Trans. Large Scale Data Knowl. Centered Syst..

[10]  Khaled Mahbub,et al.  MORPED: Monitor rules for proactive error detection based on run-time and historical data , 2014, The Fifth International Conference on the Applications of Digital Information and Web Technologies (ICADIWT 2014).

[11]  Martin Leucker,et al.  Monitoring of Real-Time Properties , 2006, FSTTCS.

[12]  Adrian Paschke.  RBSLA A declarative Rule-based Service Level Agreement Language based on RuleML , 2005, International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06).

[13]  Richard N. Taylor,et al.  A Classification and Comparison Framework for Software Architecture Description Languages , 2000, IEEE Trans. Software Eng..

[14]  Bernd Finkbeiner,et al.  Monitoring Parametric Temporal Logic , 2014, VMCAI.

[15]  Krzysztof Czarnecki,et al.  Cool features and tough decisions: a comparison of variability modeling approaches , 2012, VaMoS.

[16]  M. Mansouri-Samani,et al.  Monitoring distributed systems , 1993, IEEE Network.

[17]  Hendra Gunadi,et al.  Efficient Runtime Monitoring with Metric Temporal Logic: A Case Study in the Android Operating System , 2014, FM.

[18]  Wilhelm Hasselbring,et al.  Kieker: a framework for application performance monitoring and dynamic software analysis , 2012, ICPE '12.

[19]  Kurt Rothermel,et al.  Supporting Strong Reliability for Distributed Complex Event Processing Systems , 2011, 2011 IEEE International Conference on High Performance Computing and Communications.

[20]  Claude Godart,et al.  An Optimized Derivation of Event Queries to Monitor Choreography Violations , 2012, ICSOC.

[21]  Wilhelm Hasselbring,et al.  A Self-adaptive Monitoring Framework for Component-Based Software Systems , 2011, ECSA.

[22]  George Spanoudakis,et al.  Requirements monitoring for service-based systems: towards a framework based on event calculus , 2004 .

[23]  Andres J. Ramirez,et al.  Automatic derivation of utility functions for monitoring software requirements , 2011, MODELS'11.

[24]  Bernhard Ganter,et al.  Formal Concept Analysis: Mathematical Foundations , 1998 .

[25]  William N. Robinson.  A requirements monitoring framework for enterprise systems , 2005, Requirements Engineering.

[26]  Clinton Jeffery,et al.  Towards Fully Automatic Execution Monitoring , 2002, RISSEF.

[27]  Ann Q. Gates,et al.  A taxonomy and catalog of runtime software-fault monitoring tools , 2004, IEEE Transactions on Software Engineering.

[28]  Mahesh Viswanathan,et al.  Foundations for the Run-Time Monitoring of Reactive Systems - Fundamentals of the MaC Language , 2004, ICTAC.

[29]  Dilian Gurov,et al.  Provably Correct Runtime Monitoring , 2008, FM.

[30]  Alexander Egyed,et al.  Developing a DSL-Based Approach for Event-Based Monitoring of Systems of Systems: Experiences and Lessons Learned (E) , 2015, 2015 30th IEEE/ACM International Conference on Automated Software Engineering (ASE).

[31]  Mahesh Viswanathan,et al.  Java-MaC: a Run-time Assurance Tool for Java Programs , 2001, RV@CAV.

[32]  Carlo Ghezzi,et al.  Self-adaptive software needs quantitative verification at runtime , 2012, CACM.

[33]  Paul Grünbacher,et al.  Requirements monitoring frameworks: A systematic review , 2016, Inf. Softw. Technol..

[34]  Mari Matinlassi,et al.  Comparison of software product line architecture design methods: COPA, FAST, FORM, KobrA and QADA , 2004, Proceedings. 26th International Conference on Software Engineering.

[35]  Henry Muccini,et al.  Monitoring Architectural Properties in Dynamic Component-Based Systems , 2007, CBSE.

[36]  Jocelyn Simmonds,et al.  Runtime Monitoring of Web Service Conversations , 2007, IEEE Transactions on Services Computing.

[37]  Schahram Dustdar,et al.  Model-aware Monitoring of SOAs for Compliance , 2011 .

[38]  D. Luckham.  Event Processing for Business: Organizing the Real-Time Enterprise , 2011 .

[39]  George S. Avrunin,et al.  Patterns in property specifications for finite-state verification , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[40]  Schahram Dustdar,et al.  Specification and Deployment of Distributed Monitoring and Adaptation Infrastructures , 2012, ICSOC Workshops.

[41]  Paul Grünbacher,et al.  A qualitative study on user guidance capabilities in product configuration tools , 2012, 2012 Proceedings of the 27th IEEE/ACM International Conference on Automated Software Engineering.

[42]  Mikhail Auguston,et al.  Modeling and Verifying Business Processes with Monterey Phoenix , 2015 .