A taxonomy and catalog of runtime software-fault monitoring tools

A goal of runtime software-fault monitoring is to observe software behavior to determine whether it complies with its intended behavior. Monitoring allows one to analyze and recover from detected faults, providing additional defense against catastrophic failure. Although runtime monitoring has been in use for over 30 years, there is renewed interest in its application to fault detection and recovery, largely because of the increasing complexity and ubiquitous nature of software systems. We present taxonomy that developers and researchers can use to analyze and differentiate recent developments in runtime software fault-monitoring approaches. The taxonomy categorizes the various runtime monitoring research by classifying the elements that are considered essential for building a monitoring system, i.e., the specification language used to define properties; the monitoring mechanism that oversees the program's execution; and the event handler that captures and communicates monitoring results. After describing the taxonomy, the paper presents the classification of the software-fault monitoring systems described in the literature.

[1]  J. Nievergelt,et al.  Special Feature: Monitoring Program Execution: A Survey , 1981, Computer.

[2]  Philippe Schnoebelen,et al.  Systems and Software Verification , 2001, Springer Berlin Heidelberg.

[3]  Ann Q. Gates,et al.  Towards the design of a snoopy coprocessor for dynamic software-fault detection , 1999, 1999 IEEE International Performance, Computing and Communications Conference (Cat. No.99CH36305).

[4]  Mohamed G. Gouda,et al.  Implementation of the sentry system , 1995, Softw. Pract. Exp..

[5]  Grigore Rosu,et al.  Monitoring Java Programs with Java PathExplorer , 2001, RV@CAV.

[6]  Nelly M Delgado.  A taxonomy of dynamic software-fault monitoring tools , 2001 .

[7]  William N. Robinson,et al.  Monitoring Web service requirements , 2003, Proceedings. 11th IEEE International Requirements Engineering Conference, 2003..

[8]  Grigore Rosu,et al.  Towards Monitoring-Oriented Programming: A Paradigm Combining Specification and Implementation , 2003, RV@CAV.

[9]  Mahesh Viswanathan,et al.  Java-MaC: a Run-time Assurance Tool for Java Programs , 2001, RV@CAV.

[10]  Bowen Alpern,et al.  Verifying temporal properties without temporal logic , 1989, TOPL.

[11]  Philippe Schnoebelen,et al.  Systems and Software Verification, Model-Checking Techniques and Tools , 2001 .

[12]  Bin Luo,et al.  A framework for dynamic program analyzers , 1993, OOPSLA '93.

[13]  Mahesh Viswanathan,et al.  Steering of real-time systems based on monitoring and checking , 1999, Proceedings. Fifth International Workshop on Object-Oriented Real-Time Dependable Systems.

[14]  Doron Drusinsky,et al.  The Temporal Rover and the ATG Rover , 2000, SPIN.

[15]  Hanêne Ben-Abdallah,et al.  MaC: A Framework for Run-Time Correctness Assurance of Real-Time Systems , 1998 .

[16]  Beth A. Schroeder.  On-Line Monitoring: A Tutorial , 1995, Computer.

[17]  David Lorge Parnas,et al.  Requirements-based monitors for real-time systems , 2000, ISSTA '00.

[18]  G. Ro.  Synthesizing Dynamic Programming Algorithms from Linear Temporal Logic Formulae , 2001 .

[19]  Ann Q. Gates,et al.  FasTLInC: a constraint-based tracing approach , 2002, J. Syst. Softw..

[20]  G. Rosu,et al.  Synthesizing Dynamic Programming Algorithms fromLinear Temporal Logic Formulae , 2001 .

[21]  David Lorge Parnas,et al.  Deriving real-time monitors from system requirements documentation , 2000 .

[22]  David C. Luckham,et al.  An Overview of Anna, a Specification Language for Ada , 1985, IEEE Software.

[23]  Lori A. Clarke,et al.  FLAVER: A Finite State Verification Technique for Software Systems TITLE2: , 2001 .

[24]  Spencer Rugaber,et al.  DYNAMO: Dynamic Assembly From Models , 2004 .

[25]  Lori A. Clarke,et al.  FLAVERS: A finite state verification technique for software systems , 2002, IBM Syst. J..

[26]  Sriram Sankar,et al.  Two-dimensional pinpointing: debugging with formal specifications , 1991, IEEE Software.

[27]  Karsten Schwan,et al.  Falcon: on-line monitoring and steering of large-scale parallel programs , 1995, Proceedings Frontiers '95. The Fifth Symposium on the Frontiers of Massively Parallel Computation.

[28]  Ann Q. Gates,et al.  DynaMICs: Comprehensive Support for Run-Time Monitoring , 2001, RV@CAV.

[29]  Jeffrey J. P. Tsai,et al.  A Noninterference Monitoring and Replay Mechanism for Real-Time Software Testing and Debugging , 1990, IEEE Trans. Software Eng..

[30]  Grigore Rosu,et al.  Java PathExplorer: A Runtime Verification Tool , 2001 .

[31]  Michael D. Ernst,et al.  An overview of JML tools and applications , 2003, International Journal on Software Tools for Technology Transfer.

[32]  Surrendra Dudani,et al.  Dynamic Assertions Using TXP , 2001, Electron. Notes Theor. Comput. Sci..

[33]  Spencer Rugaber.  Position Paper : The Tradeoff Between Dependability and Efficiency in Embedded Systems , .

[34]  Bertrand Meyer,et al.  Applying 'design by contract' , 1992, Computer.

[35]  Rok Sosic,et al.  Dynascope: a tool for program directing , 1992, PLDI '92.

[36]  M. Clavel,et al.  Principles of Maude , 1996, WRLA.

[37]  Yoonsik Cheon,et al.  A Runtime Assertion Checker for the Java Modeling Language (JML) , 2003, ICSE 2003.

[38]  Heonshik Shin,et al.  Experimental Assessment of the Period Calibration Method: A Case Study , 1999, Real-Time Systems.

[39]  Clinton L. Jeffery,et al.  A lightweight architecture for program execution monitoring , 1998, PASTE '98.

[40]  Ann Q. Gates,et al.  An Integrated Development of a Dynamic Software-Fault Monitoring System , 2000, Trans. SDPS.

[41]  Grigore Rosu,et al.  Monitoring programs using rewriting , 2001, Proceedings 16th Annual International Conference on Automated Software Engineering (ASE 2001).

[42]  Hanêne Ben-Abdallah,et al.  Formally specified monitoring of temporal properties , 1999, Proceedings of 11th Euromicro Conference on Real-Time Systems. Euromicro RTS'99.

[43]  Y. Liao,et al.  A Specificational Approach to High Level Program Monitoring and Measuring , 1992, IEEE Trans. Software Eng..

[44]  Heike Wehrheim,et al.  Jass - Java with Assertions , 2001, RV@CAV.

[45]  Mahesh Viswanathan,et al.  Runtime Assurance Based On Formal Specifications , 1999, PDPTA.

[46]  Jerry Avorn.  Technology , 1929, Nature.

[47]  Karsten Schwan,et al.  An annotated bibliography of interactive program steering , 1994, SIGP.

[48]  Jaideep Srivastava,et al.  Adaptive disk scheduling algorithms for video servers , 1999, Proceedings of the 1999 International Conference on Parallel Processing.

[49]  Klaus Havelund,et al.  Using Runtime Analysis to Guide Model Checking of Java Programs , 2013, SPIN.

[50]  C L Holt,et al.  The Memorial University of Newfoundland. , 1970, JAMA.

[51]  Michael D. Ernst,et al.  Static verification of dynamically detected program invariants: Integrating Daikon and ESC/Java , 2001, RV@CAV.

[52]  Greg Eisenhauer.  The ECho Event Delivery System , 1999 .

[53]  Hanêne Ben-Abdallah,et al.  A Monitoring and Checking Framework for Run-time Correctness Assurance , 1998 .

[54]  Jeffrey J. P. Tsai,et al.  Distributed Real-Time Systems: Monitoring, Visualization, Debugging, and Analysis , 1996 .

[55]  Michael D. Ernst,et al.  An overview of JML tools and applications , 2003, Electron. Notes Theor. Comput. Sci..

[56]  Dieter Haban,et al.  A Hybrid Monitor for Behavior and Performance Analysis of Distributed Systems , 1990, IEEE Trans. Software Eng..

[57]  William N. Robinson,et al.  Monitoring software requirements using instrumented code , 2002, Proceedings of the 35th Annual Hawaii International Conference on System Sciences.

[58]  Thomas Colcombet,et al.  Enforcing trace properties by program transformation , 2000, POPL '00.

[59]  David S. Rosenblum.  A Practical Approach to Programming With Assertions , 1995, IEEE Trans. Software Eng..

[60]  Gregory Tassey,et al.  Prepared for what , 2007 .

[61]  Music Musi.  Georgia Institute of Technology , 2002 .

[62]  George S. Avrunin,et al.  PROPEL: an approach supporting property elucidation , 2002, ICSE '02.

[63]  A. Avizienis,et al.  Dependable computing: From concepts to design diversity , 1986, Proceedings of the IEEE.

[64]  Clinton L. Jeffery,et al.  A configurable automatic instrumentation tool for ANSI C , 1998, Proceedings 13th IEEE International Conference on Automated Software Engineering (Cat. No.98EX239).

[65]  Guy Juanole,et al.  Observer-A Concept for Formal On-Line Validation of Distributed Systems , 1994, IEEE Trans. Software Eng..

[66]  Bernhard Plattner,et al.  Monitoring Program Execution: A Survey. , 1981 .

[67]  Sriram Sankar,et al.  Concurrent runtime monitoring of formally specified programs , 1993, Computer.

[68]  James C. Witcher,et al.  GEOTHERMAL ENERGY AT NEW MEXICO STATE UNIVERSITY IN LAS CRUCES , 2002 .

[69]  Gerard J. Holzmann,et al.  The Model Checker SPIN , 1997, IEEE Trans. Software Eng..

[70]  T. Longstaff,et al.  Quality Attributes , 1995 .

[71]  Clinton L. Jeffery,et al.  Program monitoring and visualization - a exploratory approach , 2011 .

[72]  Guangtian Liu,et al.  Efficient Run-time Monitoring Of Timing Constraints , 1997, Proceedings Third IEEE Real-Time Technology and Applications Symposium.

[73]  Gerard J. Holzmann,et al.  The SPIN Model Checker , 2003 .