Automatic Invariant Selection for Online Anomaly Detection

Invariants are stable relationships among system metrics expected to hold during normal operating conditions. The violation of such relationships can be used to detect anomalies at runtime. However, this approach does not scale to large systems, as the number of invariants quickly grows with the number of considered metrics. The resulting “background noise” for the invariant-based detection system hinders its effectiveness. In this paper we propose a general and automatic approach for identifying a subset of mined invariants that properly model system runtime behavior with a reduced amount of background noise. This translates into better overall performance (i.e., less false positives).

[1]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .

[2]  Liming Zhu,et al.  POD-Diagnosis: Error Diagnosis of Sporadic Operations on Cloud Applications , 2014, 2014 44th Annual IEEE/IFIP International Conference on Dependable Systems and Networks.

[3]  Haifeng Chen,et al.  Discovering likely invariants of distributed transaction systems for autonomic system management , 2006, 2006 IEEE International Conference on Autonomic Computing.

[4]  Domenico Cotroneo,et al.  Workload Characterization for Software Aging Analysis , 2011, 2011 IEEE 22nd International Symposium on Software Reliability Engineering.

[5]  Stefano Russo,et al.  Mining Invariants from SaaS Application Logs (Practical Experience Report) , 2014, 2014 Tenth European Dependable Computing Conference.

[6]  Sarita V. Adve,et al.  Using likely program invariants to detect hardware errors , 2008, 2008 IEEE International Conference on Dependable Systems and Networks With FTCS and DCC (DSN).

[7]  Ray Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[8]  William G. Griswold,et al.  Dynamically discovering likely program invariants to support program evolution , 1999, Proceedings of the 1999 International Conference on Software Engineering (IEEE Cat. No.99CB37002).

[9]  Carl E. Landwehr,et al.  Basic concepts and taxonomy of dependable and secure computing , 2004, IEEE Transactions on Dependable and Secure Computing.

[10]  Stefano Russo,et al.  Using Invariants for Anomaly Detection: The Case Study of a SaaS Application , 2014, 2014 IEEE International Symposium on Software Reliability Engineering Workshops.

[11]  Haifeng Chen,et al.  Fault detection and localization in distributed systems using invariant relationships , 2013, 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[12]  Tianyin Xu,et al.  EnCore: exploiting system environment and correlation information for misconfiguration detection , 2014, ASPLOS.

[13]  Qiang Fu,et al.  Mining Invariants from Console Logs for System Problem Detection , 2010, USENIX Annual Technical Conference.