An Efficient and Scalable Intrusion Detection System on Logs of Distributed Applications

Although security issues are now addressed during the development process of distributed applications, an attack may still affect the provided services or allow access to confidential data. To detect intrusions, we consider an anomaly detection mechanism which relies on a model of the monitored application's normal behavior. During a model construction phase, the application is run multiple times to observe some of its correct behaviors. Each gathered trace enables the identification of significant events and their causality relationships, without requiring the existence of a global clock. The constructed model is dual: an automaton plus a list of likely invariants. The redundancy between the two sub-models decreases when generalization techniques are applied on the automaton. Solutions already proposed suffer from scalability issues. In particular, the time needed to build the model is important and its size impacts the duration of the detection phase. The proposed solutions address these problems, while keeping a good accuracy during the detection phase, in terms of false positive and false negative rates. To evaluate them, a real distributed application and several attacks against the service are considered.

[1]  Vijay K. Garg,et al.  Lattice Completion Algorithms for Distributed Computations , 2012, OPODIS.

[2]  Yuriy Brun,et al.  Inferring models of concurrent systems from logs of their behavior with CSight , 2014, ICSE.

[3]  Ivan Beschastnikh,et al.  Inferring and Asserting Distributed System Invariants , 2018, 2018 IEEE/ACM 40th International Conference on Software Engineering (ICSE).

[4]  Madhavan Mukund,et al.  Synthesizing Distributed Finite-State Systems from MSCs , 2000, CONCUR.

[5]  Alexander Reinefeld,et al.  XtreemFS – a File System for the Cloud , 2013 .

[6]  Carlos Maziero,et al.  A Fuzzy Model for the Composition of Intrusion Detectors , 2008, SEC.

[7]  John L. Pfaltz Using Concept Lattices to Uncover Causal Dependencies in Software , 2006, ICFCA.

[8]  Eric Totel,et al.  Inferring a Distributed Application Behavior Model for Anomaly Based Intrusion Detection , 2016, 2016 12th European Dependable Computing Conference (EDCC).

[9]  Yuriy Brun,et al.  Mining temporal invariants from partially ordered logs , 2011, OPSR.

[10]  Jerome A. Feldman,et al.  On the Synthesis of Finite-State Machines from Samples of Their Behavior , 1972, IEEE Transactions on Computers.

[11]  Leslie Lamport,et al.  Time, clocks, and the ordering of events in a distributed system , 1978, CACM.

[12]  Eric Totel,et al.  A Scalable and Efficient Correlation Engine to Detect Multi-Step Attacks in Distributed Systems , 2018, 2018 IEEE 37th Symposium on Reliable Distributed Systems (SRDS).

[13]  David Lo,et al.  Learning extended FSA from software: An empirical assessment , 2012, J. Syst. Softw..

[14]  Marco Canini,et al.  Finding Almost-Invariants in Distributed Systems , 2011, 2011 IEEE 30th International Symposium on Reliable Distributed Systems.

[15]  Leonardo Mariani,et al.  Inferring state-based behavior models , 2006, WODA '06.