Intrusion Detection using Continuous Time Bayesian Networks

Intrusion detection systems (IDSs) fall into two high-level categories: network-based systems (NIDS) that monitor network behaviors, and host-based systems (HIDS) that monitor system calls. In this work, we present a general technique for both systems. We use anomaly detection, which identifies patterns not conforming to a historic norm. In both types of systems, the rates of change vary dramatically over time (due to burstiness) and over components (due to service difference). To efficiently model such systems, we use continuous time Bayesian networks (CTBNs) and avoid specifying a fixed update interval common to discrete-time models. We build generative models from the normal training data, and abnormal behaviors are flagged based on their likelihood under this norm. For NIDS, we construct a hierarchical CTBN model for the network packet traces and use Rao-Blackwellized particle filtering to learn the parameters. We illustrate the power of our method through experiments on detecting real worms and identifying hosts on two publicly available network traces, the MAWI dataset and the LBNL dataset. For HIDS, we develop a novel learning method to deal with the finite resolution of system log file time stamps, without losing the benefits of our continuous time model. We demonstrate the method by detecting intrusions in the DARPA 1998 BSM dataset.

[1]  Nir Friedman,et al.  Mean Field Variational Approximation for Continuous-Time Bayesian Networks , 2009, J. Mach. Learn. Res..

[2]  Eric Horvitz,et al.  Continuous Time Bayesian Networks for Inferring Users’ Presence and Activities with Extensions for Modeling and Evaluation , 2003 .

[3]  Nir Friedman,et al.  Gibbs Sampling in Factorized Continuous-Time Markov Processes , 2008, UAI.

[4]  Jing Xu,et al.  Continuous Time Bayesian Networks for Host Level Network Intrusion Detection , 2008, ECML/PKDD.

[5]  Vasant Honavar,et al.  Learning Classifiers for Misuse Detection Using a Bag of System Calls Representation , 2005, ISI.

[6]  Nir Friedman,et al.  Continuous-Time Belief Propagation , 2010, ICML.

[7]  Mark Crovella,et al.  Mining anomalies using traffic feature distributions , 2005, SIGCOMM '05.

[8]  Konstantina Papagiannaki,et al.  Flow classification by histograms: or how to go on safari in the internet , 2004, SIGMETRICS '04/Performance '04.

[9]  Daphne Koller,et al.  Continuous Time Bayesian Networks , 2012, UAI.

[10]  Qiang Chen,et al.  Multivariate Statistical Analysis of Audit Trails for Host-Based Intrusion Detection , 2002, IEEE Trans. Computers.

[11]  Suchi Saria,et al.  Reasoning at the Right Time Granularity , 2007, UAI.

[12]  Kavé Salamatian,et al.  Combining filtering and statistical methods for anomaly detection , 2005, IMC '05.

[13]  Yu Fan,et al.  Sampling for Approximate Inference in Continuous Time Bayesian Networks , 2008, ISAIM.

[14]  Richard Mortier,et al.  CT-NOR: Representing and Reasoning About Events in Continuous Time , 2008, UAI.

[15]  Eleazar Eskin,et al.  Anomaly Detection over Noisy Data using Learned Probability Distributions , 2000, ICML.

[16]  Christopher Krügel,et al.  Bayesian event classification for intrusion detection , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[17]  Barak A. Pearlmutter,et al.  Detecting intrusions using system calls: alternative data models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[18]  Stephanie Forrest,et al.  Intrusion Detection Using Sequences of System Calls , 1998, J. Comput. Secur..

[19]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[20]  Avi Pfeffer,et al.  Continuous Time Particle Filtering , 2005, IJCAI.

[21]  Wenke Lee,et al.  Attack plan recognition and prediction using causal networks , 2004, 20th Annual Computer Security Applications Conference.

[22]  Kensuke Fukuda,et al.  Extracting hidden anomalies using sketch and non Gaussian multiresolution statistical detection procedures , 2007, LSAD '07.

[23]  John Mark Agosta,et al.  An adaptive anomaly detector for worm detection , 2007 .

[24]  Sushil Jajodia,et al.  Applications of Data Mining in Computer Security , 2002, Advances in Information Security.

[25]  Yu Fan,et al.  Learning Continuous-Time Social Network Dynamics , 2009, UAI.

[26]  ByungRae Cha.  Host anomaly detection performance analysis based on system call of neuro-fuzzy using Soundex algorithm and N-gram technique , 2005, 2005 Systems Communications (ICW'05, ICHSN'05, ICMCS'05, SENET'05).

[27]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[28]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[29]  V. Rao Vemuri,et al.  Robust Support Vector Machines for Anomaly Detection in Computer Security , 2003, ICMLA.

[30]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[31]  Daphne Koller,et al.  Expectation Propagation for Continuous Time Bayesian Networks , 2005, UAI.

[32]  Michalis Faloutsos,et al.  BLINC: multilevel traffic classification in the dark , 2005, SIGCOMM '05.

[33]  William H. Press,et al.  Numerical recipes in C , 2002 .

[34]  Jaideep Srivastava,et al.  A Comparative Study of Anomaly Detection Schemes in Network Intrusion Detection , 2003, SDM.

[35]  Eleazar Eskin,et al.  A GEOMETRIC FRAMEWORK FOR UNSUPERVISED ANOMALY DETECTION: DETECTING INTRUSIONS IN UNLABELED DATA , 2002 .

[36]  Konrad Rieck,et al.  Language models for detection of unknown attacks in network traffic , 2006, Journal in Computer Virology.

[37]  Andrew W. Moore,et al.  Internet traffic classification using bayesian analysis techniques , 2005, SIGMETRICS '05.

[38]  Nando de Freitas,et al.  Rao-Blackwellised Particle Filtering for Dynamic Bayesian Networks , 2000, UAI.

[39]  Yuxin Ding,et al.  User Profiling for Intrusion Detection Using Dynamic and Static Behavioral Models , 2002, PAKDD.

[40]  Daphne Koller,et al.  Learning Continuous Time Bayesian Networks , 2002, UAI.

[41]  Zhi-Li Zhang,et al.  Profiling internet backbone traffic: behavior models and applications , 2005, SIGCOMM '05.

[42]  Philip K. Chan,et al.  Learning Useful System Call Attributes for Anomaly Detection , 2005, FLAIRS Conference.

[43]  Daphne Koller,et al.  Expectation Maximization and Complex Duration Distributions for Continuous Time Bayesian Networks , 2005, UAI.

[44]  Dit-Yan Yeung,et al.  Parzen-window network intrusion detectors , 2002, Object recognition supported by user interaction for service robots.

[45]  Michael D. Smith,et al.  Host-based detection of worms through peer-to-peer cooperation , 2005, WORM '05.