Detecting Intrusions through System Call Sequence and Argument Analysis

We describe an unsupervised host-based intrusion detection system based on system call arguments and sequences. We define a set of anomaly detection models for the individual parameters of the call. We then describe a clustering process that helps to better fit models to system call arguments and creates interrelations among different arguments of a system call. Finally, we add a behavioral Markov model in order to capture time correlations and abnormal behaviors. The whole system needs no prior knowledge input; it has a good signal-to-noise ratio, and it is also able to correctly contextualize alarms, giving the user more information to understand whether a true or false positive happened, and to detect global variations over the entire execution flow, as opposed to punctual ones over individual instances.

[1]  Anup K. Ghosh,et al.  Detecting anomalous and unknown intrusions against programs , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[2]  Stephanie Forrest,et al.  Automated response using system-call delays , 2000 .

[3]  Stephanie Forrest,et al.  A sense of self for Unix processes , 1996, Proceedings 1996 IEEE Symposium on Security and Privacy.

[4]  David A. Wagner,et al.  Mimicry attacks on host-based intrusion detection systems , 2002, CCS '02.

[5]  Mark Burgess,et al.  Measuring system normality , 2002, TOCS.

[6]  Fabio A. González,et al.  A comparative analysis of artificial immune network models , 2005, GECCO '05.

[7]  David A. Wagner,et al.  Intrusion detection via static analysis , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[8]  Alan S. Perelson,et al.  Self-nonself discrimination in a computer , 1994, Proceedings of 1994 IEEE Computer Society Symposium on Research in Security and Privacy.

[9]  Raman K. Mehra,et al.  Detection and classification of intrusions and faults using sequences of system calls , 2001, SGMD.

[10]  Barak A. Pearlmutter,et al.  Detecting intrusions using system calls: alternative data models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[11]  Wiebe R. Pestman,et al.  Mathematical Statistics: An Introduction , 1998 .

[12]  R. Sekar,et al.  A fast automaton-based method for detecting anomalous program behaviors , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[13]  Christopher Krügel,et al.  Anomalous system call detection , 2006, TSEC.

[14]  Somesh Jha,et al.  Markov chains, classifiers, and intrusion detection , 2001, Proceedings. 14th IEEE Computer Security Foundations Workshop, 2001..

[15]  Richard Lippmann,et al.  The 1999 DARPA off-line intrusion detection evaluation , 2000, Comput. Networks.

[16]  Giuseppe Serazzi,et al.  Unsupervised learning algorithms for intrusion detection , 2008, NOMS 2008 - 2008 IEEE Network Operations and Management Symposium.

[17]  Yuxin Ding,et al.  Host-based intrusion detection using dynamic and static behavioral models , 2003, Pattern Recognit..

[18]  Karl N. Levitt,et al.  Automated detection of vulnerabilities in privileged programs by execution monitoring , 1994, Tenth Annual Computer Security Applications Conference.

[19]  Hervé Debar,et al.  A neural network component for an intrusion detection system , 1992, Proceedings 1992 IEEE Computer Society Symposium on Research in Security and Privacy.

[20]  Stephanie Forrest,et al.  Intrusion Detection Using Sequences of System Calls , 1998, J. Comput. Secur..

[21]  Lawrence R. Rabiner,et al.  A tutorial on hidden Markov models and selected applications in speech recognition , 1989, Proc. IEEE.

[22]  Salvatore J. Stolfo,et al.  Data Mining Approaches for Intrusion Detection , 1998, USENIX Security Symposium.

[23]  Salvatore J. Stolfo,et al.  Learning Rules from System Call Arguments and Sequences for Anomaly 20 Detection , 2003 .

[24]  LewisLundy,et al.  Detection and classification of intrusions and faults using sequences of system calls , 2001 .

[25]  Marc Dacier,et al.  Intrusion detection , 1999, Comput. Networks.

[26]  Philip K. Chan,et al.  Learning Patterns from Unix Process Execution Traces for Intrusion Detection , 1997 .

[27]  William W. Cohen.  Fast Effective Rule Induction , 1995, ICML.

[28]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.

[29]  Dorothy E. Denning,et al.  An Intrusion-Detection Model , 1987, IEEE Transactions on Software Engineering.

[30]  Dirk Ourston,et al.  Applications of hidden Markov models to detecting multi-stage network attacks , 2003, 36th Annual Hawaii International Conference on System Sciences, 2003. Proceedings of the.

[31]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[32]  Sin Yeung Lee,et al.  Learning Fingerprints for a Database Intrusion Detection System , 2002, ESORICS.

[33]  Wei Fan,et al.  Mining system audit data: opportunities and challenges , 2001, SGMD.

[34]  Stephanie Forrest,et al.  Infect Recognize Destroy , 1996 .

[35]  Somesh Jha,et al.  Environment-Sensitive Intrusion Detection , 2005, RAID.

[36]  Andreas Stolcke,et al.  Hidden Markov Model} Induction by Bayesian Model Merging , 1992, NIPS.

[37]  Robert H. Halstead,et al.  Matrix Computations , 2011, Encyclopedia of Parallel Computing.

[38]  Christopher Krügel,et al.  On the Detection of Anomalous System Call Arguments , 2003, ESORICS.

[39]  Andreas Stolcke,et al.  Inducing Probabilistic Grammars by Bayesian Model Merging , 1994, ICGI.

[40]  Qiang Chen,et al.  An anomaly detection technique based on a chi‐square statistic for detecting intrusions into information systems , 2001 .

[41]  Stefano Zanero.  Behavioral Intrusion Detection , 2004, ISCIS.

[42]  R. Sekar,et al.  Dataflow anomaly detection , 2006, 2006 IEEE Symposium on Security and Privacy (S&P'06).

[43]  Anup Ghosh,et al.  Simple, state-based approaches to program-based anomaly detection , 2002, TSEC.