Accurate and Automated System Call Policy-Based Intrusion Prevention

One way to prevent control hijacking attack is to compare a network application's run-time system calls with a pre-defined normal system call behavior model, and raise an alert upon detecting a mismatch. This paper describes a system called PAID, which can automatically derive an accurate system call pattern from the source code of an application, and use it to detect any anomalous behavior at run time with minimal overhead. Because each application's system call pattern is directly derived from its source code, PAID never raises false positive alarms. Moreover, its false negative rate is very close to zero because PAID uses the sequence of return addresses on the user/kernel stack to uniquely identify each system call instance. Experiments on a fully operational PAID prototype show that PAID can indeed stop all known control hijacking attacks. The run-time latency and throughput penalty of PAID are under 13.02% and 11.52%, respectively, when it is tested against a set of production-mode network applications

[1]  Weibo Gong,et al.  Anomaly detection using call stack information , 2003, 2003 Symposium on Security and Privacy, 2003..

[2]  Tzi-cker Chiueh,et al.  Automatic extraction of accurate application-specific sandboxing policy , 2005, MILCOM 2005 - 2005 IEEE Military Communications Conference.

[3]  Barak A. Pearlmutter,et al.  Detecting intrusions using system calls: alternative data models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[4]  Somesh Jha,et al.  Efficient Context-Sensitive Intrusion Detection , 2004, NDSS.

[5]  Stephanie Forrest,et al.  Intrusion Detection Using Sequences of System Calls , 1998, J. Comput. Secur..

[6]  Somesh Jha,et al.  Detecting Manipulated Remote Call Streams , 2002, USENIX Security Symposium.

[7]  David A. Wagner,et al.  Mimicry attacks on host-based intrusion detection systems , 2002, CCS '02.

[8]  Mark N. Wegman,et al.  Constant propagation with conditional branches , 1985, POPL.

[9]  David A. Wagner,et al.  Intrusion detection via static analysis , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[10]  Somesh Jha,et al.  Formalizing sensitivity in static analysis for intrusion detection , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[11]  Linda Torczon,et al.  Interprocedural constant propagation: a study of jump function implementation , 1993, PLDI '93.

[12]  R. Sekar,et al.  A fast automaton-based method for detecting anomalous program behaviors , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.