A Lone Wolf No More: Supporting Network Intrusion Detection with Real-Time Intelligence

For network intrusion detection systems it is becoming increasingly difficult to reliably report today's complex attacks without having external context at hand. Unfortunately, however, today's IDS cannot readily integrate intelligence, such as dynamic blacklists, into their operation. In this work, we introduce a fundamentally new capability into IDS processing that vastly broadens a system's view beyond what is visible directly on the wire. We present a novel Input Framework that integrates external information in real-time into the IDS decision process, independent of specific types of data, sources, and desired analyses. We implement our design on top of an open-source IDS, and we report initial experiences from real-world deployment in a large-scale network environment. To ensure that our system meets operational constraints, we further evaluate its technical characteristics in terms of the intelligence volume it can handle under realistic workloads, and the latency with which real-time updates become available to the IDS analysis engine. The implementation is freely available as open-source software.

[1]  Alfonso Valdes,et al.  Probabilistic Alert Correlation , 2001, Recent Advances in Intrusion Detection.

[2]  Balachander Krishnamurthy,et al.  Collaborating against common enemies , 2005, IMC '05.

[3]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[4]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[5]  Hervé Debar,et al.  Aggregation and Correlation of Intrusion-Detection Alerts , 2001, Recent Advances in Intrusion Detection.

[6]  Hari Balakrishnan,et al.  Fast portscan detection using sequential hypothesis testing , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[7]  Ravishankar K. Iyer,et al.  Analysis of security data from a large computing organization , 2011, 2011 IEEE/IFIP 41st International Conference on Dependable Systems & Networks (DSN).

[8]  Farnam Jahanian,et al.  Improving Spam Blacklisting Through Dynamic Thresholding and Speculative Aggregation , 2010, NDSS.

[9]  Jason Lee,et al.  The NIDS Cluster: Scalable, Stateful Network Intrusion Detection on Commodity Hardware , 2007, RAID.

[10]  Vern Paxson,et al.  Exploiting independent state for network intrusion detection , 2005, 21st Annual Computer Security Applications Conference (ACSAC'05).

[11]  Anja Feldmann,et al.  Operational experiences with high-volume network intrusion detection , 2004, CCS '04.

[12]  Nick Feamster,et al.  Building a Dynamic Reputation System for DNS , 2010, USENIX Security Symposium.

[13]  Vinod Yegneswaran,et al.  BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation , 2007, USENIX Security Symposium.