MIDeA: a multi-parallel intrusion detection architecture

Network intrusion detection systems are faced with the challenge of identifying diverse attacks, in extremely high speed networks. For this reason, they must operate at multi-Gigabit speeds, while performing highly-complex per-packet and per-flow data processing. In this paper, we present a multi-parallel intrusion detection architecture tailored for high speed networks. To cope with the increased processing throughput requirements, our system parallelizes network traffic processing and analysis at three levels, using multi-queue NICs, multiple CPUs, and multiple GPUs. The proposed design avoids locking, optimizes data transfers between the different processing units, and speeds up data processing by mapping different operations to the processing units where they are best suited. Our experimental evaluation shows that our prototype implementation based on commodity off-the-shelf equipment can reach processing speeds of up to 5.2 Gbit/s with zero packet loss when analyzing traffic in a real network, whereas the pattern matching engine alone reaches speeds of up to 70 Gbit/s, which is an almost four times improvement over prior solutions that use specialized hardware.

[1]  Christopher Krügel,et al.  A Parallel Architecture for Stateful, High-Speed Intrusion Detection , 2008, ICISS.

[2]  Somesh Jha,et al.  XFA: Faster Signature Matching with Extended Automata , 2008, 2008 IEEE Symposium on Security and Privacy (sp 2008).

[3]  Jason Lee,et al.  The NIDS Cluster: Scalable, Stateful Network Intrusion Detection on Commodity Hardware , 2007, RAID.

[4]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[5]  Laxmi N. Bhuyan,et al.  Compiling PCRE to FPGA for accelerating SNORT IDS , 2007, ANCS '07.

[6]  T. V. Lakshman,et al.  Gigabit rate packet pattern-matching using TCAM , 2004, Proceedings of the 12th IEEE International Conference on Network Protocols, 2004. ICNP 2004..

[7]  Sotiris Ioannidis,et al.  Regular Expression Matching on Graphics Hardware for Intrusion Detection , 2009, RAID.

[8]  T. V. Lakshman,et al.  Fast and memory-efficient regular expression matching for deep packet inspection , 2006, 2006 Symposium on Architecture For Networking And Communications Systems.

[9]  Seong-Won Lee,et al.  A high performance NIDS using FPGA-based regular expression matching , 2007, SAC '07.

[10]  Nen-Fu Huang,et al.  A GPU-Based Multiple-Pattern Matching Algorithm for Network Intrusion Detection Systems , 2008, 22nd International Conference on Advanced Information Networking and Applications - Workshops (aina workshops 2008).

[11]  George Varghese,et al.  Deterministic memory-efficient string matching algorithms for intrusion detection , 2004, IEEE INFOCOM 2004.

[12]  Eric Torng,et al.  Fast Regular Expression Matching Using Small TCAMs for Network Intrusion Detection and Prevention Systems , 2010, USENIX Security Symposium.

[13]  Dionisios N. Pnevmatikatos,et al.  Pre-decoded CAMs for efficient and high-speed NIDS pattern matching , 2004, 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[14]  Sotiris Ioannidis,et al.  Gnort: High Performance Network Intrusion Detection Using Graphics Processors , 2008, RAID.

[15]  Sangjin Han,et al.  PacketShader: a GPU-accelerated software router , 2010, SIGCOMM '10.

[16]  John W. Lockwood,et al.  Rethinking Hardware Support for Network Analysis and Intrusion Prevention , 2006, HotSec.

[17]  M. Norton Optimizing Pattern Matching for Intrusion Detection , 2004 .

[18]  Vern Paxson,et al.  Shunting: a hardware/software architecture for flexible, high-performance network intrusion prevention , 2007, CCS '07.

[19]  Fabrizio Petrini,et al.  Exact multi-pattern string matching on the cell/b.e. processor , 2008, CF '08.

[20]  Nen-Fu Huang,et al.  A fast string-matching algorithm for network processor-based intrusion detection system , 2004, TECS.

[21]  Luca Deri,et al.  High speed network traffic analysis with commodity multi-core systems , 2010, IMC '10.

[22]  Karthikeyan Sankaralingam,et al.  Evaluating GPUs for network packet signature matching , 2009, 2009 IEEE International Symposium on Performance Analysis of Systems and Software.

[23]  Wenke Lee,et al.  A hardware platform for network intrusion detection and prevention , 2005 .

[24]  John W. Lockwood,et al.  Deep packet inspection using parallel bloom filters , 2004, IEEE Micro.

[25]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[26]  Lambert Schaelicke,et al.  SPANIDS: a scalable network intrusion detection loadbalancer , 2005, CF '05.

[27]  Yung Ryn Choe,et al.  Conservative vs. Optimistic Parallelization of Stateful Network Intrusion Detection , 2007, ISPASS 2008 - IEEE International Symposium on Performance Analysis of Systems and software.

[28]  Katerina J. Argyraki,et al.  RouteBricks: exploiting parallelism to scale software routers , 2009, SOSP '09.

[29]  Haoyu Song,et al.  Snort offloader: a reconfigurable hardware NIDS filter , 2005, International Conference on Field Programmable Logic and Applications, 2005..

[30]  Alfred V. Aho,et al.  Efficient string matching , 1975, Commun. ACM.

[31]  Patrick Crowley,et al.  A hybrid finite automaton for practical deep packet inspection , 2007, CoNEXT '07.

[32]  Vern Paxson,et al.  An architecture for exploiting multi-core processors to parallelize network intrusion prevention , 2007, 2007 IEEE Sarnoff Symposium.

[33]  Evangelos P. Markatos,et al.  An active splitter architecture for intrusion detection and prevention , 2006, IEEE Transactions on Dependable and Secure Computing.

[34]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[35]  Vern Paxson,et al.  An architecture for exploiting multi-core processors to parallelize network intrusion prevention , 2009, NSS 2009.

[36]  George Varghese,et al.  Curing regular expressions matching algorithms from insomnia, amnesia, and acalculia , 2007, ANCS '07.

[37]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[38]  Christopher R. Clark,et al.  Efficient Reconfigurable Logic Circuits for Matching Complex Network Intrusion Detection Patterns , 2003, FPL.

[39]  Christopher Krügel,et al.  Stateful intrusion detection for high-speed network's , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[40]  Viktor K. Prasanna,et al.  Time and area efficient pattern matching on FPGAs , 2004, FPGA '04.

[41]  L. Deri Improving Passive Packet Capture : Beyond Device Polling , 2003 .

[42]  John W. Lockwood,et al.  Implementation of a content-scanning module for an Internet firewall , 2003, 11th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, 2003. FCCM 2003..

[43]  Patrick Crowley,et al.  Algorithms to accelerate multiple regular expressions matching for deep packet inspection , 2006, SIGCOMM.