Re-examining the performance bottleneck in a NIDS with detailed profiling

Designing a high-speed network intrusion detection system (NIDS) has attracted much attention in recent years due to ever-increasing amount of network traffic and ever-complicated attacks. Numerous studies have been focusing on accelerating pattern matching for a high-speed design because some early studies observed that pattern matching is a performance bottleneck. However, the effectiveness of such acceleration has been challenged recently. This work therefore re-examines the performance bottleneck by profiling two popular NIDSs, Snort and Bro, with various types of network traffic in detail. In the profiling, we find pattern matching can be dominant in the Snort execution if the entire packet payloads in the connections are scanned, while executing the policy scripts is an obvious bottleneck in the Bro execution. This work suggests three promising directions towards a high-speed NIDS design for future research: a method to precisely specify the possible locations of the signatures in long connections, a compiler to transform the policy scripts to efficient binary codes for execution, and an efficient design of connection tracking and packet reassembly.

[1]  Hyunjin Kim,et al.  A Memory-Efficient Bit-Split Parallel String Matching Using Pattern Dividing for Intrusion Detection Systems , 2011, IEEE Transactions on Parallel and Distributed Systems.

[2]  Somesh Jha,et al.  Backtracking Algorithmic Complexity Attacks against a NIDS , 2006, 2006 22nd Annual Computer Security Applications Conference (ACSAC'06).

[3]  Chung-Horng Lung,et al.  An Efficient Approach to Per-Flow State Tracking for High-Speed Networks , 2010, 2010 IEEE Global Telecommunications Conference GLOBECOM 2010.

[4]  Vern Paxson,et al.  A high-level programming environment for packet trace anonymization and transformation , 2003, SIGCOMM '03.

[5]  Thomas Engel,et al.  Runtime Monitoring and Dynamic Reconfiguration for Intrusion Detection Systems , 2009, RAID.

[6]  George Varghese,et al.  Deterministic memory-efficient string matching algorithms for intrusion detection , 2004, IEEE INFOCOM 2004.

[7]  Eric Torng,et al.  Fast Regular Expression Matching Using Small TCAMs for Network Intrusion Detection and Prevention Systems , 2010, USENIX Security Symposium.

[8]  T. V. Lakshman,et al.  Fast and memory-efficient regular expression matching for deep packet inspection , 2006, 2006 Symposium on Architecture For Networking And Communications Systems.

[9]  Salvatore Pontarelli,et al.  Traffic-Aware Design of a High-Speed FPGA Network Intrusion Detection System , 2013, IEEE Transactions on Computers.

[10]  Gabi Nakibly,et al.  Network-based intrusion detection systems go active! , 2012, CCS.

[11]  Anja Feldmann,et al.  Predicting the resource consumption of network intrusion detection systems , 2008, SIGMETRICS '08.

[12]  George Varghese,et al.  Applying Fast String Matching to Intrusion Detection , 2001 .

[13]  V AhoAlfred,et al.  Efficient string matching , 1975 .

[14]  Sotiris Ioannidis,et al.  Gnort: High Performance Network Intrusion Detection Using Graphics Processors , 2008, RAID.

[15]  Tsern-Huei Lee.  Hardware Architecture for High-Performance Regular Expression Matching , 2009, IEEE Transactions on Computers.

[16]  John W. Lockwood,et al.  Rethinking Hardware Support for Network Analysis and Intrusion Prevention , 2006, HotSec.

[17]  Ying-Dar Lin,et al.  On campus beta site: architecture designs, operational experience, and top product defects , 2010, IEEE Communications Magazine.

[18]  Norbik Bashah Idris,et al.  A parallel technique for improving the performance of signature-based network intrusion detection system , 2011, 2011 IEEE 3rd International Conference on Communication Software and Networks.

[19]  Evangelos P. Markatos,et al.  : A DOMAIN-SPECIFIC STRING MATCHING ALGORITHM FOR INTRUSION DETECTION , 2003 .

[20]  John W. Lockwood,et al.  Fast and scalable pattern matching for content filtering , 2005, 2005 Symposium on Architectures for Networking and Communications Systems (ANCS).

[21]  Todd L. Heberlein,et al.  Network intrusion detection , 1994, IEEE Network.

[22]  Khaled Salah,et al.  Performance evaluation comparison of Snort NIDS under Linux and Windows Server , 2010, J. Netw. Comput. Appl..

[23]  Hari Balakrishnan,et al.  Efficient and Robust TCP Stream Normalization , 2008, 2008 IEEE Symposium on Security and Privacy (sp 2008).

[24]  Anja Feldmann,et al.  Operational experiences with high-volume network intrusion detection , 2004, CCS '04.

[25]  Sarang Dharmapurikar,et al.  Robust TCP Stream Reassembly in the Presence of Adversaries , 2005, USENIX Security Symposium.

[26]  Anja Feldmann,et al.  Enriching network security analysis with time travel , 2008, SIGCOMM '08.

[27]  Salvatore J. Stolfo,et al.  Toward Cost-Sensitive Modeling for Intrusion Detection and Response , 2002, J. Comput. Secur..

[28]  Ali A. Ghorbani,et al.  Toward developing a systematic approach to generate benchmark datasets for intrusion detection , 2012, Comput. Secur..

[29]  Yuan-Cheng Lai,et al.  Low-storage capture and loss recovery selective replay of real flows , 2012, IEEE Communications Magazine.

[30]  Viktor K. Prasanna,et al.  A Memory-Efficient and Modular Approach for Large-Scale String Pattern Matching , 2013, IEEE Transactions on Computers.

[31]  Vern Paxson,et al.  An architecture for exploiting multi-core processors to parallelize network intrusion prevention , 2009, NSS 2009.

[32]  Jan van Lunteren,et al.  Hardware-accelerated regular expression matching at multiple tens of Gb/s , 2012, 2012 Proceedings IEEE INFOCOM.

[33]  J.B.D. Cabrera,et al.  On the statistical distribution of processing times in network intrusion detection , 2004, 2004 43rd IEEE Conference on Decision and Control (CDC) (IEEE Cat. No.04CH37601).

[34]  Chung-Horng Lung,et al.  An efficient hybrid approach to per-flow state tracking for high-speed networks , 2013, Comput. Commun..

[35]  Vern Paxson,et al.  Shunting: a hardware/software architecture for flexible, high-performance network intrusion prevention , 2007, CCS '07.

[36]  John W. Lockwood,et al.  Deep packet inspection using parallel bloom filters , 2004, IEEE Micro.

[37]  Alfred V. Aho,et al.  Efficient string matching , 1975, Commun. ACM.

[38]  Somesh Jha,et al.  Multi-byte Regular Expression Matching with Speculation , 2009, RAID.

[39]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[40]  Wing-Kai Hon,et al.  Memory-efficient pattern matching architectures using perfect hashing on graphic processing units , 2012, 2012 Proceedings IEEE INFOCOM.

[41]  Sheng-De Wang,et al.  Embedded Network Intrusion Detection Systems with a Multi-core Aware Packet Capture Module , 2011, 2011 40th International Conference on Parallel Processing Workshops.

[42]  Tsern-Huei Lee,et al.  Using String Matching for Deep Packet Inspection , 2008, Computer.

[43]  Yi Zhang,et al.  Performance Adaptation in Real-Time Intrusion Detection Systems , 2002, RAID.

[44]  Evangelos P. Markatos,et al.  Performance analysis of content matching intrusion detection systems , 2004, 2004 International Symposium on Applications and the Internet. Proceedings..

[45]  H. Jonathan Chao,et al.  A 10-Gbps High-Speed Single-Chip Network Intrusion Detection and Prevention System , 2007, IEEE GLOBECOM 2007 - IEEE Global Telecommunications Conference.