NetShield: massive semantics-based vulnerability signature matching for high-speed networks

Accuracy and speed are the two most important metrics for Network Intrusion Detection/Prevention Systems (NIDS/NIPSes). Due to emerging polymorphic attacks and the fact that in many cases regular expressions (regexes) cannot capture the vulnerability conditions accurately, the accuracy of existing regex-based NIDS/NIPS systems has become a serious problem. In contrast, the recently-proposed vulnerability signatures (a.k.a data patches) can exactly describe the vulnerability conditions and achieve better accuracy. However, how to efficiently apply vulnerability signatures to high speed NIDS/NIPS with a large ruleset remains an untouched but challenging issue. This paper presents the first systematic design of vulnerability signature based parsing and matching engine, NetShield, which achieves multi-gigabit throughput while offering much better accuracy. Particularly, we made the following contributions: (i) we proposed a candidate selection algorithm which efficiently matches thousands of vulnerability signatures simultaneously requiring a small amount of memory; (ii) we proposed an automatic lightweight parsing state machine achieving fast protocol parsing. Experimental results show that the core engine of NetShield achieves at least 1.9+Gbps signature matching throughput on a 3.8GHz single-core PC, and can scale-up to at least 11+Gbps under a 8-core machine for 794 HTTP vulnerability signatures.

[1]  Miguel Castro,et al.  Vigilante: end-to-end containment of internet worms , 2005, SOSP '05.

[2]  Patrick Crowley,et al.  Algorithms to accelerate multiple regular expressions matching for deep packet inspection , 2006, SIGCOMM 2006.

[3]  Nick McKeown,et al.  Classifying Packets with Hierarchical Intelligent Cuttings , 2000, IEEE Micro.

[4]  Venkatachary Srinivasan,et al.  Packet classification using tuple space search , 1999, SIGCOMM '99.

[5]  Vern Paxson,et al.  Active mapping: resisting NIDS evasion without altering traffic , 2003, 2003 Symposium on Security and Privacy, 2003..

[6]  George Varghese,et al.  Packet classification using multidimensional cutting , 2003, SIGCOMM '03.

[7]  Patrick Crowley,et al.  A hybrid finite automaton for practical deep packet inspection , 2007, CoNEXT '07.

[8]  Bernard Chazelle,et al.  Lower bounds for orthogonal range searching: part II. The arithmetic model , 1990, JACM.

[9]  Helen J. Wang,et al.  ShieldGen: Automatic Data Patch Generation for Unknown Vulnerabilities with Informed Probing , 2007, 2007 IEEE Symposium on Security and Privacy (SP '07).

[10]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[11]  Sarang Dharmapurikar,et al.  Robust TCP Stream Reassembly in the Presence of Adversaries , 2005, USENIX Security Symposium.

[12]  Somesh Jha,et al.  XFA: Faster Signature Matching with Extended Automata , 2008, 2008 IEEE Symposium on Security and Privacy (sp 2008).

[13]  Nikita Borisov,et al.  High-Speed Matching of Vulnerability Signatures , 2008, RAID.

[14]  T. V. Lakshman,et al.  High-speed policy-based packet forwarding using efficient multi-dimensional range matching , 1998, SIGCOMM '98.

[15]  Somesh Jha,et al.  Deflating the big bang: fast and scalable deep packet inspection with extended finite automata , 2008, SIGCOMM '08.

[16]  Patrick Crowley,et al.  Efficient regular expression evaluation: theory to practice , 2008, ANCS '08.

[17]  David E. Taylor.  Survey and taxonomy of packet classification techniques , 2005, CSUR.

[18]  T. V. Lakshman,et al.  Fast and memory-efficient regular expression matching for deep packet inspection , 2006, 2006 Symposium on Architecture For Networking And Communications Systems.

[19]  Bin Liu,et al.  NetShield : Matching with a Large Vulnerability Signature R uleset for High Performance Network Defense , 2010 .

[20]  Helen J. Wang,et al.  Generic Application-Level Protocol Analyzer and its Language , 2007, NDSS.

[21]  Helen J. Wang,et al.  Shield: vulnerability-driven network filters for preventing known vulnerability exploits , 2004, SIGCOMM 2004.

[22]  George Varghese,et al.  Scalable packet classification , 2001, SIGCOMM 2001.

[23]  Nick McKeown,et al.  Packet classification on multiple fields , 1999, SIGCOMM '99.

[24]  Larry L. Peterson,et al.  binpac: a yacc for writing application protocol parsers , 2006, IMC '06.

[25]  Hao Wang,et al.  Towards automatic generation of vulnerability-based signatures , 2006, 2006 IEEE Symposium on Security and Privacy (S&P'06).