A Digest and Pattern Matching-Based Intrusion Detection Engine

Intrusion detection/prevention systems (IDSs/IPSs) heavily rely on signature databases and pattern matching (PM) techniques to identify network attacks. The engines of such systems often employ traditional PM algorithms to search for telltale patterns in network flows. The observations that real-world network traffic is largely legitimate and that telltales manifested by exploits rarely appear in network streams lead us to the proposal of Fingerprinter. This framework integrates fingerprinting and PM methods to rapidly distinguish well-behaved from malicious traffic. Fingerprinter produces concise digests or fingerprints for attack signatures during its programming phase. In its querying phase, the framework quickly identifies attack-free connections by transforming input traffic into its fingerprint space and matching its digest against those of attack signatures. If the legitimacy of a stream cannot be determined by fingerprints alone, our framework uses the Boyer–Moore algorithm to ascertain whether attack signatures appear in the stream. To reduce false matches, we resort to multiple fingerprinting techniques including Bloom–Filter and Rabin–Fingerprint. Experimentation with a prototype and a variety of traces has helped us establish that Fingerprinter significantly accelerates the attack detection process.

[1]  C. Q. Lee,et al.  The Computer Journal , 1958, Nature.

[2]  Burton H. Bloom,et al.  Space/time trade-offs in hash coding with allowable errors , 1970, CACM.

[3]  H. Feistel Cryptography and Computer Privacy , 1973 .

[4]  Alfred V. Aho,et al.  Efficient string matching , 1975, Commun. ACM.

[5]  Robert S. Boyer,et al.  A fast string searching algorithm , 1977, CACM.

[6]  Donald E. Knuth,et al.  Fast Pattern Matching in Strings , 1977, SIAM J. Comput..

[7]  Ronald L. Rivest On the Worst-Case Behavior of String-Searching Algorithms , 1977, SIAM J. Comput..

[8]  Zvi Galil,et al.  On improving the worst case running time of the Boyer-Moore string matching algorithm , 1978, CACM.

[9]  Zvi Galil On improving the worst case running time of the Boyer-Moore string matching algorithm , 1979, CACM.

[10]  Beate Commentz-Walter,et al.  A String Matching Algorithm Fast on the Average , 1979, ICALP.

[11]  R. Nigel Horspool,et al.  Practical fast searching in strings , 1980, Softw. Pract. Exp..

[12]  Raffaele Giancarlo,et al.  The Boyer-Moore-Galil String Searching Strategies Revisited , 1986, SIAM J. Comput..

[13]  Ivan Damgård,et al.  A Design Principle for Hash Functions , 1989, CRYPTO.

[14]  Peter B. Danzig,et al.  Characteristics of wide-area TCP/IP conversations , 1991, SIGCOMM '91.

[15]  Henry J. Fowler,et al.  Local Area Network Traffic Characteristics, with Implications for Broadband Network Congestion Management , 1991, IEEE J. Sel. Areas Commun..

[16]  V. Paxson,et al.  Growth trends in wide-area TCP connections , 1994, IEEE Network.

[17]  Graham A. Stephen String Searching Algorithms , 1994, Lecture Notes Series on Computing.

[18]  Udi Manber,et al.  Finding Similar Files in a Large File System , 1994, USENIX Winter.

[19]  Dan Gusfield,et al.  Algorithms on Strings, Trees, and Sequences - Computer Science and Computational Biology , 1997 .

[20]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[21]  Udi Manber,et al.  A FAST ALGORITHM FOR MULTI-PATTERN SEARCHING , 1999 .

[22]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[23]  William A. Arbaugh,et al.  A trend analysis of exploitations , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[24]  Sharath Pankanti,et al.  On the individuality fingerprints , 2001, Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001.

[25]  Mark Handley,et al.  Network Intrusion Detection: Evasion, Traffic Normalization, and End-to-End Protocol Semantics , 2001, USENIX Security Symposium.

[26]  C.J. Coit,et al.  Towards faster string matching for intrusion detection or exceeding the speed of Snort , 2001, Proceedings DARPA Information Survivability Conference and Exposition II. DISCEX'01.

[27]  Christopher Krügel,et al.  Stateful intrusion detection for high-speed network's , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[28]  Steve Poole,et al.  Granidt: Towards Gigabit Rate Network Intrusion Detection Technology , 2002, FPL.

[29]  Brad L. Hutchings,et al.  Assisting network intrusion detection with reconfigurable hardware , 2002, Proceedings. 10th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[30]  Marc Najork,et al.  On the evolution of clusters of near-duplicate Web pages , 2003, Proceedings of the IEEE/LEOS 3rd International Conference on Numerical Simulation of Semiconductor Optoelectronic Devices (IEEE Cat. No.03EX726).

[31]  Maxime Crochemore,et al.  A unifying look at the Apostolico-Giancarlo string-matching algorithm , 2003, J. Discrete Algorithms.

[32]  M. Frans Kaashoek,et al.  Proceedings of the General Track: 2003 Usenix Annual Technical Conference Role Classification of Hosts within Enterprise Networks Based on Connection Patterns , 2022 .

[33]  John W. Lockwood,et al.  Implementation of a content-scanning module for an Internet firewall , 2003, 11th Annual IEEE Symposium on Field-Programmable Custom Computing Machines, 2003. FCCM 2003..

[34]  John W. Lockwood,et al.  Architecture for a hardware based, TCP/IP content scanning system [intrusion detection system applications] , 2003, 11th Symposium on High Performance Interconnects, 2003. Proceedings..

[35]  Christopher R. Clark,et al.  Efficient Reconfigurable Logic Circuits for Matching Complex Network Intrusion Detection Patterns , 2003, FPL.

[36]  Evangelos P. Markatos,et al.  : A DOMAIN-SPECIFIC STRING MATCHING ALGORITHM FOR INTRUSION DETECTION , 2003 .

[37]  Nen-Fu Huang,et al.  A fast string-matching algorithm for network processor-based intrusion detection system , 2004, TECS.

[38]  A. Arora,et al.  Impact of Vulnerability Disclosure and Patch Availability - An Empirical Analysis , 2004 .

[39]  Kei Hiraki,et al.  Over 10Gbps String Matching Mechanism for Multi-stream Packet Scanning Systems , 2004, FPL.

[40]  Viktor K. Prasanna,et al.  Time and area efficient pattern matching on FPGAs , 2004, FPGA '04.

[41]  Jia Wang,et al.  Analyzing peer-to-peer traffic across large networks , 2002, IMW '02.

[42]  John W. Lockwood,et al.  Architecture for a hardware-based, TCP/IP content-processing system , 2004, IEEE Micro.

[43]  Viktor K. Prasanna,et al.  A methodology for synthesis of efficient intrusion detection systems on FPGAs , 2004, 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[44]  George Varghese,et al.  Deterministic memory-efficient string matching algorithms for intrusion detection , 2004, IEEE INFOCOM 2004.

[45]  Sarang Dharmapurikar,et al.  Implementation results of bloom filters for string matching , 2004, 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[46]  William H. Mangione-Smith,et al.  Deep packet filter with dedicated logic and read only memories , 2004, 12th Annual IEEE Symposium on Field-Programmable Custom Computing Machines.

[47]  William H. Mangione-Smith,et al.  Fast reconfiguring deep packet filter for 1+ gigabit network , 2005, 13th Annual IEEE Symposium on Field-Programmable Custom Computing Machines (FCCM'05).

[48]  Timothy Sherwood,et al.  A high throughput string matching architecture for intrusion detection and prevention , 2005, 32nd International Symposium on Computer Architecture (ISCA'05).

[49]  Haoyu Song,et al.  Snort offloader: a reconfigurable hardware NIDS filter , 2005, International Conference on Field Programmable Logic and Applications, 2005..

[50]  Jason Lee,et al.  A first look at modern enterprise traffic , 2005, IMC '05.

[51]  Herbert Bos,et al.  Towards Software-Based Signature Detection for Intrusion Prevention on the Network Card , 2005, RAID.

[52]  Haoyu Song,et al.  Fast hash table lookup using extended bloom filter: an aid to network processing , 2005, SIGCOMM '05.

[53]  Viktor K. Prasanna,et al.  High-throughput linked-pattern matching for intrusion detection systems , 2005, 2005 Symposium on Architectures for Networking and Communications Systems (ANCS).

[54]  Patrick D. McDaniel,et al.  Analysis of Communities of Interest in Data Networks , 2005, PAM.

[55]  Lukas Kencl,et al.  Approximate fingerprinting to accelerate pattern matching , 2006, IMC '06.

[56]  Alex Delis,et al.  A Pragmatic Methodology for Testing Intrusion Prevention Systems , 2009, Comput. J..