Adaptive blacklist-based packet filter with a statistic-based approach in network intrusion detection

Network intrusion detection systems (NIDS) are widely deployed in various network environments. Compared to an anomaly based NIDS, a signature-based NIDS is more popular in real-world applications, because of its relatively lower false alarm rate. However, the process of signature matching is a key limiting factor to impede the performance of a signature-based NIDS, in which the cost is at least linear to the size of an input string and the CPU occupancy rate can reach more than 80% in the worst case. In this paper, we develop an adaptive blacklist-based packet filter using a statistic-based approach aiming to improve the performance of a signature-based NIDS. The filter employs a blacklist technique to help filter out network packets based on IP confidence and the statistic-based approach allows the blacklist generation in an adaptive way, that is, the blacklist can be updated periodically. In the evaluation, we give a detailed analysis of how to select weight values in the statistic-based approach, and investigate the performance of the packet filter with a DARPA dataset, a real dataset and in a real network environment. Our evaluation results under various scenarios show that our proposed packet filter is encouraging and effective to reduce the burden of a signature-based NIDS without affecting network security.

[1]  Udi Manber,et al.  A FAST ALGORITHM FOR MULTI-PATTERN SEARCHING , 1999 .

[2]  Domenico Cantone,et al.  On the bit-parallel simulation of the nondeterministic Aho-Corasick and suffix automata for a set of patterns , 2012, J. Discrete Algorithms.

[3]  Christian Rossow,et al.  Empirical research on IP blacklisting , 2008, CEAS.

[4]  Hyunjin Kim,et al.  A Memory-Efficient Bit-Split Parallel String Matching Using Pattern Dividing for Intrusion Detection Systems , 2011, IEEE Transactions on Parallel and Distributed Systems.

[5]  Lam-for Kwok,et al.  Adaptive context-aware packet filter scheme using statistic-based blacklist generation in network intrusion detection , 2011, 2011 7th International Conference on Information Assurance and Security (IAS).

[6]  Pi-Chung Wang,et al.  Scalable packet classification with controlled cross-producting , 2009, Comput. Networks.

[7]  George Varghese,et al.  Fast Content-Based Packet Handling for Intrusion Detection , 2001 .

[8]  Andrew W. Moore,et al.  Bayesian Neural Networks for Internet Traffic Classification , 2007, IEEE Transactions on Neural Networks.

[9]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[10]  Jorma Tarhio,et al.  Bit-Parallel Search Algorithms for Long Patterns , 2010, SEA.

[11]  R. Nigel Horspool,et al.  Practical fast searching in strings , 1980, Softw. Pract. Exp..

[12]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[13]  Hyesook Lim,et al.  Hierarchical packet classification using a Bloom filter and rule-priority tries , 2010, Comput. Commun..

[14]  Miad Faezipour,et al.  Wire-Speed TCAM-Based Architectures for Multimatch Packet Classification , 2009, IEEE Transactions on Computers.

[15]  Insup Lee,et al.  Spam mitigation using spatio-temporal reputations from blacklist history , 2010, ACSAC '10.

[16]  Seung-Woo Seo,et al.  A fast pattern matching algorithm with multi-byte search unit for high-speed network security , 2011, Comput. Commun..

[17]  Jin Xu,et al.  Chemical Reaction Optimization for Task Scheduling in Grid Computing , 2011, IEEE Transactions on Parallel and Distributed Systems.

[18]  Anat Bremler-Barr,et al.  CompactDFA: Generic State Machine Compression for Scalable Pattern Matching , 2010, 2010 Proceedings IEEE INFOCOM.

[19]  Haoyu Song,et al.  Efficient packet classification for network intrusion detection using FPGA , 2005, FPGA '05.

[20]  Alfred V. Aho,et al.  Efficient string matching , 1975, Commun. ACM.

[21]  Hyesook Lim,et al.  A new hierarchical packet classification algorithm , 2012, Comput. Networks.

[22]  Beate Commentz-Walter,et al.  A String Matching Algorithm Fast on the Average , 1979, ICALP.

[23]  Stefan Axelsson,et al.  The base-rate fallacy and the difficulty of intrusion detection , 2000, TSEC.

[24]  Ronald L. Rivest.  On the Worst-Case Behavior of String-Searching Algorithms , 1977, SIAM J. Comput..

[25]  Jetzabel M. Serna,et al.  Benchmarking IP blacklists for financial botnet detection , 2010, 2010 Sixth International Conference on Information Assurance and Security.

[26]  Lucas Vespa,et al.  MS-DFA: Multiple-Stride Pattern Matching for Scalable Deep Packet Inspection , 2011, Comput. J..

[27]  Robert S. Boyer,et al.  A fast string searching algorithm , 1977, CACM.

[28]  Stamatis Vassiliadis,et al.  Packet pre-filtering for network intrusion detection , 2006, 2006 Symposium on Architecture For Networking And Communications Systems.

[29]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[30]  F. Soldo,et al.  Filtering sources of unwanted traffic , 2008, 2008 Information Theory and Applications Workshop.

[31]  Haoyu Song,et al.  Toward Advocacy-Free Evaluation of Packet Classification Algorithms , 2011, IEEE Transactions on Computers.

[32]  Adel Bouhoula,et al.  A prefix-based approach for managing hybrid specifications in complex packet filtering , 2012, Comput. Networks.

[33]  Xing Wang,et al.  Multi-Stride String Searching for High-Speed Content Inspection , 2012, Comput. J..

[34]  Insup Lee,et al.  Towards the effective temporal association mining of spam blacklists , 2011, CEAS '11.

[35]  Emil Sit,et al.  An empirical study of spam traffic and the use of DNS black lists , 2004, IMC '04.

[36]  Karen A. Scarfone,et al.  Guide to Intrusion Detection and Prevention Systems (IDPS) , 2007 .

[37]  Anja Feldmann,et al.  Operational experiences with high-volume network intrusion detection , 2004, CCS '04.

[38]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[39]  T. V. Lakshman,et al.  Variable-Stride Multi-Pattern Matching For Scalable Deep Packet Inspection , 2009, IEEE INFOCOM 2009.

[40]  Roberto Perdisci,et al.  Early Detection of Malicious Flux Networks via Large-Scale Passive DNS Traffic Analysis , 2012, IEEE Transactions on Dependable and Secure Computing.

[41]  Stefan Axelsson,et al.  The base-rate fallacy and its implications for the difficulty of intrusion detection , 1999, CCS '99.