A fast regular expression matching engine for NIDS applying prediction scheme

Regular expression matching is considered important as it lies at the heart of many networking applications using deep packet inspection (DPI) techniques. For example, modern networking intrusion detection systems (NIDSs) typically accomplish regular expression matching using deterministic finite automata (DFA) algorithm. However, DFA suffers from the high memory consumption for the state blowup problem. Many algorithms have been proposed to compress the DFA memory storage space, meanwhile, they usually pay the price of low matching speed and high memory bandwidth. In this paper, we first propose an effective DFA compression algorithm by exploiting the similarity between DFA states. Then, we apply a next-state prediction strategy and present a fast DFA matching engine. Carefully designing the DFA matching circuit, we keep the prediction success rate by more than 99,5%, thus get a comparable matching speed with original DFA algorithm. On the side of memory consumption, experimental results show that with typical NIDS rule sets, our algorithm compressed the original DFA by more than 99%. Mapping this algorithm on Xilinx Virtex-7 FPGA chip, we get a throughput of more than 200Gbps.

[1]  Vern Paxson,et al.  Enhancing byte-level network intrusion detection signatures with context , 2003, CCS '03.

[2]  Peter J. Denning,et al.  Experiments with program locality , 1899, AFIPS '72 (Fall, part I).

[3]  Patrick Crowley,et al.  A-DFA: A Time- and Space-Efficient DFA Compression Algorithm for Fast Regular Expression Evaluation , 2013, TACO.

[4]  Li Guo,et al.  Compressing Regular Expressions' DFA Table by Matrix Decomposition , 2010, CIAA.

[5]  James E. Smith,et al.  A study of branch prediction strategies , 1981, ISCA '98.

[6]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[7]  Patrick Crowley,et al.  Algorithms to accelerate multiple regular expressions matching for deep packet inspection , 2006, SIGCOMM 2006.

[8]  Stefano Giordano,et al.  An improved DFA for fast regular expression matching , 2008, CCRV.

[9]  Li Guo,et al.  An efficient regular expressions compression algorithm from a new perspective , 2011, 2011 Proceedings IEEE INFOCOM.

[10]  Viktor K. Prasanna,et al.  FEACAN: Front-end acceleration for content-aware network processing , 2011, 2011 Proceedings IEEE INFOCOM.

[11]  Stefano Giordano,et al.  Differential Encoding of DFAs for Fast Regular Expression Matching , 2011, IEEE/ACM Transactions on Networking.

[12]  Jennifer M. Murphy,et al.  The Measurement of Locality and the Behaviour of Programs , 1984, Comput. J..

[13]  James R. Larus,et al.  Branch prediction for free , 1993, PLDI '93.

[14]  T. V. Lakshman,et al.  Fast and memory-efficient regular expression matching for deep packet inspection , 2006, 2006 Symposium on Architecture For Networking And Communications Systems.