Efficient fingerprint extraction for high performance Intrusion Detection System

Deep Packet Inspection (DPI) module in Intrusion Detection Systems (IDSes) consists of two components: Pre-filter and Rule Verification (RV). Pre-filter adopts Multi-Pattern Matching (MPM) engine to filter out the vast majority of benign packets and then leave a few suspicious packets with false positives into RV component. These false positives are due to the scanning process in the pre-filter: it detects the traffic in a single pass against a set of fingerprints, which are extracted from the given ruleset by selecting only a small portion of the patterns in each signature. RV component precisely checks the suspicious packets and eliminates these false positives. The performance of DPI module is related to the extracted fingerprint set. An efficient fingerprint set should improve the pre-filter throughput, and at the same time decrease the count of checking activities in RV component. We show in this paper that these two requirements cannot be simultaneously satisfied in the existing fingerprint extraction strategies. Pre-filter performance greatly benefits from smaller fingerprint set because of the more compact MPM engine. But RV component suffers from the higher rate of false positives caused by the smaller fingerprint set. We optimally trade off these two requirements with a new extraction method in this work. Through analysing a small amount of training traffic in the initial phase, our strategy gives each fingerprint candidate an empirical weight for the subsequent extraction. Experimental results obtained by integrating our proposed method into the Snort IDS show that our strategy improves the IDS average throughput by at least 69% over the latest real ruleset and real traffic.