On the statistical distribution of processing times in network intrusion detection

Intrusion detection systems (IDSs) are relatively complex devices that monitor information systems in search for security violations. Characterizing the service times of network IDSs is a crucial step in improving their real time performance. We analyzed about 41 million packets organized in five data sets of 10 minutes each collected at the entry point of a large production network and processed by Snort, a commonly used IDS. The processing times of the three main stages in Snort were measured. The main conclusions of our study were: (1) rule checking accounts for about 75% of the total processing time in IDSs, with mean pay load checking time being 4.5 times larger than mean header checking time. (2) The distribution of rule checking limes is markedly bimodal, a direct consequence of the bimodality in packet composition in current high speed Internet traffic. (3) Header processing times have a small variance and small correlation coefficients. (4) In contrast, the distribution of payload processing times displays high variance, in a form that can be generally characterized as "slightly heavy-tailed". Explicitly, payload processing times have a lognormal upper tail, clipped at the top 1%. This extreme 1% upper tail is better fit by an exponential distribution. (5) Additionally, payload processing times were shown to be highly correlated, with correlation coefficients several orders of magnitude higher than the confidence bands for the standard whiteness test. The impact of these findings in the design of IDSs for real time operation in networks is discussed, and compared with existing results for processing times for Unix processes, which were shown to display pronounced heavy-tailed characteristics.