Design of intelligent KNN-based alarm filter using knowledge-based alert verification in intrusion detection

Network intrusion detection systems NIDSs have been widely deployed in various network environments to defend against different kinds of network attacks. However, a large number of alarms especially unwanted alarms such as false alarms and non-critical alarms could be generated during the detection, which can greatly decrease the efficiency of the detection and increase the burden of analysis. To address this issue, we advocate that constructing an alarm filter in terms of expert knowledge is a promising solution. In this paper, we develop a method of knowledge-based alert verification and design an intelligent alarm filter based on a multi-class k-nearest-neighbor classifier to filter out unwanted alarms. In particular, the alarm filter employs a rating mechanism by means of expert knowledge to classify incoming alarms to proper clusters for labeling. We further analyze the effect of different classifier settings on classification accuracy with two alarm datasets. In the evaluation, we investigate the performance of the alarm filter with a real dataset and in a network environment, respectively. Experimental results indicate that our alarm filter can effectively filter out a number of NIDS alarms and can achieve a better outcome under the advanced mode. Copyright © 2015 John Wiley & Sons, Ltd.

[1]  Wenjuan Li,et al.  Constructing Context-based Non-Critical Alarm Filter in Intrusion Detection , 2012 .

[2]  Lam-for Kwok,et al.  Adaptive non-critical alarm reduction using hash-based contextual signatures in intrusion detection , 2014, Comput. Commun..

[3]  Matt Bishop,et al.  Verify results of network intrusion alerts using lightweight protocol analysis , 2005, 21st Annual Computer Security Applications Conference (ACSAC'05).

[4]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[5]  Tadeusz Pietraszek,et al.  Using Adaptive Alert Classification to Reduce False Positives in Intrusion Detection , 2004, RAID.

[6]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[7]  Thomas Henry Ptacek,et al.  Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection , 1998 .

[8]  Hideki Imai,et al.  IDS False Alarm Reduction Using Continuous and Discontinuous Patterns , 2005, ACNS.

[9]  Humphrey Waita Njogu,et al.  Network specific vulnerability based alert reduction approach , 2013, Secur. Commun. Networks.

[10]  Lam-For Kwok,et al.  Adaptive False Alarm Filter Using Machine Learning in Intrusion Detection , 2011 .

[11]  Sandro Etalle,et al.  ATLANTIDES: An Architecture for Alert Verification in Network Intrusion Detection Systems , 2007, LISA.

[12]  Stefan Axelsson,et al.  The base-rate fallacy and the difficulty of intrusion detection , 2000, TSEC.

[13]  Humphrey Waita Njogu,et al.  A comprehensive vulnerability based alert management approach for large networks , 2013, Future Gener. Comput. Syst..

[14]  Giovanni Vigna,et al.  NetSTAT: a network-based intrusion detection approach , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[15]  Neha Mehra,et al.  Survey on Multiclass Classification Methods , 2013 .

[16]  Xiangjian He,et al.  RePIDS: A multi tier Real-time Payload-based Intrusion Detection System , 2013, Comput. Networks.

[17]  Houkuan Huang,et al.  Intrusion Detection Alert Verification Based on Multi-level Fuzzy Comprehensive Evaluation , 2005, CIS.

[18]  R.K. Cunningham,et al.  Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[19]  Humphrey Waita Njogu,et al.  An efficient approach to reduce alerts generated by multiple IDS products , 2014, Int. J. Netw. Manag..

[20]  Yuh-Jye Lee,et al.  Semi-supervised Learning for False Alarm Reduction , 2010, ICDM.

[21]  Wenjuan Li,et al.  Intelligent Alarm Filter Using Knowledge-Based Alert Verification in Network Intrusion Detection , 2012, ISMIS.

[22]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[23]  Christopher Krügel,et al.  Alert Verification Determining the Success of Intrusion Attempts , 2004, DIMVA.

[24]  Ahmad Akbari,et al.  New class-dependent feature transformation for intrusion detection systems , 2012, Secur. Commun. Networks.

[25]  Ramesh Chandra Joshi,et al.  Parallel Misuse and Anomaly Detection Model , 2012, Int. J. Netw. Secur..

[26]  Lam-for Kwok,et al.  Enhancing False Alarm Reduction Using Pool-Based Active Learning in Network Intrusion Detection , 2013, ISPEC.

[27]  Lam-for Kwok,et al.  Adaptive blacklist-based packet filter with a statistic-based approach in network intrusion detection , 2014, J. Netw. Comput. Appl..

[28]  Zhi-Hua Zhou,et al.  ML-KNN: A lazy learning approach to multi-label learning , 2007, Pattern Recognit..

[29]  Tsuhan Chen,et al.  Semi-supervised co-training and active learning based approach for multi-view intrusion detection , 2009, SAC '09.

[30]  Vern Paxson,et al.  Outside the Closed World: On Using Machine Learning for Network Intrusion Detection , 2010, 2010 IEEE Symposium on Security and Privacy.

[31]  Maghsoud Abbaspour,et al.  Adaptive Anomaly-Based Intrusion Detection System Using Fuzzy Controller , 2012, Int. J. Netw. Secur..

[32]  Lam For Kwok,et al.  IDS False Alarm Filtering Using KNN Classifier , 2004, WISA.

[33]  Karen A. Scarfone,et al.  Guide to Intrusion Detection and Prevention Systems (IDPS) , 2007 .

[34]  Santosh Biswas,et al.  Towards reducing false alarms in network intrusion detection systems with data summarization technique , 2013, Secur. Commun. Networks.

[35]  Karim Tabia,et al.  Alert correlation: Severe attack prediction and controlling false alarm rate tradeoffs , 2011, Intell. Data Anal..

[36]  Anup K. Ghosh,et al.  Detecting anomalous and unknown intrusions against programs , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).