Semi-supervised Learning for False Alarm Reduction

Intrusion Detection Systems (IDSs) which have been deployed in computer networks to detect a wide variety of attacks are suffering how to manage of a large number of triggered alerts. Thus, reducing false alarms efficiently has become the most important issue in IDS. In this paper, we introduce the semi-supervised learning mechanism to build an alert filter, which will reduce up to 85% false alarms and still keep a high detection rate. In our semi-supervised learning approach, we only need a very small amount of label information. This will save a huge security officer's effort and make the alert filter be more practical for the real systems. Numerical comparison with conventional supervised learning approach with the same small portion labeled data, our method has significantly superior detection rate as well as in the false alarm reduction rate.

[1]  Salvatore J. Stolfo,et al.  A framework for constructing features and models for intrusion detection systems , 2000, TSEC.

[2]  Hideki Imai,et al.  IDS False Alarm Reduction Using Continuous and Discontinuous Patterns , 2005, ACNS.

[3]  Hsing-Kuo Kenneth Pao,et al.  An RSVM based two-teachers-one-student semi-supervised learning algorithm , 2012, Neural Networks.

[4]  Tsuhan Chen,et al.  Semi-supervised co-training and active learning based approach for multi-view intrusion detection , 2009, SAC '09.

[5]  Avrim Blum,et al.  The Bottleneck , 2021, Monopsony Capitalism.

[6]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[7]  Zhi-Hua Zhou,et al.  Tri-training: exploiting unlabeled data using three classifiers , 2005, IEEE Transactions on Knowledge and Data Engineering.

[8]  Vern Paxson,et al.  Bro: a system for detecting network intruders in real-time , 1998, Comput. Networks.

[9]  Yuh-Jye Lee,et al.  RSVM: Reduced Support Vector Machines , 2001, SDM.

[10]  Paola Lecca,et al.  A new probabilistic generative model of parameter inference in biochemical networks , 2009, SAC '09.

[11]  Tadeusz Pietraszek,et al.  Using Adaptive Alert Classification to Reduce False Positives in Intrusion Detection , 2004, RAID.

[12]  Yingjie Tian,et al.  Semi-supervised learning methods for network intrusion detection , 2008, 2008 IEEE International Conference on Systems, Man and Cybernetics.

[13]  Terran Lane,et al.  A Decision-Theoritic, Semi-Supervised Model for Intrusion Detection , 2006 .

[14]  Martin Roesch,et al.  Snort - Lightweight Intrusion Detection for Networks , 1999 .

[15]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.