Toward a more practical unsupervised anomaly detection system

During the last decade, various machine learning and data mining techniques have been applied to Intrusion Detection Systems (IDSs) which have played an important role in defending critical computer systems and networks from cyber attacks. Unsupervised anomaly detection techniques have received a particularly great amount of attention because they enable construction of intrusion detection models without using labeled training data (i.e., with instances preclassified as being or not being an attack) in an automated manner and offer intrinsic ability to detect unknown attacks; i.e., 0-day attacks. Despite the advantages, it is still not easy to deploy them into a real network environment because they require several parameters during their building process, and thus IDS operators and managers suffer from tuning and optimizing the required parameters based on changes of their network characteristics. In this paper, we propose a new anomaly detection method by which we can automatically tune and optimize the values of parameters without predefining them. We evaluated the proposed method over real traffic data obtained from Kyoto University honeypots. The experimental results show that the performance of the proposed method is superior to that of the previous one.

[1]  Leonid Portnoy,et al.  Intrusion detection with unlabeled data using clustering , 2000 .

[2]  Andrew H. Sung,et al.  Identifying Significant Features for Network Forensic Analysis Using Artificial Intelligence Techniques , 2003, Int. J. Digit. EVid..

[3]  Ali A. Ghorbani,et al.  Y-means: a clustering method for intrusion detection , 2003, CCECE 2003 - Canadian Conference on Electrical and Computer Engineering. Toward a Caring and Humane Technology (Cat. No.03CH37436).

[4]  Nello Cristianini,et al.  An introduction to Support Vector Machines , 2000 .

[5]  Jungsuk Song,et al.  Cooperation of Intelligent Honeypots to Detect Unknown Malicious Codes , 2008, 2008 WOMBAT Workshop on Information Security Threats Data Collection and Sharing.

[6]  Alexander Hofmann,et al.  On the versatility of radial basis function neural networks: A case study in the field of intrusion detection , 2010, Inf. Sci..

[7]  Chris Clifton,et al.  Developing custom intrusion detection filters using data mining , 2000, MILCOM 2000 Proceedings. 21st Century Military Communications. Architectures and Technologies for Information Superiority (Cat. No.00CH37155).

[8]  Eleazar Eskin,et al.  A GEOMETRIC FRAMEWORK FOR UNSUPERVISED ANOMALY DETECTION: DETECTING INTRUSIONS IN UNLABELED DATA , 2002 .

[9]  Hiroki Takakura,et al.  A Clustering Method for Improving Performance of Anomaly-Based Intrusion Detection System , 2008, IEICE Trans. Inf. Syst..

[10]  Chih-Jen Lin,et al.  LIBSVM: A library for support vector machines , 2011, TIST.

[11]  William L. Fithen,et al.  State of the Practice of Intrusion Detection Technologies , 2000 .

[12]  Dorothy E. Denning,et al.  An Intrusion-Detection Model , 1987, IEEE Transactions on Software Engineering.

[13]  Tadeusz Pietraszek,et al.  Using Adaptive Alert Classification to Reduce False Positives in Intrusion Detection , 2004, RAID.

[14]  Hiroki Takakura,et al.  Unsupervised Anomaly Detection Based on Clustering and Multiple One-Class SVM , 2009, IEICE Trans. Commun..

[15]  Nello Cristianini,et al.  An Introduction to Support Vector Machines and Other Kernel-based Learning Methods , 2000 .

[16]  Deborah A. Frincke,et al.  A Novel Framework for Alert Correlation and Understanding , 2004, ACNS.

[17]  Klaus Julisch,et al.  Clustering intrusion detection alarms to support root cause analysis , 2003, TSEC.

[18]  Christopher Leckie,et al.  Unsupervised Anomaly Detection in Network Intrusion Detection Using Clusters , 2005, ACSC.

[19]  Fabio Roli,et al.  Alarm clustering for intrusion detection systems in computer networks , 2005, Eng. Appl. Artif. Intell..

[20]  Wei Xu,et al.  Improving one-class SVM for anomaly detection , 2003, Proceedings of the 2003 International Conference on Machine Learning and Cybernetics (IEEE Cat. No.03EX693).

[21]  David G. Stork,et al.  Pattern Classification , 1973 .

[22]  P. Laskov,et al.  Intrusion Detection in Unlabeled Data with Quarter-sphere Support Vector Machines , 2004, Prax. Inf.verarb. Kommun..

[23]  Hongli Zhang,et al.  Intrusion detection alarms reduction using root cause analysis and clustering , 2009, Comput. Commun..

[24]  Peter Mell,et al.  Intrusion Detection Systems , 2001 .