Effectiveness Evaluation of Data Mining Based IDS

Data mining has been widely applied to the problem of Intrusion Detection in computer networks. However, the misconception of the underlying problem has led to out of context results. This paper shows that factors such as the probability of intrusion and the costs of responding to detected intrusions must be taken into account in order to compare the effectiveness of machine learning algorithms over the intrusion detection domain. Furthermore, we show the advantages of combining different detection techniques. Results regarding the well known 1999 KDD dataset are shown.

[1]  Richard P. Lippmann,et al.  An Overview of Issues in Testing Intrusion Detection Systems , 2003 .

[2]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[3]  Marcus A. Maloof,et al.  Machine Learning and Data Mining for Computer Security , 2006 .

[4]  ElkanCharles.  Results of the KDD'99 classifier learning , 2000 .

[5]  Gürsel Serpen,et al.  KDD Feature Set Complaint Heuristic Rules for R2L Attack Detection , 2003, Security and Management.

[6]  Marcus A. Maloof,et al.  Machine Learning and Data Mining for Computer Security: Methods and Applications (Advanced Information and Knowledge Processing) , 2005 .

[7]  Arturo Ribagorda,et al.  Intrusion Detection Effectiveness Improvement by a Multiagent System , 2005, Int. J. Comput. Sci. Appl..

[8]  Fabio Roli,et al.  Image Analysis and Processing - ICIAP 2005, 13th International Conference, Cagliari, Italy, September 6-8, 2005, Proceedings , 2005, ICIAP.

[9]  J A Swets,et al.  Psychological Science Can Improve Diagnostic Decisions , 2000, Psychological science in the public interest : a journal of the American Psychological Society.

[10]  John E. Gaffney,et al.  Evaluation of intrusion detectors: a decision theory approach , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[11]  George F. Riley,et al.  Intrusion detection testing and benchmarking methodologies , 2003, First IEEE International Workshop on Information Assurance, 2003. IWIAS 2003. Proceedings..

[12]  Salvatore J. Stolfo,et al.  A framework for constructing features and models for intrusion detection systems , 2000, TSEC.

[13]  A. Sen,et al.  Choice Functions and Revealed Preference , 1971 .

[14]  John E. Gaffney,et al.  Evaluation of Intrusion Detection Systems , 2003, Journal of research of the National Institute of Standards and Technology.

[15]  A. H. Murphy,et al.  Economic Value of Weather And Climate Forecasts: Contents , 1997 .

[16]  Robert C. Holte,et al.  Explicitly representing expected cost: an alternative to ROC representation , 2000, KDD '00.

[17]  Bon K. Sy,et al.  Signature-Based Approach for Intrusion Detection , 2005, MLDM.

[18]  Stefan Axelsson,et al.  The base-rate fallacy and the difficulty of intrusion detection , 2000, TSEC.

[19]  Fabio Roli,et al.  Alarm clustering for intrusion detection systems in computer networks , 2005, Eng. Appl. Artif. Intell..

[20]  Charles Elkan,et al.  Results of the KDD'99 classifier learning , 2000, SKDD.

[21]  R.K. Cunningham,et al.  Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[22]  Atsushi Imiya,et al.  Machine Learning and Data Mining in Pattern Recognition: 4th International Conference, MLDM 2005, Leipzig, Germany, July 9-11, 2005, Proceedings (Lecture ... / Lecture Notes in Artificial Intelligence) , 2005 .

[23]  Fabio Roli,et al.  Fusion of multiple classifiers for intrusion detection in computer networks , 2003, Pattern Recognit. Lett..

[24]  Christin Schäfer,et al.  Learning Intrusion Detection: Supervised or Unsupervised? , 2005, ICIAP.

[25]  John E. Gaffney,et al.  A Decision Analysis Method for Evaluating Computer Intrusion Detection Systems , 2004, Decis. Anal..

[26]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.

[27]  Arturo Ribagorda,et al.  Fuzzy logic on decision model for IDS , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[28]  Richard Lippmann,et al.  The 1999 DARPA off-line intrusion detection evaluation , 2000, Comput. Networks.

[29]  Ian Witten,et al.  Data Mining , 2000 .

[30]  Daniel Rousseau,et al.  Economic value of weather and Climate forecasts , 1998 .