DNIDS: A dependable network intrusion detection system using the CSI-KNN algorithm

The dependability of an Intrusion Detection System (IDS) relies on two factors: ability to detect intrusions and survivability in hostile environments. Machine learningbased anomaly detection approaches are gaining increasing attention in the network intrusion detection community because of their intrinsic ability to discover novel attacks. This ability has become critical since the number of new attacks has kept growing in recent years. However, most of today’s anomaly-based IDSs generate high false positive rates and miss many attacks because of a deficiency in their ability to discriminate attacks from legitimate behaviors. These unreliable results damage the dependability of IDSs. In addition, even if the detection method is sound and effective, the IDS might still be unable to deliver detection service when under attack. With the increasing importance of the IDS, some attackers attempt to disable the IDS before they launch a thorough attack. In this thesis, we propose a Dependable Network Intrusion Detection System (DNIDS) based on the Combined Strangeness and Isolation measure K-Nearest Neighbor (CSI-KNN) algorithm. The DNIDS can effectively detect network intrusions while providing continued service even under attacks. The intrusion detection algorithm analyzes different characteristics of network data by employing two measures:

[1]  Fabio A. González,et al.  An immunity-based technique to characterize intrusions in computer networks , 2002, IEEE Trans. Evol. Comput..

[2]  Belur V. Dasarathy,et al.  Nearest neighbor (NN) norms: NN pattern classification techniques , 1991 .

[3]  John McHugh,et al.  Defending Yourself: The Role of Intrusion Detection Systems , 2000, IEEE Software.

[4]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[5]  Simon Pietro Romano,et al.  Real Time Detection of Novel Attacks by Means of Data Mining Techniques , 2005, ICEIS.

[6]  Richard A. Kemmerer,et al.  NSTAT: A Model-based Real-time Network Intrusion Detection System , 1998 .

[7]  Salvatore J. Stolfo,et al.  A framework for constructing features and models for intrusion detection systems , 2000, TSEC.

[8]  Biswanath Mukherjee,et al.  DIDS (distributed intrusion detection system)—motivation, architecture, and an early prototype , 1997 .

[9]  Jon Louis Bentley,et al.  K-d trees for semidynamic point sets , 1990, SCG '90.

[10]  Zheng Zhang,et al.  HIDE : a Hierarchical Network Intrusion Detection System Using Statistical Preprocessing and Neural Network Classification , 2001 .

[11]  Pankaj Jalote,et al.  Fault tolerance in distributed systems , 1994 .

[12]  Deborah A. Frincke,et al.  Towards survivable intrusion detection system , 2004, 37th Annual Hawaii International Conference on System Sciences, 2004. Proceedings of the.

[13]  Peng Liu.  Architectures for intrusion tolerant database systems , 2003, Foundations of Intrusion Tolerant Systems, 2003 [Organically Assured and Survivable Information Systems].

[14]  Li Guo,et al.  Network anomaly detection based on TCM-KNN algorithm , 2007, ASIACCS '07.

[15]  H. J. Arnold.  Introduction to the Practice of Statistics , 1990 .

[16]  Zair Abdelouahab,et al.  A fault tolerance mechanism for network intrusion detection system based on intelligent agents (NIDIA) , 2006, The Fourth IEEE Workshop on Software Technologies for Future Embedded and Ubiquitous Systems, and the Second International Workshop on Collaborative Computing, Integration, and Assurance (SEUS-WCCIA'06).

[17]  Feiyi Wang,et al.  SITAR: a scalable intrusion-tolerant architecture for distributed services , 2003, Foundations of Intrusion Tolerant Systems, 2003 [Organically Assured and Survivable Information Systems].

[18]  Aurobindo Sundaram,et al.  An introduction to intrusion detection , 1996, CROS.

[19]  Sebastiaan Tesink,et al.  Improving Intrusion Detection Systems through Machine Learning , 2007 .

[20]  Koral Ilgun,et al.  USTAT: a real-time intrusion detection system for UNIX , 1993, Proceedings 1993 IEEE Computer Society Symposium on Research in Security and Privacy.

[21]  Richard A. Kemmerer,et al.  State Transition Analysis: A Rule-Based Intrusion Detection Approach , 1995, IEEE Trans. Software Eng..

[22]  Harry Wechsler,et al.  Open World Face Recognition with Credibility and Confidence Measures , 2003, AVBPA.

[23]  Marc Dacier,et al.  A revised taxonomy for intrusion-detection systems , 2000, Ann. des Télécommunications.

[24]  Brian Randell,et al.  Fundamental Concepts of Dependability , 2000 .

[25]  Alfonso Valdes,et al.  Next-generation Intrusion Detection Expert System (NIDES)A Summary , 1997 .

[26]  Sadie Creese,et al.  Conceptual Model and Architecture of MAFTIA , 2003 .

[27]  Mohammad Zulkernine,et al.  Random-Forests-Based Network Intrusion Detection Systems , 2008, IEEE Transactions on Systems, Man, and Cybernetics, Part C (Applications and Reviews).

[28]  Sara Matzner,et al.  An application of machine learning to network intrusion detection , 1999, Proceedings 15th Annual Computer Security Applications Conference (ACSAC'99).

[29]  Daniel Barbará,et al.  Detecting outliers using transduction and statistical testing , 2006, KDD '06.

[30]  Todd L. Heberlein,et al.  Network intrusion detection , 1994, IEEE Network.

[31]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[32]  Ernest Koh,et al.  Reliability Block Diagrams , 2006 .

[33]  Vladimir Vovk,et al.  Criterion of Calibration for Transductive Confidence Machine with Limited Feedback , 2003, ALT.

[34]  W.-T. Tsai,et al.  Attack tolerant enhancement of intrusion detection systems , 2000, MILCOM 2000 Proceedings. 21st Century Military Communications. Architectures and Technologies for Information Superiority (Cat. No.00CH37155).

[35]  Marc Dacier,et al.  Design of an Intrusion-Tolerant Intrusion Detection System , 2002 .

[36]  Yuh-Jye Lee,et al.  A three-tier IDS via data mining approach , 2007, MineNet '07.

[37]  V. Rao Vemuri,et al.  Use of K-Nearest Neighbor classifier for intrusion detection , 2002, Comput. Secur..

[38]  Paul Helman,et al.  Statistical Foundations of Audit Trail Analysis for the Detection of Computer Misuse , 1993, IEEE Trans. Software Eng..

[39]  Giovanni Vigna,et al.  NetSTAT: a network-based intrusion detection approach , 1998, Proceedings 14th Annual Computer Security Applications Conference (Cat. No.98EX217).

[40]  Karl N. Levitt,et al.  Execution monitoring of security-critical programs in distributed systems: a specification-based approach , 1997, Proceedings. 1997 IEEE Symposium on Security and Privacy (Cat. No.97CB36097).

[41]  Fabio A. González,et al.  An Intelligent Decision Support System for Intrusion Detection and Response , 2001, MMM-ACNS.

[42]  Salvatore J. Stolfo,et al.  A data mining framework for building intrusion detection models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[43]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[44]  Peter G. Neumann,et al.  EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances , 1997, CCS 2002.

[45]  Salvatore J. Stolfo,et al.  Real time data mining-based intrusion detection , 2001, Proceedings DARPA Information Survivability Conference and Exposition II. DISCEX'01.

[46]  Alexander Gammerman,et al.  Transductive Confidence Machines for Pattern Recognition , 2002, ECML.

[47]  Thomas Henry Ptacek,et al.  Insertion, Evasion, and Denial of Service: Eluding Network Intrusion Detection , 1998 .

[48]  John C. Knight,et al.  Towards Survivable Intrusion Detection , 2000 .

[49]  Rong Wang.  Intrusion Tolerant Systems Characterization and Acceptance Monitor Design , 2001 .

[50]  Mohammad Zulkernine,et al.  A hybrid network intrusion detection technique using random forests , 2006, First International Conference on Availability, Reliability and Security (ARES'06).

[51]  Yihua Liao,et al.  Machine learning in intrusion detection , 2005 .