A hybrid intrusion detection system

Anomaly intrusion detection normally has high false alarm rates, and a high volume of false alarms will prevent system administrators identifying the real attacks. Machine learning methods provide an effective way to decrease the false alarm rate and improve the detection rate of anomaly intrusion detection. In this research, we propose a novel approach using kernel methods and Support Vector Machine (SVM) for improving anomaly intrusion detectors' accuracy. Two kernels, STIDE kernel and Markov Chain kernel, are developed specially for intrusion detection applications. The experiments show the STIDE and Markov Chain kernel based two class SVM anomaly detectors have better accuracy rate than the original STIDE and Markov Chain anomaly detectors. Generally, anomaly intrusion detection approaches build normal profiles from labeled training data. However, labeled training data for intrusion detection is expensive and not easy to obtain. We propose an anomaly detection approach, using STIDE kernel and Markov Chain kernel based one class SVM, that does not need labeled training data. To further increase the detection rate and lower the false alarm rate, an approach of integrating specification based intrusion detection with anomaly intrusion detection is also proposed. This research also establish a platform which generates automatically both misuse and anomaly intrusion detection software agents. In our method, a SIFT representing an intrusion is automatically converted to a Colored Petri Net (CPNs) representing an intrusion detection template, subsequently, the CPN is compiled into code for misuse intrusion detection software agents using a compiler and dynamically loaded and launched for misuse intrusion detection. On the other hand, a model representing a normal profile is automatically generated from training data, subsequently, an anomaly intrusion detection agent which carries this model is generated and launched for anomaly intrusion detection. By engaging both misuse and anomaly intrusion detection agents, our system can detect known attacks as well as novel unknown attacks.

[1]  Vasant Honavar,et al.  Towards the automatic generation of mobile agents for distributed intrusion detection system , 2006, J. Syst. Softw..

[2]  Raj Jain,et al.  The art of computer systems performance analysis - techniques for experimental design, measurement, simulation, and modeling , 1991, Wiley professional computing.

[3]  Emin Anarim,et al.  An intelligent intrusion detection system (IDS) for anomaly and misuse detection in computer networks , 2005, Expert Syst. Appl..

[4]  Salvatore J. Stolfo,et al.  Cost-based modeling for fraud and intrusion detection: results from the JAM project , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[5]  Mohammad Zulkernine,et al.  A hybrid network intrusion detection technique using random forests , 2006, First International Conference on Availability, Reliability and Security (ARES'06).

[6]  Vasant Honavar,et al.  Feature Selection Using a Genetic Algorithm for Intrusion Detection , 1999, GECCO.

[7]  Mehryar Mohri,et al.  Positive Definite Rational Kernels , 2003, COLT.

[8]  Bernhard E. Boser,et al.  A training algorithm for optimal margin classifiers , 1992, COLT '92.

[9]  James F. Allen,et al.  Actions and Events in Interval Temporal Logic , 1994, J. Log. Comput..

[10]  Hervé Debar,et al.  A serial combination of anomaly and misuse IDSes applied to HTTP traffic , 2004, 20th Annual Computer Security Applications Conference.

[11]  Kymie M. C. Tan,et al.  Anomaly Detection in Embedded Systems , 2002, IEEE Trans. Computers.

[12]  Vern Paxson,et al.  Empirically derived analytic models of wide-area TCP connections , 1994, TNET.

[13]  Andrew S. Miner,et al.  Anomaly intrusion detection using one class SVM , 2004, Proceedings from the Fifth Annual IEEE SMC Information Assurance Workshop, 2004..

[14]  John J. Kinney.  Statistics for Science and Engineering , 2001 .

[15]  John McHugh,et al.  The 1998 Lincoln Laboratory IDS Evaluation , 2000, Recent Advances in Intrusion Detection.

[16]  Peter Mell,et al.  A denial-of-service resistant intrusion detection architecture , 2000, Comput. Networks.

[17]  Leonid Portnoy,et al.  Intrusion detection with unlabeled data using clustering , 2000 .

[18]  M Damashek,et al.  Gauging Similarity with n-Grams: Language-Independent Categorization of Text , 1995, Science.

[19]  A.H. Sung,et al.  Identifying important features for intrusion detection using support vector machines and neural networks , 2003, 2003 Symposium on Applications and the Internet, 2003. Proceedings..

[20]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.

[21]  David Moore,et al.  The CoralReef Software Suite as a Tool for System and Network Administrators , 2001, LISA.

[22]  Dipankar Dasgupta,et al.  Anomaly detection in multidimensional data using negative selection algorithm , 2002, Proceedings of the 2002 Congress on Evolutionary Computation. CEC'02 (Cat. No.02TH8600).

[23]  R. Sekar,et al.  A fast automaton-based method for detecting anomalous program behaviors , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[24]  Ron Kohavi,et al.  Scaling Up the Accuracy of Naive-Bayes Classifiers: A Decision-Tree Hybrid , 1996, KDD.

[25]  B. Scholkopf,et al.  Fisher discriminant analysis with kernels , 1999, Neural Networks for Signal Processing IX: Proceedings of the 1999 IEEE Signal Processing Society Workshop (Cat. No.98TH8468).

[26]  M. Bar-Hillel.  The base-rate fallacy in probability judgments. , 1980 .

[27]  Arie van Deursen,et al.  Domain-specific languages: an annotated bibliography , 2000, SIGP.

[28]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[29]  Carrie Gates,et al.  Challenging the anomaly detection paradigm: a provocative discussion , 2006, NSPW '06.

[30]  Christopher Krügel,et al.  Anomaly detection of web-based attacks , 2003, CCS '03.

[31]  John E. Gaffney,et al.  Evaluation of Intrusion Detection Systems , 2003, Journal of research of the National Institute of Standards and Technology.

[32]  D. Vere-Jones.  Markov Chains , 1972, Nature.

[33]  Kishor S. Trivedi,et al.  Power-hierarchy of dependability-model types , 1994 .

[34]  Michalis Faloutsos,et al.  Is P2P dying or just hiding? [P2P traffic measurement] , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[35]  Guy Helmer,et al.  Intelligent multi-agent system for intrusion detection and countermeasures , 2000 .

[36]  M. Shyu,et al.  A Novel Anomaly Detection Scheme Based on Principal Component Classifier , 2003 .

[37]  Wei Li,et al.  Using Genetic Algorithm for Network Intrusion Detection , 2004 .

[38]  Charles P. Pfleeger,et al.  Security in computing , 1988 .

[39]  Somesh Jha,et al.  Markov chains, classifiers, and intrusion detection , 2001, Proceedings. 14th IEEE Computer Security Foundations Workshop, 2001..

[40]  Renata Teixeira,et al.  Early application identification , 2006, CoNEXT '06.

[41]  Gürsel Serpen,et al.  Application of Machine Learning Algorithms to KDD Intrusion Detection Dataset within Misuse Detection Context , 2003, MLMTA.

[42]  Kari Sentz,et al.  Combination of Evidence in Dempster-Shafer Theory , 2002 .

[43]  Sven Ehlert,et al.  Analysis and Signature of Skype VoIP Session Traffic , 2006 .

[44]  Andrew H. Sung,et al.  Intrusion detection using neural networks and support vector machines , 2002, Proceedings of the 2002 International Joint Conference on Neural Networks. IJCNN'02 (Cat. No.02CH37290).

[45]  Sebastian Zander,et al.  A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification , 2006, CCRV.

[46]  Michael Schatz,et al.  Learning Program Behavior Profiles for Intrusion Detection , 1999, Workshop on Intrusion Detection and Network Monitoring.

[47]  Simson L. Garfinkel,et al.  Practical UNIX Security , 1991 .

[48]  R. Sekar,et al.  Experiences with Specification-Based Intrusion Detection , 2001, Recent Advances in Intrusion Detection.

[49]  Bernhard Schölkopf,et al.  Estimating the Support of a High-Dimensional Distribution , 2001, Neural Computation.

[50]  S. E. Smaha.  Haystack: an intrusion detection system , 1988, [Proceedings 1988] Fourth Aerospace Computer Security Applications.

[51]  Vasant Honavar,et al.  A Software Fault Tree Approach to Requirements Analysis of an Intrusion Detection System , 2002, Requirements Engineering.

[52]  T. Lane,et al.  Sequence Matching and Learning in Anomaly Detection for Computer Security , 1997 .

[53]  Eugene H. Spafford,et al.  Intrusion detection using autonomous agents , 2000, Comput. Networks.

[54]  Aurobindo Sundaram,et al.  An introduction to intrusion detection , 1996, CROS.

[55]  David A. Wagner,et al.  Intrusion detection via static analysis , 2001, Proceedings 2001 IEEE Symposium on Security and Privacy. S&P 2001.

[56]  Salvatore J. Stolfo,et al.  A framework for constructing features and models for intrusion detection systems , 2000, TSEC.

[57]  James P. Egan,et al.  Signal detection theory and ROC analysis , 1975 .

[58]  Jia Wang,et al.  Analyzing peer-to-peer traffic across large networks , 2002, IMW '02.

[59]  Rajendra P. Srivastava.  Alternative form of Dempster's rule for binary variables , 2005, Int. J. Intell. Syst..

[60]  Phillip A. Porras,et al.  STAT -- A State Transition Analysis Tool For Intrusion Detection , 1993 .

[61]  Bruce W. Suter,et al.  The multilayer perceptron as an approximation to a Bayes optimal discriminant function , 1990, IEEE Trans. Neural Networks.

[62]  Oliver Spatscheck,et al.  Accurate, scalable in-network identification of p2p traffic using application signatures , 2004, WWW '04.

[63]  Peter G. Neumann,et al.  EMERALD: Event Monitoring Enabling Responses to Anomalous Live Disturbances , 1997, CCS 2002.

[64]  Anthony McGregor,et al.  Flow Clustering Using Machine Learning Techniques , 2004, PAM.

[65]  Audun Jøsang,et al.  The consensus operator for combining beliefs , 2002, Artif. Intell..

[66]  John S. Baras,et al.  B-ROC Curves for the Assessment of Classifiers over Imbalanced Data Sets , 2006, AAAI.

[67]  Julie Greensmith,et al.  Immune system approaches to intrusion detection – a review , 2004, Natural Computing.

[68]  Eleazar Eskin,et al.  The Spectrum Kernel: A String Kernel for SVM Protein Classification , 2001, Pacific Symposium on Biocomputing.

[69]  Salvatore J. Stolfo,et al.  A data mining framework for building intrusion detection models , 1999, Proceedings of the 1999 IEEE Symposium on Security and Privacy (Cat. No.99CB36344).

[70]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[71]  Vinod Yegneswaran,et al.  Internet intrusions: global characteristics and prevalence , 2003, SIGMETRICS '03.

[72]  Patrick Haffner,et al.  ACAS: automated construction of application signatures , 2005, MineNet '05.

[73]  Dorothy E. Denning,et al.  An Intrusion-Detection Model , 1987, IEEE Transactions on Software Engineering.

[74]  Stephanie Forrest,et al.  Infect Recognize Destroy , 1996 .

[75]  Nello Cristianini,et al.  Classification using String Kernels , 2000 .

[76]  M. Reilly,et al.  Open infrastructure for scalable intrusion detection , 1998, 1998 IEEE Information Technology Conference, Information Environment for the Future (Cat. No.98EX228).

[77]  Alfonso Valdes,et al.  Adaptive, Model-Based Monitoring for Cyber Attack Detection , 2000, Recent Advances in Intrusion Detection.

[78]  Kymie M. C. Tan,et al.  "Why 6?" Defining the operational limits of stide, an anomaly-based intrusion detector , 2002, Proceedings 2002 IEEE Symposium on Security and Privacy.

[79]  John S. Baras,et al.  A framework for the evaluation of intrusion detection systems , 2006, 2006 IEEE Symposium on Security and Privacy (S&P'06).

[80]  R.K. Cunningham,et al.  Evaluating intrusion detection systems: the 1998 DARPA off-line intrusion detection evaluation , 2000, Proceedings DARPA Information Survivability Conference and Exposition. DISCEX'00.

[81]  Jerzy W. Rozenblit,et al.  A hybrid intrusion detection and visualization system , 2006, 13th Annual IEEE International Symposium and Workshop on Engineering of Computer-Based Systems (ECBS'06).

[82]  Biswanath Mukherjee,et al.  A system for distributed intrusion detection , 1991, COMPCON Spring '91 Digest of Papers.

[83]  Naji Habra,et al.  Distributed audit trail analysis , 1995, Proceedings of the Symposium on Network and Distributed System Security.

[84]  Vipin Kumar,et al.  Predicting rare classes: can boosting make any weak learner strong? , 2002, KDD.

[85]  Cheng Xiang,et al.  Design of Multiple-Level Hybrid Classifier for Intrusion Detection System , 2005, 2005 IEEE Workshop on Machine Learning for Signal Processing.

[86]  Karl N. Levitt,et al.  Automated detection of vulnerabilities in privileged programs by execution monitoring , 1994, Tenth Annual Computer Security Applications Conference.

[87]  Pat Langley,et al.  Estimating Continuous Distributions in Bayesian Classifiers , 1995, UAI.

[88]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[89]  Eugene H. Spafford,et al.  A PATTERN MATCHING MODEL FOR MISUSE INTRUSION DETECTION , 1994 .

[90]  Dipankar Dasgupta,et al.  A comparison of negative and positive selection algorithms in novel pattern detection , 2000, Smc 2000 conference proceedings. 2000 ieee international conference on systems, man and cybernetics. 'cybernetics evolving to systems, humans, organizations, and their complex interactions' (cat. no.0.

[91]  G. S. Hura,et al.  The use of Petri nets to analyze coherent fault trees , 1988 .

[92]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[93]  John J. Snyder,et al.  End-user systems, reusability, and high-level design , 1994 .

[94]  Zhi-Li Zhang,et al.  Profiling internet backbone traffic: behavior models and applications , 2005, SIGCOMM '05.

[95]  Renata Teixeira,et al.  Traffic classification on the fly , 2006, CCRV.

[96]  Stefan Axelsson,et al.  The base-rate fallacy and the difficulty of intrusion detection , 2000, TSEC.

[97]  Somesh Jha,et al.  Formalizing sensitivity in static analysis for intrusion detection , 2004, IEEE Symposium on Security and Privacy, 2004. Proceedings. 2004.

[98]  Nancy G. Leveson,et al.  Safeware: System Safety and Computers , 1995 .

[99]  Thorsten Joachims,et al.  Making large scale SVM learning practical , 1998 .

[100]  Gunnar Rätsch,et al.  Kernel PCA and De-Noising in Feature Spaces , 1998, NIPS.

[101]  Harold S. Javitz,et al.  The SRI IDES statistical anomaly detector , 1991, Proceedings. 1991 IEEE Computer Society Symposium on Research in Security and Privacy.

[102]  Kishor S. Trivedi,et al.  Dependability modeling using Petri-nets , 1995 .

[103]  Leo Breiman,et al.  Random Forests , 2001, Machine Learning.

[104]  Peter B. Danzig,et al.  tcplib: A Library of TCP Internetwork Traffic Characteristics , 2002 .

[105]  Salvatore J. Stolfo,et al.  Anomalous Payload-Based Network Intrusion Detection , 2004, RAID.

[106]  Susan M. Bridges,et al.  FUZZY DATA MINING AND GENETIC ALGORITHMS APPLIED TO INTRUSION DETECTION , 2002 .

[107]  Glenn Shafer,et al.  A Mathematical Theory of Evidence , 2020, A Mathematical Theory of Evidence.

[108]  Mehryar Mohri.  Edit-Distance Of Weighted Automata: General Definitions And Algorithms , 2003, Int. J. Found. Comput. Sci..

[109]  Christopher J. C. Burges,et al.  A Tutorial on Support Vector Machines for Pattern Recognition , 1998, Data Mining and Knowledge Discovery.

[110]  Vladimir Vapnik,et al.  Statistical learning theory , 1998 .

[111]  Andrew W. Moore,et al.  X-means: Extending K-means with Efficient Estimation of the Number of Clusters , 2000, ICML.