Effects-based feature identification for network intrusion detection

Intrusion detection systems (IDS) are an important element in a network's defences to help protect against increasingly sophisticated cyber attacks. IDS that rely solely on a database of stored known attacks are no longer sufficient for effectively detecting modern day threats. This paper presents a novel anomaly detection technique that can be used to detect previously unknown attacks on a network by identifying attack features. This effects-based feature identification method uniquely combines k-means clustering, Naive Bayes feature selection and C4.5 decision tree classification for pinpointing cyber attacks with a high degree of accuracy in order to increase the situational awareness of cyber network operators.

[1]  Yen-Liang Chen,et al.  Constructing a multi-valued and multi-labeled decision tree , 2003, Expert Syst. Appl..

[2]  Huijun Gao,et al.  A Constrained Evolutionary Computation Method for Detecting Controlling Regions of Cortical Networks , 2012, IEEE/ACM Transactions on Computational Biology and Bioinformatics.

[3]  John McHugh,et al.  Testing Intrusion detection systems: a critique of the 1998 and 1999 DARPA intrusion detection system evaluations as performed by Lincoln Laboratory , 2000, TSEC.

[4]  Yasser Yasami,et al.  A novel unsupervised classification approach for network anomaly detection by k-Means clustering and ID3 decision tree learning methods , 2010, The Journal of Supercomputing.

[5]  Jeffrey Posluns,et al.  Snort 2.0 Intrusion Detection , 2003 .

[6]  Jagath Samarabandu,et al.  Possibilistic decision trees for Intrusion Detection in IEC61850 automated substations , 2009, 2009 International Conference on Industrial and Information Systems (ICIIS).

[7]  Christopher Krügel,et al.  Bayesian event classification for intrusion detection , 2003, 19th Annual Computer Security Applications Conference, 2003. Proceedings..

[8]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[9]  Youlin Shang,et al.  Semi-supervised outlier detection based on fuzzy rough C-means clustering , 2010, Math. Comput. Simul..

[10]  Christin Schäfer,et al.  Learning Intrusion Detection: Supervised or Unsupervised? , 2005, ICIAP.

[11]  Salem Benferhat,et al.  On the combination of naive Bayes and decision trees for intrusion detection , 2005, International Conference on Computational Intelligence for Modelling, Control and Automation and International Conference on Intelligent Agents, Web Technologies and Internet Commerce (CIMCA-IAWTIC'06).

[12]  Huijun Gao,et al.  Distributed Filtering for a Class of Time-Varying Systems Over Sensor Networks With Quantization Errors and Successive Packet Dropouts , 2012, IEEE Transactions on Signal Processing.

[13]  Jun Hu,et al.  Extended Kalman filtering with stochastic nonlinearities and multiple missing measurements , 2012, Autom..

[14]  Anupam Joshi,et al.  Fuzzy clustering for intrusion detection , 2003, The 12th IEEE International Conference on Fuzzy Systems, 2003. FUZZ '03..

[15]  Malcolm I. Heywood,et al.  Selecting Features for Intrusion Detection: A Feature Relevance Analysis on KDD 99 , 2005, PST.

[16]  Hillol Kargupta,et al.  Approximate Distributed K-Means Clustering over a Peer-to-Peer Network , 2009, IEEE Transactions on Knowledge and Data Engineering.

[17]  Philip S. Yu,et al.  Top 10 algorithms in data mining , 2007, Knowledge and Information Systems.

[18]  Peter Rockett,et al.  Multi-class pattern classification using single, multi-dimensional feature-space feature extraction evolved by multi-objective genetic programming and its application to network intrusion detection , 2011, Genetic Programming and Evolvable Machines.

[19]  Syed Haque,et al.  Towards Cyber Defense: Research in Intrusion Detection and Intrusion Prevention Systems , 2010 .

[20]  Feng Jiang,et al.  A Rough Set Based Decision Tree Algorithm and Its Application in Intrusion Detection , 2011, PReMI.

[21]  Anil K. Jain.  Data clustering: 50 years beyond K-means , 2010, Pattern Recognit. Lett..

[22]  T. Velmurugan,et al.  A Survey of Partition based Clustering Algorithms in Data Mining: An Experimental Approach , 2011 .

[23]  Jaekyung Yang,et al.  Optimization-based feature selection with adaptive instance sampling , 2006, Comput. Oper. Res..

[24]  Vir V. Phoha,et al.  K-Means+ID3: A Novel Method for Supervised Anomaly Detection by Cascading K-Means Clustering and ID3 Decision Tree Learning Methods , 2007, IEEE Transactions on Knowledge and Data Engineering.

[25]  Alvaro Soto,et al.  Using data mining techniques to predict industrial wine problem fermentations , 2007 .

[26]  Ajith Abraham,et al.  Modeling intrusion detection system using hybrid intelligent systems , 2007, J. Netw. Comput. Appl..

[27]  Peng Liu,et al.  Using Bayesian networks for cyber security analysis , 2010, 2010 IEEE/IFIP International Conference on Dependable Systems & Networks (DSN).

[28]  Taghi M. Khoshgoftaar,et al.  CLUSTERING-BASED NETWORK INTRUSION DETECTION , 2007 .

[29]  Gulshan Kumar,et al.  The use of artificial intelligence based techniques for intrusion detection: a review , 2010, Artificial Intelligence Review.

[30]  Jan H. P. Eloff,et al.  Information security: The moving target , 2009, Comput. Secur..

[31]  Nandita Sengupta,et al.  Designing of on line intrusion detection system using rough set theory and Q-learning algorithm , 2013, Neurocomputing.

[32]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[33]  Amit Sharma.  Cyber Wars: A Paradigm Shift from Means to Ends , 2010 .

[34]  J. MacQueen.  Some methods for classification and analysis of multivariate observations , 1967 .

[35]  Jugal K. Kalita,et al.  A Survey of Outlier Detection Methods in Network Anomaly Identification , 2011, Comput. J..

[36]  Zhang Yi,et al.  A hierarchical intrusion detection model based on the PCA neural networks , 2007, Neurocomputing.

[37]  Zied Elouedi,et al.  Naive Bayes vs decision trees in intrusion detection systems , 2004, SAC '04.

[38]  Gregory W. Corder,et al.  Nonparametric Statistics for Non-Statisticians: A Step-by-Step Approach , 2009 .

[39]  Yang Li,et al.  Building lightweight intrusion detection system using wrapper-based feature selection mechanisms , 2009, Comput. Secur..

[40]  J. Bezdek,et al.  FCM: The fuzzy c-means clustering algorithm , 1984 .

[41]  Peter J. Denning,et al.  The Profession of IT, Discussing Cyber Attack , 2010 .

[42]  Ali A. Ghorbani,et al.  Y-means: a clustering method for intrusion detection , 2003, CCECE 2003 - Canadian Conference on Electrical and Computer Engineering. Toward a Caring and Humane Technology (Cat. No.03CH37436).

[43]  Shaomin Mu,et al.  High-order Markov kernels for intrusion detection , 2008, Neurocomputing.

[44]  Wolfgang Banzhaf,et al.  The use of computational intelligence in intrusion detection systems: A review , 2010, Appl. Soft Comput..

[45]  Stan Matwin,et al.  Parallelizing Feature Selection , 2006, Algorithmica.

[46]  Tom Kellerman.  Cyber-Threat Proliferation: Today's Truly Pervasive Global Epidemic , 2010, IEEE Security & Privacy.

[47]  Andrew J. Clark,et al.  Data preprocessing for anomaly based network intrusion detection: A review , 2011, Comput. Secur..

[48]  Leonid Portnoy,et al.  Intrusion detection with unlabeled data using clustering , 2000 .

[49]  Philip K. Chan,et al.  An Analysis of the 1999 DARPA/Lincoln Laboratory Evaluation Data for Network Anomaly Detection , 2003, RAID.

[50]  Chih-Fong Tsai,et al.  A triangle area based nearest neighbors approach to intrusion detection , 2010, Pattern Recognit..

[51]  Andreas Fuchsberger,et al.  Intrusion Detection Systems and Intrusion Prevention Systems , 2005, Inf. Secur. Tech. Rep..

[52]  Jung-Min Park,et al.  An overview of anomaly detection techniques: Existing solutions and latest technological trends , 2007, Comput. Networks.

[53]  Michael E. Lesk,et al.  The New Front Line: Estonia under Cyberassault , 2007, IEEE Security & Privacy.

[54]  Wenke Lee,et al.  Intrusion Detection Techniques for Mobile Wireless Networks , 2003, Wirel. Networks.

[55]  Gabriel Maciá-Fernández,et al.  Anomaly-based network intrusion detection: Techniques, systems and challenges , 2009, Comput. Secur..

[56]  Ajith Abraham,et al.  Feature deduction and ensemble design of intrusion detection systems , 2005, Comput. Secur..

[57]  Peter J. Denning,et al.  Discussing cyber attack , 2010, Commun. ACM.

[58]  Amparo Alonso-Betanzos,et al.  A Wrapper Method for Feature Selection in Multiple Classes Datasets , 2009, IWANN.

[59]  J. Hair.  Multivariate data analysis , 1972 .