Analysis of Feature Selection Techniques for Correlation-Based Network Anomaly Detection

Although the application of feature selection techniques is common practice for performance enhancements of network anomaly detection techniques, it has rarely been used for recently proposed multivariate correlation-based methods. This paper investigates qualitatively why this is the case and evaluates several feature selection techniques empirically in combination with the state-of-the-art multivariate correlation anomaly detection technique on the UNSW-NB15 network intrusion detection dataset. Our results show that there is a significant enhancement possibility using feature selection techniques. Furthermore, we believe that if the underlying nature of multivariate correlation techniques, temporal correlation changes between attributes, is considered in the selection process the performance could be further improved.

[1]  Li Guo,et al.  Survey and Taxonomy of Feature Selection Algorithms in Intrusion Detection System , 2006, Inscrypt.

[2]  Martin May,et al.  Applying PCA for Traffic Anomaly Detection: Problems and Solutions , 2009, IEEE INFOCOM 2009.

[3]  Tai-hoon Kim,et al.  Linear Correlation-Based Feature Selection for Network Intrusion Detection Model , 2013, SecNet.

[4]  Ali A. Ghorbani,et al.  A detailed analysis of the KDD CUP 99 data set , 2009, 2009 IEEE Symposium on Computational Intelligence for Security and Defense Applications.

[5]  Xiangjian He,et al.  Intrusion detection method based on nonlinear correlation measure , 2014, Int. J. Internet Protoc. Technol..

[6]  Achim P. Karduck,et al.  SIM in light of big data , 2015, 2015 11th International Conference on Innovations in Information Technology (IIT).

[7]  D. S. Yeung,et al.  Network intrusion detection in covariance feature space , 2007, Pattern Recognit..

[8]  Paul Geladi,et al.  Principal Component Analysis , 1987, Comprehensive Chemometrics.

[9]  Achim P. Karduck,et al.  Securing Smart Cities—A Big Data Challenge , 2018 .

[10]  Vyas Sekar,et al.  An empirical evaluation of entropy-based traffic anomaly detection , 2008, IMC '08.

[11]  Jill Slay,et al.  The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set , 2016, Inf. Secur. J. A Glob. Perspect..

[12]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[13]  Xiangjian He,et al.  A System for Denial-of-Service Attack Detection Based on Multivariate Correlation Analysis , 2014, IEEE Transactions on Parallel and Distributed Systems.

[14]  Xiangjian He,et al.  A System for Denial-of-Service Attack Detection Based on Multivariate Correlation Analysis , 2011, IEEE Transactions on Parallel and Distributed Systems.

[15]  Xiangjian He,et al.  Triangle-Area-Based Multivariate Correlation Analysis for Effective Denial-of-Service Attack Detection , 2012, 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications.

[16]  Mark A. Hall,et al.  Correlation-based Feature Selection for Machine Learning , 2003 .

[17]  Xiangjian He,et al.  Detection of Denial-of-Service Attacks Based on Computer Vision Techniques , 2015, IEEE Transactions on Computers.

[18]  Daniel S. Yeung,et al.  A covariance analysis model for DDoS attack detection , 2004, 2004 IEEE International Conference on Communications (IEEE Cat. No.04CH37577).

[19]  Nour Moustafa,et al.  UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set) , 2015, 2015 Military Communications and Information Systems Conference (MilCIS).

[20]  Xiangjian He,et al.  An Intrusion Detection System Based on Polynomial Feature Correlation Analysis , 2017, 2017 IEEE Trustcom/BigDataSE/ICESS.