An Improved Parallel Network Traffic Anomaly Detection Method Based on Bagging and GRU

In the current large-scale and complex network environment, the types of networks are gradually diversified and the scale is constantly expanding. The network traffic has increased dramatically, and has the characteristics of the high-dimensional multivariable structure, which makes network traffic anomaly detection more and more difficult. Therefore, the paper proposes an improved parallel network traffic anomaly detection method based on Bagging and GRU (PB-GRU). This method uses GRU deep neural network to perform efficient hierarchical feature representation and learn the time-dependent characteristics of network traffic data to achieve more accurate detection. Then use Spark technology to process the training and testing of GRU detector in parallel to improve the overall performance. In order to reduce the individual differences between parallel detectors and improve the generalization error, and Bagging algorithm is used to improve the training process of GRU detector, so that the combined GRU detector has better detection performance. Experimental results show that the proposed method achieves a detection accuracy of 99.6\(\%\), and the error rate is only 0.0036\(\%\). In addition, after parallel processing with Spark, the overall efficiency and scalability have been improved.

[1]  Jiguo Yu,et al.  Latent-Data Privacy Preserving With Customized Data Utility for Social Network Data , 2018, IEEE Transactions on Vehicular Technology.

[2]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[3]  James Z. Zhang,et al.  Network traffic anomaly detection using weighted self-similarity based on EMD , 2013, 2013 Proceedings of IEEE Southeastcon.

[4]  Pengju Liu,et al.  Network Traffic Anomaly Detection Based on Information Gain and Deep Learning , 2019, ICISDM.

[5]  Yoshua Bengio,et al.  On the Properties of Neural Machine Translation: Encoder–Decoder Approaches , 2014, SSST@EMNLP.

[6]  Chin-Tser Huang,et al.  Wavelet-based Real Time Detection of Network Traffic Anomalies , 2006, 2006 Securecomm and Workshops.

[7]  Sung-Bae Cho,et al.  Web traffic anomaly detection using C-LSTM neural networks , 2018, Expert Syst. Appl..

[8]  Lei Yu,et al.  Cooperative end-to-end traffic redundancy elimination for reducing cloud bandwidth cost , 2017, 2012 20th IEEE International Conference on Network Protocols (ICNP).

[9]  Tim Moors,et al.  Detection and Identification of Anomalies in Wireless Mesh Networks Using Principal Component Analysis (PCA) , 2008, 2008 International Symposium on Parallel Architectures, Algorithms, and Networks (i-span 2008).

[10]  Bu-Sung Lee,et al.  Detection of network anomalies using Improved-MSPCA with sketches , 2017, Comput. Secur..

[11]  Wanwei Huang,et al.  Network traffic anomaly detection based on self-similarity using FRFT , 2013, 2013 IEEE 4th International Conference on Software Engineering and Service Science.

[12]  Petter Holme,et al.  Efficient local strategies for vaccination and network attack , 2004, q-bio/0403021.

[13]  Yoshua Bengio,et al.  Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation , 2014, EMNLP.

[14]  Jill Slay,et al.  The evaluation of Network Anomaly Detection Systems: Statistical analysis of the UNSW-NB15 data set and the comparison with the KDD99 data set , 2016, Inf. Secur. J. A Glob. Perspect..

[15]  Donald F. Towsley,et al.  Detecting anomalies in network traffic using maximum entropy estimation , 2005, IMC '05.

[16]  Hiroaki Hazeyama,et al.  Detecting anomalies in massive traffic with sketches , 2014, CFI '14.

[17]  K. P. Soman,et al.  Evaluation of Recurrent Neural Network and its Variants for Intrusion Detection System (IDS) , 2017, Int. J. Inf. Syst. Model. Des..

[18]  Matthew V. Mahoney,et al.  Network traffic anomaly detection based on packet bytes , 2003, SAC '03.

[19]  Kun Xie,et al.  A new evolutionary neural networks based on intrusion detection systems using multiverse optimization , 2017, Applied Intelligence.

[20]  Bu-Sung Lee,et al.  A novel anomaly detection system using feature-based MSPCA with sketch , 2017, 2017 26th Wireless and Optical Communication Conference (WOCC).

[21]  Fang Liu,et al.  Real-time anomaly traffic monitoring based on dynamic k-NN cumulative-distance abnormal detection algorithm , 2014, 2014 IEEE 3rd International Conference on Cloud Computing and Intelligence Systems.

[22]  Yennun Huang,et al.  Network traffic anomaly detection based on growing hierarchical SOM , 2013, 2013 43rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks (DSN).

[23]  Nour Moustafa,et al.  UNSW-NB15: a comprehensive data set for network intrusion detection systems (UNSW-NB15 network data set) , 2015, 2015 Military Communications and Information Systems Conference (MilCIS).