DDoS Detection System: Utilizing Gradient Boosting Algorithm and Apache Spark

Distributed Denial of Service (DDoS) is one of the major threats to the Internet security. Various DDoS attacks have been reported against many organizations in recent years. There have been numerous studies investigating the effects of utilizing classification algorithms to detect and prevent DDoS attacks. However, the existing research has many obstacles including the achievement of practical performance rates of the detection system, the delay of detection, as well as the ability to deal with the large dataset. In this research, we propose a DDoS detection framework that mainly consists of Gradient Boosting classification algorithm (GBT) and the Apache Processing Engine Spark. Experimental results conducted in a Spark and Hadoop cluster, for evaluating the proposed framework regarding the performances as well as the delays using a real DDoS Dataset, show that the integration of the GBT algorithm with Apache Spark works excellently to detect DDoS attack. The volume of the dataset and the features space, as well as the depth of decision trees and number of iterations parameters, have a direct impact on the GBT algorithm performance rates and the delays.

[1]  Fred A. Hamprecht,et al.  Cost efficient gradient boosting , 2017, NIPS.

[2]  Jessica Granderson,et al.  Gradient boosting machine for modeling the energy consumption of commercial buildings , 2018 .

[3]  Ebrahim A. Gharavol,et al.  A Novel DoS and DDoS Attacks Detection Algorithm Using ARIMA Time Series Model and Chaotic System in Computer Networks , 2016, IEEE Communications Letters.

[4]  Bo Du,et al.  Scene Classification via a Gradient Boosting Random Convolutional Network Framework , 2016, IEEE Transactions on Geoscience and Remote Sensing.

[5]  Masaharu Munetomo,et al.  Distributed denial of services attack protection system with genetic algorithms on Hadoop cluster computing framework , 2015, 2015 IEEE Congress on Evolutionary Computation (CEC).

[6]  P. J. Criscuolo.  Distributed Denial of Service Tools, Trin00, Tribe Flood Network, Tribe Flood Network 2000 and Stacheldraht. , 2000 .

[7]  K. M Prasad,et al.  Discriminating DDoS Attack traffic from Flash Crowds on Internet Threat Monitors (ITM) Using Entropy variations , 2013 .

[8]  Matt J. Kusner,et al.  Classifier cascades and trees for minimizing feature evaluation cost , 2014, J. Mach. Learn. Res..

[9]  Eric R. Ziegel,et al.  The Elements of Statistical Learning , 2003, Technometrics.

[10]  Bin Li,et al.  A Host-Based Intrusion Detection System Using Architectural Features to Improve Sophisticated Denial-of-Service Attack Detections , 2010, Int. J. Inf. Secur. Priv..

[11]  Xiaohong Huang,et al.  A DDoS Attack Detection Method Based on Hybrid Heterogeneous Multiclassifier Ensemble Learning , 2017, J. Electr. Comput. Eng..

[12]  John T. Ormerod,et al.  Wavelet-based gradient boosting , 2016, Stat. Comput..

[13]  S. Selvakumar,et al.  Detection of distributed denial of service attacks using an ensemble of adaptive and hybrid neuro-fuzzy systems , 2013, Comput. Commun..

[14]  Emin Anarim,et al.  Frequency based DDoS attack detection approach using naive Bayes classification , 2016, 2016 39th International Conference on Telecommunications and Signal Processing (TSP).

[15]  Rida Khatoun,et al.  Understanding botclouds from a system perspective: A principal component analysis , 2014, 2014 IEEE Network Operations and Management Symposium (NOMS).