Apache Hadoop Based Distributed Denial of Service Detection Framework

Distributed Denial of Service (DDoS) attack is one of the most powerful and immense threats to internet-based services. It hinders the victim services within a short duration of time by overwhelming with the huge amount of attack traffic. A sophisticated attacker closely follows the current research of DDoS defense, perform a sophisticated attack by compromising millions of unsecured devices, and send a huge amount of attack traffic (Big Data) to destroy a victim. The attack volume size pattern is shifted to Terabits per second (Tbps) from Gigabits per second (Gbps). When a large amount of traffic is processed by the defense system to identify attack traffic, seldom defense system itself can become a victim of DDoS attack. Therefore, there is a demand to implement DDoS defense system which can efficiently process a massive amount of network traffic and immediately distinguish attack traffic. In this paper, we propose a victim-end Hadoop based DDoS defense framework to identify an attack using the MapReduce programming model based on information theory metric. Further, we have implemented Hadoop based DDoS testbed and validated proposed framework using real datasets, such as, MIT Lincoln LLDDoS1.0, CAIDA and live traffic generated using testbed. The experimental result of proposed framework shows higher detection accuracy (average detection accuracy is 97%).

[1]  Jugal K. Kalita,et al.  E-LDAT: a lightweight system for DDoS flooding attack detection and IP traceback using extended entropy metric , 2016, Secur. Commun. Networks.

[2]  Jugal K. Kalita,et al.  Detecting Distributed Denial of Service Attacks: Methods, Tools and Future Directions , 2014, Comput. J..

[3]  Rana Khattak,et al.  DOFUR: DDoS Forensics Using MapReduce , 2011, 2011 Frontiers of Information Technology.

[4]  Youngseok Lee,et al.  Detecting DDoS attacks with Hadoop , 2011, CoNEXT '11 Student.

[5]  R.C. Joshi,et al.  A Distributed Approach using Entropy to Detect DDoS Attacks in ISP Domain , 2007, 2007 International Conference on Signal Processing, Communications and Networking.

[6]  C. E. SHANNON,et al.  A mathematical theory of communication , 1948, MOCO.

[7]  Paul J Criscuolo,et al.  Distributed Denial of Service: Trin00, Tribe Flood Network, Tribe Flood Network 2000, and Stacheldraht CIAC-2319 , 2000 .

[8]  P. J. Criscuolo.  Distributed Denial of Service Tools, Trin00, Tribe Flood Network, Tribe Flood Network 2000 and Stacheldraht. , 2000 .

[9]  Song Guo,et al.  Discriminating DDoS Attacks from Flash Crowds Using Flow Correlation Coefficient , 2012, IEEE Transactions on Parallel and Distributed Systems.

[10]  R. S. Dayama,et al.  Secured Network from Distributed DOS through HADOOP , 2015 .

[11]  Maninder Singh,et al.  Hadoop‐based analytic framework for cyber forensics , 2018, Int. J. Commun. Syst..

[12]  Kai Qian,et al.  A Neural-Network Based DDoS Detection System Using Hadoop and HBase , 2015, 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems.

[13]  Sunny Behal,et al.  Detection of DDoS attacks and flash events using novel information theory metrics , 2017, Comput. Networks.

[14]  Sufian Hameed,et al.  HADEC: Hadoop-based live DDoS detection framework , 2018, EURASIP J. Inf. Secur..

[15]  Kotagiri Ramamohanarao,et al.  Survey of network-based defense mechanisms countering the DoS and DDoS problems , 2007, CSUR.

[16]  Krishan Kumar,et al.  A traffic cluster entropy based approach to distinguish DDoS attacks from flash event using DETER testbed , 2014 .

[17]  Sufian Hameed,et al.  Efficacy of Live DDoS Detection with Hadoop , 2015, NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium.