Network traffic fusion and analysis against DDoS flooding attacks with a novel reversible sketch

Abstract Distributed Denial of Service (DDoS) flooding attacks are one of the typical attacks over the Internet. They aim to prevent normal users from accessing specific network resources. How to detect DDoS flooding attacks arises a significant and timely research topic. However, with the continuous increase of network scale, the continuous growth of network traffic brings great challenges to the detection of DDoS flooding attacks. Incomplete network traffic collection or non-real-time processing of big-volume network traffic will seriously affect the accuracy and efficiency of attack detection. Recently, sketch data structures are widely applied in high-speed networks to compress and fuse network traffic. But sketches suffer from a reversibility problem that it is difficult to reconstruct a set of keys that exhibit abnormal behavior due to the irreversibility of hash functions. In order to address the above challenges, in this paper, we first design a novel Chinese Remainder Theorem based Reversible Sketch (CRT-RS). CRT-RS is not only capable of compressing and fusing big-volume network traffic but also has the ability of reversely discovering the anomalous keys (e.g., the sources of malicious or unwanted traffic). Then, based on traffic records generated by CRT-RS, we propose a Modified Multi-chart Cumulative Sum (MM-CUSUM) algorithm that supports self-adaptive and protocol independent detection to detect DDoS flooding attacks. The performance of the proposed detection method is experimentally examined by two open source datasets. The experimental results show that the method can detect DDoS flooding attacks with efficiency, accuracy, adaptability, and protocol independability. Moreover, by comparing with other attack detection methods using sketch techniques, our method has quantifiable lower computation complexity when recovering the anomalous source addresses, which is the most important merit of the developed method.

[1]  Witold Pedrycz,et al.  Security Data Collection and Data Analytics in the Internet: A Survey , 2019, IEEE Communications Surveys & Tutorials.

[2]  Graham Cormode,et al.  An improved data stream summary: the count-min sketch and its applications , 2004, J. Algorithms.

[3]  Xiapu Luo,et al.  SkyShield: A Sketch-Based Defense System Against Application Layer DDoS Attacks , 2018, IEEE Transactions on Information Forensics and Security.

[4]  Zheng Yan,et al.  A survey on network data collection , 2018, J. Netw. Comput. Appl..

[5]  Douglas C. Montgomery,et al.  Introduction to Statistical Quality Control , 1986 .

[6]  Fei Wang,et al.  Bitwise sketch for lightweight reverse IP reconstruction in network anomaly detection , 2012, 2012 IEEE 9th International Conference on Mobile Ad-Hoc and Sensor Systems (MASS 2012).

[7]  Tao Qin,et al.  A Data Streaming Method for Monitoring Host Connection Degrees of High-Speed Links , 2011, IEEE Transactions on Information Forensics and Security.

[8]  Yan Gao,et al.  HiFIND: A high-speed flow-level intrusion detection approach with DoS resiliency , 2010, Comput. Networks.

[9]  Gustavo Alonso,et al.  Augmented Sketch: Faster and More Accurate Stream Processing , 2016, SIGMOD Conference.

[10]  Erhan Guven,et al.  A Survey of Data Mining and Machine Learning Methods for Cyber Security Intrusion Detection , 2016, IEEE Communications Surveys & Tutorials.

[11]  Ming-Yang Kao,et al.  Reversible sketches: enabling monitoring and analysis over high-speed data streams , 2007, TNET.

[12]  Song Guo,et al.  Discriminating DDoS Attacks from Flash Crowds Using Flow Correlation Coefficient , 2012, IEEE Transactions on Parallel and Distributed Systems.

[13]  Zheng Yan,et al.  A Survey on Network Security-Related Data Collection Technologies , 2018, IEEE Access.

[14]  Christian Callegari,et al.  An information-theoretic method for the detection of anomalies in network traffic , 2017, Comput. Secur..

[15]  H. Jonathan Chao,et al.  PacketScore: a statistics-based packet filtering scheme against distributed denial-of-service attacks , 2006, IEEE Transactions on Dependable and Secure Computing.

[16]  Bu-Sung Lee,et al.  Detection of network anomalies using Improved-MSPCA with sketches , 2017, Comput. Secur..

[17]  Moses Charikar,et al.  Finding frequent items in data streams , 2004, Theor. Comput. Sci..

[18]  Kensuke Fukuda,et al.  MAWILab: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking , 2010, CoNEXT.

[19]  Osman Salem,et al.  A scalable, efficient and informative approach for anomaly‐based intrusion detection systems: theory and practice , 2010, Int. J. Netw. Manag..

[20]  Tong Yang,et al.  Pyramid Sketch: a Sketch Framework for Frequency Estimation of Data Streams , 2017, Proc. VLDB Endow..

[21]  Sotiris Ioannidis,et al.  Network Topology Effects on the Detectability of Crossfire Attacks , 2018, IEEE Transactions on Information Forensics and Security.

[22]  Zheng Yan,et al.  Data Fusion for Network Intrusion Detection: A Review , 2018, Secur. Commun. Networks.

[23]  Ahmed Mehaoua,et al.  Flooding attacks detection in traffic of backbone networks , 2011, 2011 IEEE 36th Conference on Local Computer Networks.

[24]  Xiapu Luo,et al.  Characterizing the Impacts of Application Layer DDoS Attacks , 2017, 2017 IEEE International Conference on Web Services (ICWS).

[25]  Xenofontas A. Dimitropoulos,et al.  A novel framework for modeling and mitigating distributed link flooding attacks , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[26]  Martine Bellaïche,et al.  SYN Flooding Attack Detection Based on Entropy Computing , 2009, GLOBECOM 2009 - 2009 IEEE Global Telecommunications Conference.

[27]  Li Aiping,et al.  Detecting Hidden Anomalies Using Sketch for High-speed Network Data Stream Monitoring , 2012 .

[28]  Ali A. Ghorbani,et al.  Toward Generating a New Intrusion Detection Dataset and Intrusion Traffic Characterization , 2018, ICISSP.

[29]  Jun Zhang,et al.  Internet Traffic Classification Using Constrained Clustering , 2014, IEEE Transactions on Parallel and Distributed Systems.

[30]  Yu Cheng,et al.  SIP Flooding Attack Detection with a Multi-Dimensional Sketch Design , 2014, IEEE Transactions on Dependable and Secure Computing.

[31]  Naixue Xiong,et al.  Anomaly secure detection methods by analyzing dynamic characteristics of the network traffic in cloud communications , 2014, Inf. Sci..

[32]  Witold Pedrycz,et al.  Data collection for attack detection and security measurement in Mobile Ad Hoc Networks: A survey , 2018, J. Netw. Comput. Appl..

[33]  Martin J. Reed,et al.  Denial of service detection through TCP congestion window analysis , 2013, World Congress on Internet Security (WorldCIS-2013).

[34]  Mohammed Atiquzzaman,et al.  LTE/LTE-A Network Security Data Collection and Analysis for Security Measurement: A Survey , 2018, IEEE Access.

[35]  Saman Taghavi Zargar,et al.  A Survey of Defense Mechanisms Against Distributed Denial of Service (DDoS) Flooding Attacks , 2013, IEEE Communications Surveys & Tutorials.

[36]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[37]  Christian Rossow,et al.  Amplification Hell: Revisiting Network Protocols for DDoS Abuse , 2014, NDSS.

[38]  Christian Callegari,et al.  Combining sketches and wavelet analysis for multi time-scale network anomaly detection , 2011, Comput. Secur..