A Super Point Detection Algorithm Under Sliding Time Windows Based on Rough and Linear Estimators

Detecting super points from high-speed networks can effectively help to monitor networks, which is a hot topic in network fields. Most existing algorithms are carried out under discrete time windows and the results are in a certain percentage of omission. In this paper, the phenomenon of missed super points detection in discrete time windows is analyzed based on real-world traffic. Then a new algorithm, which detects the super points under sliding time windows, is proposed. Our algorithm uses a lightweight estimator to identify candidate super points and a linear estimator to filter super points. The lightweight estimator is fast, and the linear estimator has high accuracy. Both the lightweight estimator and the linear estimator adopt a data structure, called distance recorder, to support sliding time windows. Moreover, our algorithm is also a parallel algorithm. On the basis of thoroughly discussing the mathematic principles and operation steps of our algorithm, two groups of real-world traffic from a 40-Gb/s high-speed network are applied in the experiments which running on a graphic processing unit (GPU). The experiments are conducted under discrete time windows and sliding time windows separately. The former results show that our algorithm is superior to other existing algorithms in the comprehensive performance, and the latter results indicate that our algorithm can run steadily under sliding time windows.

[1]  K. K. Ramakrishnan,et al.  Understanding the super-sized traffic of the super bowl , 2013, Internet Measurement Conference.

[2]  Piotr Indyk,et al.  Maintaining Stream Statistics over Sliding Windows , 2002, SIAM J. Comput..

[3]  Yuan He,et al.  Towards Constant-Time Cardinality Estimation for Large-Scale RFID Systems , 2015, 2015 44th International Conference on Parallel Processing.

[4]  Yuan Cao,et al.  Understanding Internet DDoS Mitigation from Academic and Industrial Perspectives , 2018, IEEE Access.

[5]  Kun Yang,et al.  A DDoS Attack Detection and Mitigation With Software-Defined Internet of Things Framework , 2018, IEEE Access.

[6]  Shiping Chen,et al.  Estimating the Cardinality of a Mobile Peer-to-Peer Network , 2013, IEEE Journal on Selected Areas in Communications.

[7]  Michal Mrozowski,et al.  Communication and Load Balancing Optimization for Finite Element Electromagnetic Simulations Using Multi-GPU Workstation , 2017, IEEE Transactions on Microwave Theory and Techniques.

[8]  Yao Hu,et al.  A Box-Covering-Based Routing Algorithm for Large-Scale SDNs , 2017, IEEE Access.

[9]  Wenyu Qu,et al.  A novel data streaming method detecting superpoints , 2011, 2011 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[10]  Tao Qin,et al.  A Data Streaming Method for Monitoring Host Connection Degrees of High-Speed Links , 2011, IEEE Transactions on Information Forensics and Security.

[11]  Gustavo Fraidenraich,et al.  Performance Analysis of the Classic and Robust Chinese Remainder Theorems in Pulsed Doppler Radars , 2018, IEEE Transactions on Signal Processing.

[12]  Xing Xie,et al.  Network Motif Discovery: A GPU Approach , 2015, IEEE Transactions on Knowledge and Data Engineering.

[13]  Kyu-Young Whang,et al.  A linear-time probabilistic counting algorithm for database applications , 1990, TODS.

[14]  Guiqiang Ni,et al.  Fast counting the cardinality of flows for big traffic over sliding windows , 2017, Frontiers of Computer Science.

[15]  MyungKeun Yoon,et al.  A grand spread estimator using a graphics processing unit , 2014, J. Parallel Distributed Comput..

[16]  Akira Sato,et al.  Enhancing performance of cardinality analysis by packet filtering , 2016, 2016 International Conference on Information Networking (ICOIN).

[17]  Chuyen T. Nguyen,et al.  A simple method for anonymous tag cardinality estimation in RFID systems with false detection , 2017, 2017 4th NAFOSTED Conference on Information and Computer Science.

[18]  Denis Foley,et al.  Ultra-Performance Pascal GPU and NVLink Interconnect , 2017, IEEE Micro.

[19]  Jie Xu,et al.  SRLA: A Real Time Sliding Time Window Super Point Cardinality Estimation Algorithm for High Speed Network Based on GPU , 2018, 2018 IEEE 20th International Conference on High Performance Computing and Communications; IEEE 16th International Conference on Smart City; IEEE 4th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[20]  Zhiyang Li,et al.  A Continuous Virtual Vector-Based Algorithm for Measuring Cardinality Distribution , 2014, ICA3PP.

[21]  Yong Xiang,et al.  A General QoS Aware Flow-Balancing and Resource Management Scheme in Distributed Software-Defined Networks , 2016, IEEE Access.

[22]  Kenli Li,et al.  BAG: Managing GPU as Buffer Cache in Operating Systems , 2014, IEEE Transactions on Parallel and Distributed Systems.

[23]  Xiaomei Wang,et al.  An efficient RFID tag cardinality estimation protocol based on bit detection , 2017, 2017 IEEE 17th International Conference on Communication Technology (ICCT).

[24]  Jia Zhu,et al.  Joint Cooperative Beamforming and Jamming for Physical-Layer Security of Decode-and-Forward Relay Networks , 2017, IEEE Access.

[25]  Hamed Al-Raweshidy,et al.  OLC: Open-Level Control Plane Architecture for Providing Better Scalability in an SDN Network , 2018, IEEE Access.

[26]  Shinpei Kato,et al.  Real-Time GPU Resource Management with Loadable Kernel Modules , 2017, IEEE Transactions on Parallel and Distributed Systems.

[27]  Yong Guan,et al.  Identifying High-Cardinality Hosts from Network-Wide Traffic Measurements , 2016, IEEE Trans. Dependable Secur. Comput..

[28]  Dong Li,et al.  Optimizing Data Placement on GPU Memory: A Portable Approach , 2017, IEEE Transactions on Computers.

[29]  Jie Liu,et al.  High Speed Network Super Points Detection Based on Sliding Time Window by GPU , 2017, 2017 IEEE International Symposium on Parallel and Distributed Processing with Applications and 2017 IEEE International Conference on Ubiquitous Computing and Communications (ISPA/IUCC).

[30]  Victor C. M. Leung,et al.  Fast and accurate cardinality estimation in cellular-based wireless communications , 2015, 2015 IEEE Wireless Communications and Networking Conference (WCNC).

[31]  Weijiang Liu,et al.  A hash-based algorithm for measuring cardinality distribution in network traffic , 2016, Int. J. Auton. Adapt. Commun. Syst..

[32]  Keqiu Li,et al.  Detection of Superpoints Using a Vector Bloom Filter , 2016, IEEE Transactions on Information Forensics and Security.

[33]  Min Chen,et al.  Cardinality Estimation for Elephant Flows: A Compact Solution Based on Virtual Register Sharing , 2017, IEEE/ACM Transactions on Networking.

[34]  Bormin Huang,et al.  GPU Compute Unified Device Architecture (CUDA)-based Parallelization of the RRTMG Shortwave Rapid Radiative Transfer Model , 2016, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing.

[35]  Rafael Asenjo,et al.  Mapping Streaming Applications on Commodity Multi-CPU and GPU On-Chip Processors , 2016, IEEE Transactions on Parallel and Distributed Systems.

[36]  Guiqiang Ni,et al.  CVS: Fast cardinality estimation for large-scale data streams over sliding windows , 2016, Neurocomputing.

[37]  Andrew Tanny Liem,et al.  P2P Live-Streaming Application-Aware Architecture for QoS Enhancement in the EPON , 2018, IEEE Systems Journal.

[38]  Shigang Chen,et al.  Per-flow counting for big network data stream over sliding windows , 2017, 2017 IEEE/ACM 25th International Symposium on Quality of Service (IWQoS).

[39]  Wei-Shi Zheng,et al.  Online Hashing , 2013, IEEE Transactions on Neural Networks and Learning Systems.

[40]  Sparsh Mittal,et al.  A Survey of Techniques for Architecting and Managing GPU Register File , 2017, IEEE Transactions on Parallel and Distributed Systems.

[41]  Zhixin Sun,et al.  A Detection Method for Anomaly Flow in Software Defined Network , 2018, IEEE Access.

[42]  David P. Woodruff,et al.  An optimal algorithm for the distinct elements problem , 2010, PODS '10.