Toward Stream-Based IP Flow Analysis

Analyzing IP flows is an essential part of traffic measurement for cyber security. Based on information from IP flows, it is possible to discover the majority of concurrent cyber threats in highspeed, large-scale networks. Some major prevailing challenges for IP flow analysis include, but are not limited to, analysis over a large volume of IP flows, scalability issues, and detecting cyber threats in real time. In this article, we discuss the transformation of present IP flow analysis into a stream-based approach to face current challenges in IP flow analysis. We examine the possible positive and negative impacts of the transformation and present examples of real-world applications, along with our recommendations. Our ongoing results show that stream-based IP flow analysis successfully meets the above-mentioned challenges and is suitable for achieving real-time network security analysis and situational awareness.

[1]  Ulrik Franke,et al.  Availability of enterprise IT systems: an expert-based Bayesian framework , 2011, Software Quality Journal.

[2]  Pavel Celeda,et al.  A performance benchmark for NetFlow data analysis on distributed stream processing systems , 2016, NOMS 2016 - 2016 IEEE/IFIP Network Operations and Management Symposium.

[3]  Jennifer Widom,et al.  Models and issues in data stream systems , 2002, PODS.

[4]  Anna Sperotto,et al.  Flow-based intrusion detection , 2011, 12th IFIP/IEEE International Symposium on Integrated Network Management (IM 2011) and Workshops.

[5]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[6]  Aiko Pras,et al.  Flow Monitoring Explained: From Packet Capture to Data Analysis With NetFlow and IPFIX , 2014, IEEE Communications Surveys & Tutorials.

[7]  Alessandro Margara,et al.  Processing flows of information: From data stream to complex event processing , 2012, CSUR.