Degree-based Outlier Detection within IP Traffic Modelled as a Link Stream

Precise detection and identification of anomalous events in IP traffic are crucial in many applications. This paper intends to address this task by adopting the link stream formalism which properly captures temporal and structural features of the data. Within this framework we focus on finding anomalous behaviours with the degree of IP addresses over time. Due to diversity in IP profiles, this feature is typically distributed heterogeneously, preventing us to find anomalies. To deal with this challenge, we design a method to detect outliers as well as precisely identify their cause in a sequence of similar heterogeneous distributions. We apply it to a MAWI capture of IP traffic and we show that it succeeds at detecting relevant patterns in terms of anomalous network activity.

[1]  Christos Faloutsos,et al.  SedanSpot: Detecting Anomalies in Edge Streams , 2018, 2018 IEEE International Conference on Data Mining (ICDM).

[2]  Sebastian Zander,et al.  A preliminary performance comparison of five machine learning algorithms for practical IP traffic flow classification , 2006, CCRV.

[3]  Scott R. Eliason.  Maximum likelihood estimation: Logic and practice. , 1994 .

[4]  R. Suganya,et al.  Data Mining Concepts and Techniques , 2010 .

[5]  Benjamin Aziz,et al.  Comparison between divergence measures for anomaly detection of mobile agents in IP networks , 2017 .

[6]  Danai Koutra,et al.  Graph based anomaly detection and description: a survey , 2014, Data Mining and Knowledge Discovery.

[7]  Shoichiro Asano,et al.  Detecting Anomalous Traffic using Communication Graphs , 2011 .

[8]  Michel L. Goldstein,et al.  Problems with fitting to the power-law distribution , 2004, cond-mat/0402322.

[9]  Deepayan Chakrabarti,et al.  AutoPart: Parameter-Free Graph Partitioning and Outlier Detection , 2004, PKDD.

[10]  Hisashi Kashima,et al.  Eigenspace-based anomaly detection in computer systems , 2004, KDD.

[11]  Emmanuelle Anceaume,et al.  Sketch *-Metric: Comparing Data Streams via Sketching , 2012, 2013 IEEE 12th International Symposium on Network Computing and Applications.

[12]  Hector Garcia-Molina,et al.  Web graph similarity for anomaly detection , 2010, Journal of Internet Services and Applications.

[13]  Philip S. Yu,et al.  GraphScope: parameter-free mining of large time-evolving graphs , 2007, KDD '07.

[14]  Steve Harenberg,et al.  Anomaly detection in dynamic networks: a survey , 2015 .

[15]  Hong Huang,et al.  Network Traffic Anomaly Detection , 2014, ArXiv.

[16]  Leman Akoglu,et al.  Fast Memory-efficient Anomaly Detection in Streaming Heterogeneous Graphs , 2016, KDD.

[17]  Debasis Dash,et al.  Leveraging Structural Hierarchy for Scalable Network Comparison , 2016, DEXA.

[18]  Nagiza F. Samatova,et al.  Community-based anomaly detection in evolutionary networks , 2012, Journal of Intelligent Information Systems.

[19]  Ananthram Swami,et al.  Com2: Fast Automatic Discovery of Temporal ('Comet') Communities , 2014, PAKDD.

[20]  Brandon Pincombea,et al.  Anomaly Detection in Time Series of Graphs using ARMA Processes , 2007 .

[21]  Clémence Magnien,et al.  Detecting events in the dynamics of ego-centered measurements of the internet topology , 2010, 8th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks.

[22]  Zhengding Lu,et al.  Community mining on dynamic weighted directed graphs , 2009, CIKM-CNIKM.

[23]  Jennifer Rexford,et al.  Sensitivity of PCA for traffic anomaly detection , 2007, SIGMETRICS '07.

[24]  Jennifer Neville,et al.  Anomaly Detection in Dynamic Networks of Varying Size , 2014, ArXiv.

[25]  Yizhou Sun,et al.  Integrating community matching and outlier detection for mining evolutionary community outliers , 2012, KDD.

[26]  Hiroshi Esaki,et al.  Network application profiling with traffic causality graphs , 2014, Int. J. Netw. Manag..

[27]  Charu C. Aggarwal,et al.  Outlier Analysis , 2013, Springer New York.

[28]  Salvatore J. Stolfo,et al.  Data Mining Approaches for Intrusion Detection , 1998, USENIX Security Symposium.

[29]  C. Faloutsos,et al.  EVENT DETECTION IN TIME SERIES OF MOBILE COMMUNICATION GRAPHS , 2010 .

[30]  Harvey J. Motulsky,et al.  Detecting outliers when fitting data with nonlinear regression – a new method based on robust nonlinear regression and the false discovery rate , 2006, BMC Bioinformatics.

[31]  Jean-Loup Guillaume,et al.  Temporal reachability graphs , 2012, Mobicom '12.

[32]  Steve Harenberg,et al.  A Scalable Approach for Outlier Detection in Edge Streams Using Sketch-based Approximations , 2016, SDM.

[33]  Charu C. Aggarwal,et al.  On Anomalous Hotspot Discovery in Graph Streams , 2013, 2013 IEEE 13th International Conference on Data Mining.

[34]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[35]  Danai Koutra,et al.  NetSimile: A Scalable Approach to Size-Independent Network Similarity , 2012, ArXiv.

[36]  Kensuke Fukuda,et al.  A taxonomy of anomalies in backbone network traffic , 2014, 2014 International Wireless Communications and Mobile Computing Conference (IWCMC).

[37]  Eric Fleury,et al.  A unifying model for representing time-varying graphs , 2014, 2015 IEEE International Conference on Data Science and Advanced Analytics (DSAA).

[38]  Michael D. Iannacone,et al.  GraphPrints: Towards a Graph Analytic Method for Network Anomaly Detection , 2016, CISRC.

[39]  Michael Frankfurter,et al.  Numerical Recipes In C The Art Of Scientific Computing , 2016 .

[40]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[41]  Yizhou Sun,et al.  Community Trend Outlier Detection Using Soft Temporal Pattern Mining , 2012, ECML/PKDD.

[42]  Vladimir Batagelj,et al.  An algebraic approach to temporal network analysis based on temporal quantities , 2015, Social Network Analysis and Mining.

[43]  F. E. Grubbs.  Procedures for Detecting Outlying Observations in Samples , 1969 .

[44]  Kuai Xu,et al.  Behavior Analysis of Internet Traffic via Bipartite Graphs and One-Mode Projections , 2014, IEEE/ACM Trans. Netw..

[45]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[46]  Mark Crovella,et al.  Characterization of network-wide anomalies in traffic flows , 2004, IMC '04.

[47]  Yogesh Virkar,et al.  Power-law distributions in binned empirical data , 2012, 1208.3524.

[48]  Kensuke Fukuda,et al.  Seven Years and One Day: Sketching the Evolution of Internet Traffic , 2009, IEEE INFOCOM 2009.

[49]  Michalis Faloutsos,et al.  Exploiting dynamicity in graph-based traffic analysis: techniques and applications , 2009, CoNEXT '09.

[50]  Clémence Magnien,et al.  Discovering Patterns of Interest in IP Traffic Using Cliques in Bipartite Link Streams , 2017, ArXiv.

[51]  Panos M. Pardalos,et al.  Quantification of network structural dissimilarities , 2017, Nature Communications.

[52]  Matthieu Latapy,et al.  Stream graphs and link streams for the modeling of interactions over time , 2017, Social Network Analysis and Mining.

[53]  Ambuj K. Singh,et al.  NetSpot: Spotting Significant Anomalous Regions on Dynamic Networks , 2013, SDM.

[54]  Marylyn D Ritchie,et al.  Basic Statistics , 2003, Current protocols in human genetics.

[55]  Paul Barford,et al.  A signal analysis of network traffic anomalies , 2002, IMW '02.

[56]  Kensuke Fukuda,et al.  MAWILab: combining diverse anomaly detectors for automated anomaly labeling and performance benchmarking , 2010, CoNEXT.