Detecting events in the dynamics of ego-centered measurements of the internet topology

Detecting events such as major routing changes or congestions in the dynamics of the internet topology is an important but challenging task. We explore here a top-down approach based on a notion of statistically significant events. It consists in identifying statistics which exhibit a homogeneous distribution with outliers, which correspond to events. We apply this approach to ego-centerd measurements of the internet topology (views obtained from a single monitor) and show that it succeeds in detecting meaningful events. Finally, we give some hints for the interpretation of such events in terms of network events.

[1]  A. Madansky.  Identification of Outliers , 1988 .

[2]  Albert,et al.  Topology of evolving networks: local events and universality , 2000, Physical review letters.

[3]  Nick Feamster,et al.  Practical issues with using network tomography for fault diagnosis , 2008, CCRV.

[4]  Matthieu Latapy,et al.  Rigorous Measurement of IP-Level Neighborhood of Internet Core Routers , 2010, 2010 INFOCOM IEEE Conference on Computer Communications Workshops.

[5]  Dorothy E. Denning,et al.  A Multilevel Relational Data Model , 1987, 1987 IEEE Symposium on Security and Privacy.

[6]  Christophe Diot,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM.

[7]  Kensuke Fukuda,et al.  Seven Years and One Day: Sketching the Evolution of Internet Traffic , 2009, IEEE INFOCOM 2009.

[8]  R. Geary,et al.  Testing for Normality , 2003 .

[9]  Matthieu Latapy,et al.  Link prediction in bipartite graphs using internal links and weighted projection , 2011, 2011 IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[10]  Clémence Magnien,et al.  Impact of power-law topology on IP-level routing dynamics: Simulation results , 2012, 2012 Proceedings IEEE INFOCOM Workshops.

[11]  Jean-Loup Guillaume,et al.  Statistical Analysis of a P2P Query Graph Based on Degrees and Their Time-Evolution , 2004, IWDC.

[12]  Vern Paxson.  End-to-end internet packet dynamics , 1999, TNET.

[13]  Larry Wasserman,et al.  All of Statistics: A Concise Course in Statistical Inference , 2004 .

[14]  Mark Crovella,et al.  Mining anomalies using traffic feature distributions , 2005, SIGCOMM '05.

[15]  Clémence Magnien,et al.  Quantifying paedophile queries in a large P2P system , 2011, 2011 Proceedings IEEE INFOCOM.

[16]  Philippe Owezarski,et al.  Non-Gaussian and Long Memory Statistical Characterizations for Internet Traffic with Anomalies , 2007, IEEE Transactions on Dependable and Secure Computing.

[17]  Matthieu Latapy,et al.  Measurement of eDonkey activity with distributed honeypots , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[18]  Douglas M. Hawkins.  Identification of Outliers , 1980, Monographs on Applied Probability and Statistics.

[19]  Johan Karlsson,et al.  Metrics for Power Spectra: An Axiomatic Approach , 2009, IEEE Transactions on Signal Processing.

[20]  David M. Pennock,et al.  Comparing static and dynamic measurements and models of the Internet's AS topology , 2004, IEEE INFOCOM 2004.

[21]  H. Thode.  Testing For Normality , 2002 .

[22]  Matthieu Latapy,et al.  Combining the Use of Clustering and Scale-Free Nature of User Exchanges into a Simple and Efficient P2P System , 2005, Euro-Par.

[23]  Brice Augustin,et al.  Avoiding traceroute anomalies with Paris traceroute , 2006, IMC '06.

[24]  Jake D. Brutlag,et al.  Aberrant Behavior Detection in Time Series for Network Monitoring , 2000, LISA.

[25]  Matthieu Latapy,et al.  Efficient Measurement of Complex Networks Using Link Queries , 2009, IEEE INFOCOM Workshops 2009.

[26]  Daniel Massey,et al.  Inferring the Origin of Routing Changes using Link Weights , 2007, 2007 IEEE International Conference on Network Protocols.

[27]  Salvatore J. Stolfo,et al.  Privacy-preserving payload-based correlation for accurate malicious traffic detection , 2006, LSAD '06.

[28]  Matthieu Latapy,et al.  Ten weeks in the life of an eDonkey server , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[29]  Mark E. J. Newman,et al.  Power-Law Distributions in Empirical Data , 2007, SIAM Rev..

[30]  Renata Teixeira,et al.  Characterizing network events and their impact on routing , 2007, CoNEXT '07.

[31]  J. E. Glynn,et al.  Numerical Recipes: The Art of Scientific Computing , 1989 .

[32]  Pieter H. Hartel,et al.  POSEIDON: a 2-tier anomaly-based network intrusion detection system , 2006, Fourth IEEE International Workshop on Information Assurance (IWIA'06).

[33]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[34]  VARUN CHANDOLA,et al.  Anomaly detection: A survey , 2009, CSUR.

[35]  Lixia Zhang,et al.  Observing the evolution of internet as topology , 2007, SIGCOMM.

[36]  Matthieu Latapy,et al.  Fast dynamics in Internet topology: preliminary observations and explanations , 2009, ArXiv.

[37]  Clémence Magnien,et al.  Towards realistic modeling of IP-level routing topology dynamics , 2011, ArXiv.

[38]  Jean-Jacques Pansiot,et al.  Local and dynamic analysis of Internet multicast router topology , 2007, Ann. des Télécommunications.

[39]  Matthieu Latapy,et al.  A Radar for the Internet , 2008, 2008 IEEE International Conference on Data Mining Workshops.

[40]  Matthieu Latapy,et al.  Complex Network Measurements: Estimating the Relevance of Observed Properties , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[41]  Matthieu Latapy,et al.  Efficient and simple generation of random simple connected graphs with prescribed degree sequence , 2005, J. Complex Networks.

[42]  Matthieu Latapy,et al.  Fast Dynamics in Internet Topology: Observations and First Explanations , 2009, 2009 Fourth International Conference on Internet Monitoring and Protection.

[43]  Brice Augustin,et al.  Detection, understanding, and prevention of traceroute measurement artifacts , 2008, Comput. Networks.

[44]  Balachander Krishnamurthy,et al.  Sketch-based change detection: methods, evaluation, and applications , 2003, IMC '03.

[45]  D. Bolzoni,et al.  Approaches in anomaly-based intrusion detection systems , 2006 .

[46]  Scott R. Eliason.  Maximum likelihood estimation: Logic and practice. , 1994 .

[47]  L. D. Costa,et al.  What are the best concentric descriptors for complex networks? , 2007, 0705.4251.

[48]  T. Moors,et al.  Streamlining traceroute by estimating path lengths , 2004, 2004 IEEE International Workshop on IP Operations and Management.

[49]  Damien Magoni,et al.  Analysis of the autonomous system network topology , 2001, CCRV.

[50]  Ramesh Govindan,et al.  An analysis of Internet inter-domain topology and route stability , 1997, Proceedings of INFOCOM '97.

[51]  Walter Willinger,et al.  A first-principles approach to understanding the internet's router-level topology , 2004, SIGCOMM '04.

[52]  Walter Willinger,et al.  Towards a Theory of Scale-Free Graphs: Definition, Properties, and Implications , 2005, Internet Math..

[53]  Kensuke Fukuda,et al.  Extracting hidden anomalies using sketch and non Gaussian multiresolution statistical detection procedures , 2007, LSAD '07.

[54]  Jean-Loup Guillaume,et al.  Relevance of massively distributed explorations of the Internet topology: Qualitative results , 2006, Comput. Networks.

[55]  Ramesh Govindan,et al.  Detection and identification of network anomalies using sketch subspaces , 2006, IMC '06.

[56]  Matthieu Latapy,et al.  Termination of Multipartite Graph Series Arising from Complex Network Modelling , 2010, COCOA.

[57]  Martin Crowder.  Parameter Estimation for Scientists and Engineers by Adriaan van den Bos , 2007 .

[58]  Felix Naumann,et al.  Data fusion , 2009, CSUR.

[59]  Daniel Massey,et al.  Visualizing Internet Routing Changes , 2006, IEEE Transactions on Visualization and Computer Graphics.

[60]  Walter Willinger,et al.  The origin of power laws in Internet topologies revisited , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[61]  Walter Willinger,et al.  A first-principles approach to understanding the internet's router-level topology , 2004, SIGCOMM 2004.

[62]  Paul Barford,et al.  A signal analysis of network traffic anomalies , 2002, IMW '02.

[63]  Farnam Jahanian,et al.  Origins of Internet routing instability , 1999, IEEE INFOCOM '99. Conference on Computer Communications. Proceedings. Eighteenth Annual Joint Conference of the IEEE Computer and Communications Societies. The Future is Now (Cat. No.99CH36320).