The many facets of internet topology and traffic

The Internet's layered architecture and organizational structure give rise to a number of different topologies, with the lower layers defining more physical and the higher layers more virtual/logical types of connectivity structures. These structures are very different, and successful Internet topology modeling requires annotating the nodes and edges of the corresponding graphs with information that reflects their network-intrinsic meaning. These structures also give rise to different representations of the traffic that traverses the heterogeneous Internet, and a traffic matrix is a compact and succinct description of the traffic exchanges between the nodes in a given connectivity structure. In this paper, we summarize recent advances in Internet research related to (i) inferring and modeling the router-level topologies of individual service providers (i.e., the physical connectivity structure of an ISP, where nodes are routers/switches and links represent physical connections), (ii) estimating the intra-AS traffic matrix when the AS's router-level topology and routing configuration are known, (iii) inferring and modeling the Internet's AS-level topology, and (iv) estimating the inter-AS traffic matrix. We will also discuss recent work on Internet connectivity structures that arise at the higher layers in the TCP/IP protocol stack and are more virtual and dynamic; e.g., overlay networks like the WWW graph, where nodes are web pages and edges represent existing hyperlinks, or P2P networks like Gnutella, where nodes represent peers and two peers are connected if they have an active network connection.

[1]  Erica Jen,et al.  Robust design : a repertoire of biological, ecological, and engineering case studies , 2005 .

[2]  F. G. Kilgour The evolution of the book , 1998 .

[3]  Edward R. Scheinerman,et al.  Dot product representations of graphs , 1998, Discret. Math..

[4]  Donald F. Towsley,et al.  On distinguishing between Internet power law topology generators , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[5]  David Bawden,et al.  Book Review: Evolution and Structure of the Internet: A Statistical Physics Approach. , 2006 .

[6]  Peng Xie,et al.  Sampling biases in IP topology measurements , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[7]  Cristopher Moore,et al.  On the bias of traceroute sampling: or, power-law degree distributions in regular graphs , 2005, STOC '05.

[8]  Leonard Kleinrock,et al.  On measured behavior of the ARPA network , 1899, AFIPS '74.

[9]  Marc Najork,et al.  A large‐scale study of the evolution of Web pages , 2004, Softw. Pract. Exp..

[10]  Walter Willinger,et al.  An empirical approach to modeling inter-AS traffic matrices , 2005, IMC '05.

[11]  Béla Bollobás,et al.  The degree sequence of a scale‐free random graph process , 2001, Random Struct. Algorithms.

[12]  Walter Willinger,et al.  More "normal" than normal: scaling distributions and complex systems , 2004, Proceedings of the 2004 Winter Simulation Conference, 2004..

[13]  Daniel Stutzbach,et al.  Capturing Accurate Snapshots of the Gnutella Network , 2005, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[14]  Bing Yu,et al.  Time-Varying Network Tomography: Router Link Data , 2000 .

[15]  J M Carlson,et al.  Highly optimized tolerance: a mechanism for power laws in designed systems. , 1999, Physical review. E, Statistical physics, plasmas, fluids, and related interdisciplinary topics.

[16]  Walter Willinger,et al.  Robustness and the Internet: Design and evolution , 2002 .

[17]  Ramesh Govindan,et al.  Heuristics for Internet map discovery , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[18]  Alan M. Frieze,et al.  Random Deletion in a Scale-Free Random Graph Process , 2004, Internet Math..

[19]  Walter Willinger,et al.  Towards capturing representative AS-level Internet topologies , 2004, Comput. Networks.

[20]  Albert G. Greenberg,et al.  Fast accurate computation of large-scale IP traffic matrices from link loads , 2003, SIGMETRICS '03.

[21]  Rami Cohen,et al.  The Internet Dark Matter - on the Missing Links in the AS Connectivity Map , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[22]  Eli Upfal,et al.  The Web as a graph , 2000, PODS.

[23]  Konstantina Papagiannaki,et al.  Structural analysis of network traffic flows , 2004, SIGMETRICS '04/Performance '04.

[24]  Walter Willinger,et al.  Towards a Theory of Scale-Free Graphs: Definition, Properties, and Implications , 2005, Internet Math..

[25]  Jon M. Kleinberg,et al.  The Web as a Graph: Measurements, Models, and Methods , 1999, COCOON.

[26]  Sandrine Vaton,et al.  Iterative Bayesian estimation of network traffic matrices in the case of bursty flows , 2002, IMW '02.

[27]  John W. Stewart,et al.  BGP4 : inter-domain routing in the Internet , 1998 .

[28]  Matthew Roughan,et al.  Simplifying the synthesis of internet traffic matrices , 2005, CCRV.

[29]  Walter Willinger,et al.  To Peer or Not to Peer: Modeling the Evolution of the Internet's AS-Level Topology , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[30]  Mira Dontcheva,et al.  How to Resolve IP Aliases , 2008 .

[31]  Abraham Silberschatz,et al.  Stable egress route selection for interdomain traffic engineering: model and analysis , 2005, 13TH IEEE International Conference on Network Protocols (ICNP'05).

[32]  Antonio Nucci,et al.  Design of IGP Link Weights for Estimation of Traffic Matrices. , 2004, INFOCOM 2004.

[33]  Kenneth L. Calvert,et al.  Modeling Internet topology , 1997, IEEE Commun. Mag..

[34]  Walter Willinger,et al.  Understanding Internet topology: principles, models, and validation , 2005, IEEE/ACM Transactions on Networking.

[35]  Carsten Lund,et al.  Estimating point-to-point and point-to-multipoint traffic matrices: an information-theoretic approach , 2005, IEEE/ACM Transactions on Networking.

[36]  Renata Teixeira,et al.  In search of path diversity in ISP networks , 2003, IMC '03.

[37]  Kavé Salamatian,et al.  Traffic matrix estimation: existing techniques and new directions , 2002, SIGCOMM '02.

[38]  Albert G. Greenberg,et al.  Experience in measuring backbone traffic variability: models, metrics, measurements and meaning , 2002, IMW '02.

[39]  Hyunseok Chang,et al.  An economic-based empirical approach to modeling the Internet's inter-domain topology and traffic matrix , 2006 .

[40]  Walter Willinger,et al.  Sampling Techniques for Large, Dynamic Graphs , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[41]  Michael A. West,et al.  Bayesian Inference on Network Traffic Using Link Count Data , 1998 .

[42]  S. Low,et al.  The "robust yet fragile" nature of the Internet. , 2005, Proceedings of the National Academy of Sciences of the United States of America.

[43]  Anthony Bonato,et al.  Infinite Limits of Copying Models of the Web Graph , 2004, Internet Math..

[44]  Matthew Roughan,et al.  Traffic Matrix Reloaded: Impact of Routing Changes , 2005, PAM.

[45]  Walter Willinger,et al.  The origin of power laws in Internet topologies revisited , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[46]  Alan M. Frieze,et al.  Random graphs , 2006, SODA '06.

[47]  Priya Mahadevan,et al.  Systematic topology analysis and generation using degree correlations , 2006, SIGCOMM.

[48]  Walter Willinger,et al.  A first-principles approach to understanding the internet's router-level topology , 2004, SIGCOMM '04.

[49]  Anders Gunnar,et al.  Performance of Traffic Engineering using Estimated Traffic Matrices , 2005 .

[50]  Ravi Kumar,et al.  Self-similarity in the web , 2001, TOIT.

[51]  Ian T. Foster,et al.  Mapping the Gnutella Network: Properties of Large-Scale Peer-to-Peer Systems and Implications for System Design , 2002, ArXiv.

[52]  Mikkel Thorup,et al.  Traffic engineering with estimated traffic matrices , 2003, IMC '03.

[53]  Matei Ripeanu,et al.  Peer-to-peer architecture case study: Gnutella network , 2001, Proceedings First International Conference on Peer-to-Peer Computing.

[54]  Michael Mitzenmacher,et al.  A Brief History of Generative Models for Power Law and Lognormal Distributions , 2004, Internet Math..

[55]  Bin Yu,et al.  Maximum pseudo likelihood estimation in network tomography , 2003, IEEE Trans. Signal Process..

[56]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[57]  Micah Adler,et al.  Towards compressing Web graphs , 2001, Proceedings DCC 2001. Data Compression Conference.

[58]  Ibrahim Matta,et al.  On the origin of power laws in Internet topologies , 2000, CCRV.

[59]  Eli Upfal,et al.  Stochastic models for the Web graph , 2000, Proceedings 41st Annual Symposium on Foundations of Computer Science.

[60]  Mikael Johansson,et al.  Traffic matrix estimation on a large IP backbone: a comparison on real data , 2004, IMC '04.

[61]  Lixin Gao,et al.  On inferring autonomous system relationships in the Internet , 2000, Globecom '00 - IEEE. Global Telecommunications Conference. Conference Record (Cat. No.00CH37137).

[62]  Matthew Doar,et al.  A better model for generating test networks , 1996, Proceedings of GLOBECOM'96. 1996 IEEE Global Telecommunications Conference.

[63]  Mark E. J. Newman,et al.  The Structure and Function of Complex Networks , 2003, SIAM Rev..

[64]  C. Berenstein,et al.  Network tomography , 2007 .

[65]  Gary Scott Malkin,et al.  Traceroute Using an IP Option , 1993, RFC.

[66]  Walter Willinger,et al.  Internet connectivity at the AS-level: an optimization-driven modeling approach , 2003, MoMeTools '03.

[67]  Anja Feldmann,et al.  Building an AS-topology model that captures route diversity , 2006, SIGCOMM.

[68]  Alan M. Frieze,et al.  A general model of web graphs , 2003, Random Struct. Algorithms.

[69]  Fan Chung Graham,et al.  A random graph model for massive graphs , 2000, STOC '00.

[70]  Lixin Gao On inferring autonomous system relationships in the internet , 2001, TNET.

[71]  Antonio Nucci,et al.  The problem of synthetically generating IP traffic matrices: initial recommendations , 2005, CCRV.

[72]  Lada A. Adamic,et al.  Search in Power-Law Networks , 2001, Physical review. E, Statistical, nonlinear, and soft matter physics.

[73]  J. Tinbergen Shaping the World Economy: Suggestions for an International Economic Policy , 1964 .

[74]  Walter Willinger,et al.  Toward an optimization-driven framework for designing and generating realistic Internet topologies , 2003, CCRV.

[75]  Matthew Roughan,et al.  Experience in measuring internet backbone traffic variability: Models metrics, measurements and meaning , 2003 .

[76]  Anthony Bonato,et al.  A Survey of Models of the Web Graph , 2004, CAAN.

[77]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[78]  George Varghese,et al.  The measurement manifesto , 2004, Comput. Commun. Rev..

[79]  Mikkel Thorup,et al.  Traffic engineering with traditional IP routing protocols , 2002, IEEE Commun. Mag..

[80]  Ratul Mahajan,et al.  Measuring ISP topologies with Rocketfuel , 2004, IEEE/ACM Transactions on Networking.

[81]  P. Hansen Rank-Deficient and Discrete Ill-Posed Problems: Numerical Aspects of Linear Inversion , 1987 .

[82]  Renata Teixeira,et al.  Network sensitivity to hot-potato disruptions , 2004, SIGCOMM '04.

[83]  Daniel Stutzbach,et al.  Evaluating the Accuracy of Captured Snapshots by Peer-to-Peer Crawlers , 2005, PAM.

[84]  Stefan Saroiu,et al.  A Measurement Study of Peer-to-Peer File Sharing Systems , 2001 .

[85]  Yuval Shavitt,et al.  DIMES: let the internet measure itself , 2005, CCRV.

[86]  Anja Feldmann,et al.  Deriving traffic demands for operational IP networks: methodology and experience , 2000, SIGCOMM.

[87]  BERNARD M. WAXMAN,et al.  Routing of multipoint connections , 1988, IEEE J. Sel. Areas Commun..

[88]  Mark Crovella,et al.  Diagnosing network-wide traffic anomalies , 2004, SIGCOMM '04.

[89]  Walter Willinger,et al.  Network topology generators: degree-based vs. structural , 2002, SIGCOMM '02.

[90]  Anja Feldmann,et al.  A methodology for estimating interdomain web traffic demand , 2004, IMC '04.

[91]  Vern Paxson,et al.  Strategies for sound internet measurement , 2004, IMC '04.

[92]  Matthew Roughan,et al.  Large-scale measurement and modeling of backbone Internet traffic , 2002, SPIE ITCom.

[93]  Fan Chung Graham,et al.  Coupling Online and Offline Analyses for Random Power Law Graphs , 2004, Internet Math..

[94]  Jacek Kowalski,et al.  Modelling Traffic Demand Between Nodes in a Telecommunications Network , 1995 .

[95]  R. Gomory,et al.  Fractals and Scaling in Finance: Discontinuity, Concentration, Risk. Selecta Volume E , 1997 .

[96]  Walter Willinger,et al.  Towards capturing representative AS-level Internet topologies , 2002, SIGMETRICS '02.

[97]  W. Willinger,et al.  Difficulties Measuring the Internet's AS-Level Ecosystem , 2006, 2006 40th Annual Conference on Information Sciences and Systems.

[98]  Ajoy Kumar Datta,et al.  Stabilizing inter-domain routing in the Internet , 2005, J. High Speed Networks.

[99]  Mikkel Thorup,et al.  Performance of estimated traffic matrices in traffic engineering , 2003, SIGMETRICS '03.

[100]  Randy H. Katz,et al.  Characterizing the Internet hierarchy from multiple vantage points , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[101]  Albert-László Barabási,et al.  Statistical mechanics of complex networks , 2001, ArXiv.

[102]  Mark Crovella,et al.  Characterization of network-wide anomalies in traffic flows , 2004, IMC '04.

[103]  Daniel Stutzbach,et al.  Understanding churn in peer-to-peer networks , 2006, IMC '06.

[104]  Andrei Z. Broder,et al.  Graph structure in the Web , 2000, Comput. Networks.

[105]  Larry Peterson,et al.  Inter-AS traffic patterns and their implications , 1999, Seamless Interconnection for Universal Services. Global Telecommunications Conference. GLOBECOM'99. (Cat. No.99CH37042).

[106]  Albert-László Barabási,et al.  Internet: Diameter of the World-Wide Web , 1999, Nature.

[107]  W. Norton The Evolution of the U.S. Internet Peering Ecosystem , 2003 .

[108]  Anja Feldmann,et al.  NetScope: traffic engineering for IP networks , 2000, IEEE Netw..

[109]  Anees Shaikh,et al.  A measurement-based analysis of multihoming , 2003, SIGCOMM '03.

[110]  Priya Mahadevan,et al.  The internet AS-level topology: three data sources and one definitive metric , 2005, Comput. Commun. Rev..

[111]  Walter Willinger,et al.  Scaling phenomena in the Internet: Critically examining criticality , 2002, Proceedings of the National Academy of Sciences of the United States of America.

[112]  Doyle,et al.  Power laws, highly optimized tolerance, and generalized source coding , 2000, Physical review letters.

[113]  Fan Chung Graham,et al.  A Random Graph Model for Power Law Graphs , 2001, Exp. Math..

[114]  M. Newman Power laws, Pareto distributions and Zipf's law , 2005 .

[115]  Steve Uhlig,et al.  Providing public intradomain traffic matrices to the research community , 2006, CCRV.

[116]  Ellen W. Zegura,et al.  A quantitative comparison of graph-based models for Internet topology , 1997, TNET.

[117]  Christos H. Papadimitriou,et al.  Heuristically Optimized Trade-Offs: A New Paradigm for Power Laws in the Internet , 2002, ICALP.

[118]  Walter Willinger,et al.  A pragmatic approach to dealing with high-variability in network measurements , 2004, IMC '04.

[119]  R. M. Oliver,et al.  Flows in transportation networks , 1972 .

[120]  Olivier Bonaventure,et al.  Implications of the topological properties of Internet traffic on traffic engineering , 2004, SAC '04.

[121]  Michalis Faloutsos,et al.  Power laws and the AS-level internet topology , 2003, TNET.

[122]  Carsten Lund,et al.  An information-theoretic approach to traffic matrix estimation , 2003, SIGCOMM '03.

[123]  Sandrine Vaton,et al.  Network tomography: an iterative Bayesian analysis , 2003 .