On the internet delay space dimensionality

We investigate the dimensionality properties of the Internet delay space, i.e., the matrix of measured round-trip latencies between Internet hosts. Previous work on network coordinates has indicated that this matrix can be embedded, with reasonably low distortion, into a 4- to 9-dimensional Euclidean space. The application of Principal Component Analysis (PCA) reveals the same dimensionality values. Our work addresses the question: to what extent is the dimensionality an intrinsic property of the delay space, defined without reference to a host metric such as Euclidean space? Is the intrinsic dimensionality of the Internet delay space approximately equal to the dimension determined using embedding techniques or PCA? If not, what explains the discrepancy? What properties of the network contribute to its overall dimensionality? Using datasets obtained via the King [14] method, we study different measures of dimensionality to establish the following conclusions. First, based on its power-law behavior, the structure of the delay space can be better characterized by fractal measures. Second, the intrinsic dimension is significantly smaller than the value predicted by the previous studies; in fact by our measures it is less than 2. Third, we demonstrate a particular way in which the AS topology is reflected in the delay space; subnetworks composed of hosts which share an upstream Tier-1 autonomous system in common possess lower dimensionality than the combined delay space. Finally, we observe that fractal measures, due to their sensitivity to non-linear structures, display higher precision for measuring the influence of subtle features of the delay space geometry.

[1]  Margo I. Seltzer,et al.  Network Coordinates in the Wild , 2007, NSDI.

[2]  Sally Floyd,et al.  Why we don't know how to simulate the Internet , 1997, WSC '97.

[3]  Gunnar Rätsch,et al.  An introduction to kernel-based learning algorithms , 2001, IEEE Trans. Neural Networks.

[4]  Piotr Indyk,et al.  Low-distortion embeddings of general metrics into the line , 2005, STOC '05.

[5]  Jon Crowcroft,et al.  On the accuracy of embeddings for internet coordinate systems , 2005, IMC '05.

[6]  Robert Tappan Morris,et al.  Vivaldi: a decentralized network coordinate system , 2004, SIGCOMM '04.

[7]  Jon M. Kleinberg,et al.  Metric embeddings with relaxed guarantees , 2005, 46th Annual IEEE Symposium on Foundations of Computer Science (FOCS'05).

[8]  Laurent Viennot,et al.  The Inframetric Model for the Internet , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[9]  J. Bourgain On lipschitz embedding of finite metric spaces in Hilbert space , 1985 .

[10]  Azer Bestavros,et al.  Self-similarity in World Wide Web traffic: evidence and possible causes , 1996, SIGMETRICS '96.

[11]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[12]  W. B. Johnson,et al.  Extensions of Lipschitz mappings into Hilbert space , 1984 .

[13]  B. Huffaker,et al.  Distance Metrics in the Internet , 2002, Anais do 2002 International Telecommunications Symposium.

[14]  Mark Crovella,et al.  Geometric Exploration of the Landmark Selection Problem , 2004, PAM.

[15]  Sally Floyd,et al.  Wide area traffic: the failure of Poisson modeling , 1995, TNET.

[16]  Jon M. Kleinberg,et al.  Triangulation and embedding using small sets of beacons , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[17]  Ann B. Lee,et al.  Diffusion maps and coarse-graining: a unified framework for dimensionality reduction, graph partitioning, and data set parameterization , 2006, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[18]  Bo Zhang,et al.  Measurement-Based Analysis, Modeling, and Synthesis of the Internet Delay Space , 2006, IEEE/ACM Transactions on Networking.

[19]  Yuval Shavitt,et al.  On the curvature of the Internet and its usage for overlay construction and distance estimation , 2004, IEEE INFOCOM 2004.

[20]  Hui Zhang,et al.  Predicting Internet network distance with coordinates-based approaches , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[21]  Robert D. Kleinberg,et al.  On the internet delay space dimensionality , 2008, PODC '08.

[22]  Sanjoy Dasgupta,et al.  Adaptive Control Processes , 2010, Encyclopedia of Machine Learning and Data Mining.

[23]  Yuval Shavitt,et al.  Big-Bang simulation for embedding network distances in Euclidean space , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[24]  Sonia Fahmy,et al.  A Hierarchical Approach to Internet Distance Prediction , 2006, 26th IEEE International Conference on Distributed Computing Systems (ICDCS'06).

[25]  Robert Krauthgamer,et al.  Bounded geometries, fractals, and low-distortion embeddings , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[26]  Krishna P. Gummadi,et al.  King: estimating latency between arbitrary internet end hosts , 2002, IMW '02.

[27]  Walter Willinger,et al.  In search of the elusive ground truth: the internet's as-level connectivity structure , 2008, SIGMETRICS '08.

[28]  Walter Willinger,et al.  On the self-similar nature of Ethernet traffic , 1993, SIGCOMM '93.

[29]  Miguel Castro,et al.  PIC: practical Internet coordinates for distance estimation , 2004, 24th International Conference on Distributed Computing Systems, 2004. Proceedings..

[30]  Mark Handley,et al.  Topologically-aware overlay construction and server selection , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[31]  Jon Crowcroft,et al.  Lighthouses for Scalable Distributed Location , 2003, IPTPS.

[32]  Arun Venkataramani,et al.  A structural approach to latency prediction , 2006, IMC '06.

[33]  Piotr Indyk,et al.  Algorithmic applications of low-distortion geometric embeddings , 2001, Proceedings 2001 IEEE International Conference on Cluster Computing.

[34]  Funabashi,et al.  Omori's law in the Internet traffic , 2003 .

[35]  Ratul Mahajan,et al.  Colt ? ? ? ? ? ? ◦ DTAG ? ◦ • ◦ ? ? ? ? ! ◦ ? ? ? ◦ ◦ ? ? Eqip ? ? ? ? ? ? , 2003 .

[36]  Mark Crovella,et al.  Virtual landmarks for the internet , 2003, IMC '03.

[37]  Emin Gün Sirer,et al.  Meridian: a lightweight network location service without virtual coordinates , 2005, SIGCOMM '05.

[38]  R. Bellman,et al.  V. Adaptive Control Processes , 1964 .

[39]  Bo Zhang,et al.  Towards network triangle inequality violation aware distributed systems , 2007, IMC '07.

[40]  Ratul Mahajan,et al.  The causes of path inflation , 2003, SIGCOMM '03.

[41]  Arun Venkataramani,et al.  iPlane: an information plane for distributed services , 2006, OSDI '06.

[42]  Christos Faloutsos,et al.  Estimating the Selectivity of Spatial Queries Using the 'Correlation' Fractal Dimension , 1995, VLDB.

[43]  J. Tenenbaum,et al.  A global geometric framework for nonlinear dimensionality reduction. , 2000, Science.