The Internet Dark Matter - on the Missing Links in the AS Connectivity Map

The topological structure of the Internet infrastructure is an important and interesting subject that attracted significant research attention in the last few years. Apart from the pure intellectual challenge of understanding a very big, complex, and ever evolving system, knowing the structure of the Internet topology is very important for developing and studying new protocols and algorithms. Starting with the fundamental work of Falostous et. al, a considerable amount of work was done recently in this field, improving our knowledge and understanding of the Internet structure. However, one basic problem is still unanswered: how big is the Internet. In the AS level this means: how many peering relations exist between ASs. Finding this number is hard since there is no direct way to retrieve information from all nodes regarding their direct neighbors, and all our knowledge is based on sampling processes. Thus, it is very difficult to characterize the Internet since it may well be the case that this characterization is a result of the sampling process, and it does not hold for the “real” Internet. In this paper we attack this problem by suggesting a novel usage of the measurements themselves in order to infer information regarding the whole system. In other words, in addition to looking at the overall graph that is generated from the union of the data obtained by performing many measurements, we consider the actual different measurements and the amount of new data obtained in each of them with respect to the previous collected data. Using the second moment allows us to reach conclusions regarding the structure of the system we are measuring, and in particular to estimate its total size. We present strong evidence to the fact that a considerable amount (at least 35%) of the links in the AS level are still to be unveiled. Our findings indicate that almost all these missing links are of type peer-peer, and we provide novel insight regarding the structure of the AS connectivity map with respect to the peering type.

[1]  Lixin Gao.  On inferring autonomous system relationships in the internet , 2001, TNET.

[2]  Shi Zhou,et al.  The missing links in the BGP-based AS connectivity maps , 2003, ArXiv.

[3]  Leonard M. Freeman,et al.  A set of measures of centrality based upon betweenness , 1977 .

[4]  Cengiz Alaettinoglu,et al.  Using RPSL in Practice , 1999, RFC.

[5]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[6]  Peng Xie,et al.  Sampling biases in IP topology measurements , 2003, IEEE INFOCOM 2003. Twenty-second Annual Joint Conference of the IEEE Computer and Communications Societies (IEEE Cat. No.03CH37428).

[7]  G. Huston,et al.  Interconnection, Peering and Settlements , 2003 .

[8]  Cengiz Alaettinoglu,et al.  Routing Policy Specification Language (RPSL) , 1998, RFC.

[9]  Walter Willinger,et al.  Towards capturing representative AS-level Internet topologies , 2004, Comput. Networks.

[10]  Ramesh Govindan,et al.  An analysis of Internet inter-domain topology and route stability , 1997, Proceedings of INFOCOM '97.

[11]  Thomas Erlebach,et al.  Computing the types of the relationships between autonomous systems , 2007, IEEE/ACM Trans. Netw..

[12]  BERNARD M. WAXMAN,et al.  Routing of multipoint connections , 1988, IEEE J. Sel. Areas Commun..

[13]  Walter Willinger,et al.  Towards capturing representative AS-level Internet topologies , 2002, SIGMETRICS '02.

[14]  V. Rich.  Personal communication , 1989, Nature.

[15]  Michalis Faloutsos,et al.  Analyzing BGP policies: methodology and tool , 2004, IEEE INFOCOM 2004.

[16]  Albert,et al.  Topology of evolving networks: local events and universality , 2000, Physical review letters.

[17]  Walter Willinger,et al.  Internet connectivity at the AS-level: an optimization-driven modeling approach , 2003, MoMeTools '03.

[18]  Albert,et al.  Emergence of scaling in random networks , 1999, Science.

[19]  Jia Wang,et al.  Towards an accurate AS-level traceroute tool , 2003, SIGCOMM '03.

[20]  Lixin Gao,et al.  Stable Internet routing without global coordination , 2000, SIGMETRICS '00.

[21]  Randy H. Katz,et al.  Characterizing the Internet hierarchy from multiple vantage points , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[22]  Jennifer Rexford,et al.  Inherently safe backup routing with BGP , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).