Where the Sidewalk Ends: Extending theInternet AS Graph Using Traceroutesfrom P2P Users

An accurate Internet topology graph is important in many areas of networking, from understanding ISP business relationships to diagnosing network anomalies. Most Internet mapping efforts have derived the network structure, at the level of interconnected autonomous systems (ASes), from a rather limited set of vantage points. In this paper, we argue that a promising approach to revealing the hidden areas of the Internet topology is through active measurement from an observation platform that scales with the growing Internet. By leveraging measurements performed by an extension to a popular P2P system, we show that this approach indeed exposes significant new topological information. Our study is based on traceroute measurements from more than 992,000 IPs in over 3,700 ASes distributed across the Internet hierarchy, many in regions of the Internet not covered by publicly available path information. To address this issue we develop heuristics that identify 23,914 new AS links not visible in the publicly-available BGP data-12.86 percent more customer-provider links and 40.99 percent more peering links, than previously reported. We validate our heuristics using data from a tier-1 ISP, and show that they successfully filter out all false links introduced by public IP-to-AS mapping. We analyze properties of the Internet graph that includes these new links and characterize why they are missing. Finally, we have made the identified set of links and their inferred relationships publicly available.

[1]  W. Weibull A Statistical Distribution Function of Wide Applicability , 1951 .

[2]  [PCB]. , 2020, Nihon eiseigaku zasshi. Japanese journal of hygiene.

[3]  Michalis Faloutsos,et al.  On power-law relationships of the Internet topology , 1999, SIGCOMM '99.

[4]  Lixin Gao,et al.  On inferring autonomous system relationships in the Internet , 2000, Globecom '00 - IEEE. Global Telecommunications Conference. Conference Record (Cat. No.00CH37137).

[5]  Sally Floyd,et al.  Difficulties in simulating the internet , 2001, TNET.

[6]  Kihong Park,et al.  On the effectiveness of route-based packet filtering for distributed DoS attack prevention in power-law internets , 2001, SIGCOMM '01.

[7]  A. Feldmann,et al.  Realistic BGP traffic for test labs , 2002, SIGCOMM '02.

[8]  Walter Willinger,et al.  The origin of power laws in Internet topologies revisited , 2002, Proceedings.Twenty-First Annual Joint Conference of the IEEE Computer and Communications Societies.

[9]  Jia Wang,et al.  Towards an accurate AS-level traceroute tool , 2003, SIGCOMM '03.

[10]  Renata Teixeira,et al.  A measurement framework for pin-pointing routing changes , 2004, NetT '04.

[11]  Anja Feldmann,et al.  Locating internet routing instabilities , 2004, SIGCOMM 2004.

[12]  Walter Willinger,et al.  Towards capturing representative AS-level Internet topologies , 2004, Comput. Networks.

[13]  Lixin Gao,et al.  On the evaluation of AS relationship inferences [Internet reachability/traffic flow applications] , 2004, IEEE Global Telecommunications Conference, 2004. GLOBECOM '04..

[14]  Yuval Shavitt,et al.  DIMES: let the internet measure itself , 2005, CCRV.

[15]  Mark Handley,et al.  HLP: a next generation inter-domain routing protocol , 2005, SIGCOMM '05.

[16]  Rami Cohen,et al.  The Internet Dark Matter - on the Missing Links in the AS Connectivity Map , 2006, Proceedings IEEE INFOCOM 2006. 25TH IEEE International Conference on Computer Communications.

[17]  Brice Augustin,et al.  Avoiding traceroute anomalies with Paris traceroute , 2006, IMC '06.

[18]  Arun Venkataramani,et al.  iPlane: an information plane for distributed services , 2006, OSDI '06.

[19]  Lixia Zhang,et al.  Observing the evolution of internet as topology , 2007, SIGCOMM 2007.

[20]  Lorenzo Colitti,et al.  An active approach to measuring routing dynamics induced by autonomous systems , 2007, ExpCS '07.

[21]  Michalis Faloutsos,et al.  A Systematic Framework for Unearthing the Missing Links: Measurements and Impact , 2007, NSDI.

[22]  Walter Willinger,et al.  In search of the elusive ground truth: the internet's as-level connectivity structure , 2008, SIGMETRICS '08.

[23]  Ricardo V. Oliveira,et al.  Quantifying the Completeness of the Observed Internet AS-level Structure , 2008 .

[24]  Fabián E. Bustamante,et al.  Taming the torrent: a practical approach to reducing cross-isp traffic in peer-to-peer systems , 2008, SIGCOMM '08.

[25]  Yuval Shavitt,et al.  Quantifying the Importance of Vantage Points Distribution in Internet Topology Measurements , 2009, IEEE INFOCOM 2009.

[26]  Brice Augustin,et al.  IXPs: mapped? , 2009, IMC '09.

[27]  Kimberly C. Claffy,et al.  Toward Topology Dualism: Improving the Accuracy of AS Annotations for Routers , 2010, PAM.

[28]  Walter Willinger,et al.  The (In)Completeness of the Observed Internet AS-level Structure , 2010, IEEE/ACM Transactions on Networking.

[29]  Yuval Shavitt,et al.  Quantifying the Importance of Vantage Point Distribution in Internet Topology Mapping (Extended Version) , 2011, IEEE Journal on Selected Areas in Communications.