BGP-Multipath Routing in the Internet

BGP-Multipath (BGP-M) is a multipath routing technique for load balancing. Distinct from other techniques deployed at a router inside an Autonomous System (AS), BGP-M is deployed at a border router that has installed multiple inter-domain border links to a neighbour AS. It uses the equal-cost multi-path (ECMP) function of a border router to share traffic to a destination prefix on different border links. Despite recent research interests in multipath routing, there is little study on BGP-M. Here we provide the first measurement and a comprehensive analysis of BGPM routing in the Internet. We extracted information on BGP-M from query data collected from Looking Glass (LG) servers. We revealed that BGP-M has already been extensively deployed and used in the Internet. A particular example is Hurricane Electric (AS6939), a Tier-1 network operator, which has implemented >1,000 cases of BGP-M at 69 of its border routers to prefixes in 611 of its neighbour ASes, including many hyper-giant ASes and large content providers, on both IPv4 and IPv6 Internet. We examined the distribution and operation of BGP-M. We also ran traceroute using RIPE Atlas to infer the routing paths, the schemes of traffic allocation, and the delay on border links. This study provided the state-of-the-art knowledge on BGP-M with novel insights into the unique features and the distinct advantages of BGP-M as an effective and readily available technique for load balancing.

[1]  Clémence Magnien,et al.  Impact of power-law topology on IP-level routing dynamics: Simulation results , 2012, 2012 Proceedings IEEE INFOCOM Workshops.

[2]  Walter Willinger,et al.  On Mapping the Interconnections in Today’s Internet , 2019, IEEE/ACM Transactions on Networking.

[3]  Kimberly C. Claffy,et al.  Toward Topology Dualism: Improving the Accuracy of AS Annotations for Routers , 2010, PAM.

[4]  Robert Beverly,et al.  Diamond-Miner: Comprehensive Discovery of the Internet's Topology Diamonds , 2020, NSDI.

[5]  Christian E. Hopps,et al.  Analysis of an Equal-Cost Multi-Path Algorithm , 2000, RFC.

[6]  David Watson,et al.  An Experimental Study of Internet Path Diversity , 2006, IEEE Transactions on Dependable and Secure Computing.

[7]  Admela Jukan,et al.  A Survey on Internet Multipath Routing and Provisioning , 2015, IEEE Communications Surveys & Tutorials.

[8]  Jeff Tantsura,et al.  Equal-Cost Multipath Considerations for BGP , 2019 .

[9]  Brice Augustin,et al.  Multipath tracing with Paris traceroute , 2007, 2007 Workshop on End-to-End Monitoring Techniques and Services.

[10]  M. Engin Tozal,et al.  Cross-AS (X-AS) Internet topology mapping , 2018, Comput. Networks.

[11]  Steve Uhlig,et al.  Looking for hypergiants in peeringDB , 2018, CCRV.

[12]  Brice Augustin,et al.  Measuring Multipath Routing in the Internet , 2011, IEEE/ACM Transactions on Networking.

[13]  Brice Augustin,et al.  Avoiding traceroute anomalies with Paris traceroute , 2006, IMC '06.

[14]  Rafael Almeida,et al.  Classification of Load Balancing in the Internet , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications.

[15]  Yanghee Choi,et al.  AS-level topology collection through looking glass servers , 2013, Internet Measurement Conference.

[16]  Robert Raszuk,et al.  Internet Exchange BGP Route Server , 2016, RFC.

[17]  Kamil Saraç,et al.  An experimental study on inter-domain routing dynamics using IP-level path traces , 2015, 2015 IEEE 40th Conference on Local Computer Networks (LCN).

[18]  Ítalo S. Cunha,et al.  DTRACK: A System to Predict and Track Internet Path Changes , 2014, IEEE/ACM Transactions on Networking.

[19]  Vaibhav Bajpai,et al.  Revealing the Load-Balancing Behavior of YouTube Traffic on Interdomain Links , 2018, PAM.

[20]  Jonathan M. Smith,et al.  MAP-IT: Multipass Accurate Passive Inferences from Traceroute , 2016, Internet Measurement Conference.

[21]  Benoit Donnet,et al.  NETPerfTrace: Predicting Internet Path Dynamics and Performance with Machine Learning , 2017, Big-DAMA@SIGCOMM.

[22]  V. Paxson End-to-end routing behavior in the internet , 2006, CCRV.

[23]  Ítalo S. Cunha,et al.  PoiRoot: investigating the root cause of interdomain path changes , 2013, SIGCOMM.

[24]  Walter Willinger,et al.  How Cloud Traffic Goes Hiding: A Study of Amazon's Peering Fabric , 2019, Internet Measurement Conference.

[25]  Giuseppe Di Battista,et al.  Towards an Automated Investigation of the Impact of BGP Routing Changes on Network Delay Variations , 2014, PAM.

[26]  Vasileios Giotsas,et al.  Periscope: Unifying Looking Glass Querying , 2016, PAM.

[27]  Timur Friedman,et al.  Multilevel MDA-Lite Paris Traceroute , 2018, Internet Measurement Conference.

[28]  Hiroshi Fujinoki,et al.  Multi-path BGP (MBGP): A solution for improving network bandwidth utilization and defense against link failures in inter-domain routing , 2008, 2008 16th IEEE International Conference on Networks.

[29]  Jonathan M. Smith,et al.  Pushing the Boundaries with bdrmapIT: Mapping Router Ownership at Internet Scale , 2018, Internet Measurement Conference.

[30]  Ananthram Swami,et al.  Network Topology Inference With Partial Information , 2015, IEEE Transactions on Network and Service Management.

[31]  Walter Willinger,et al.  Towards capturing representative AS-level Internet topologies , 2002, SIGMETRICS '02.

[32]  Emile Aben,et al.  On the Diversity of Interdomain Routing in Africa , 2015, PAM.

[33]  Olivier Bonaventure,et al.  Extracting Intra-domain Topology from mrinfo Probing , 2010, PAM.

[34]  Vasileios Giotsas,et al.  Mapping peering interconnections to a facility , 2015, CoNEXT.

[35]  Mark Crovella,et al.  Studying interdomain routing over long timescales , 2013, Internet Measurement Conference.

[36]  David D. Clark,et al.  bdrmap: Inference of Borders Between IP Networks , 2016, Internet Measurement Conference.

[37]  Susan Hares,et al.  A Border Gateway Protocol 4 (BGP-4) , 1994, RFC.

[38]  Brice Augustin,et al.  Failure Control in Multipath Route Tracing , 2009, IEEE INFOCOM 2009.

[39]  Yakov Rekhter,et al.  A Border Gateway Protocol 4 (BGP-4) , 1994, RFC.

[40]  Kok-Lim Alvin Yau,et al.  Exploiting the Power of Multiplicity: A Holistic Survey of Network-Layer Multipath , 2015, IEEE Communications Surveys & Tutorials.

[41]  Arun Venkataramani,et al.  iPlane: an information plane for distributed services , 2006, OSDI '06.

[42]  Ítalo S. Cunha,et al.  A Characterization of Load Balancing on the IPv6 Internet , 2017, PAM.