B4: experience with a globally-deployed software defined wan

We present the design, implementation, and evaluation of B4, a private WAN connecting Google's data centers across the planet. B4 has a number of unique characteristics: i) massive bandwidth requirements deployed to a modest number of sites, ii) elastic traffic demand that seeks to maximize average bandwidth, and iii) full control over the edge servers and network, which enables rate limiting and demand measurement at the edge. These characteristics led to a Software Defined Networking architecture using OpenFlow to control relatively simple switches built from merchant silicon. B4's centralized traffic engineering service drives links to near 100% utilization, while splitting application flows among multiple paths to balance capacity against application priority/demands. We describe experience with three years of B4 production deployment, lessons learned, and areas for future work.

[1]  David Johnson,et al.  Network architecture for joint failure recovery and traffic engineering , 2011, SIGMETRICS '11.

[2]  Mung Chiang,et al.  Link-State Routing with Hop-by-Hop Forwarding Can Achieve Optimal Traffic Engineering , 2008, IEEE INFOCOM 2008 - The 27th Conference on Computer Communications.

[3]  Daniel O. Awduche,et al.  Requirements for Traffic Engineering Over MPLS , 1999, RFC.

[4]  Martín Casado,et al.  NOX: towards an operating system for networks , 2008, CCRV.

[5]  Albert G. Greenberg,et al.  COPE: traffic engineering in dynamic networks , 2006, SIGCOMM.

[6]  Nick Feamster,et al.  Design and implementation of a routing control platform , 2005, NSDI.

[7]  Martín Casado,et al.  Onix: A Distributed Control Platform for Large-scale Production Networks , 2010, OSDI.

[8]  Christophe Diot,et al.  Traffic matrix estimation: existing techniques and new directions , 2002, SIGCOMM 2002.

[9]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[10]  Mikkel Thorup,et al.  Traffic engineering with traditional IP routing protocols , 2002, IEEE Commun. Mag..

[11]  Nick McKeown,et al.  OpenFlow: enabling innovation in campus networks , 2008, CCRV.

[12]  Ian F. Akyildiz,et al.  TEAM: A traffic engineering automated manager for DiffServ-based MPLS networks , 2004, IEEE Communications Magazine.

[13]  M. Allalouf,et al.  Centralized and Distributed Algorithms for Routing and Weighted Max-Min Fair Bandwidth Allocation , 2008, IEEE/ACM Transactions on Networking.

[14]  Cheng Jin,et al.  MATE: MPLS adaptive traffic engineering , 2001, Proceedings IEEE INFOCOM 2001. Conference on Computer Communications. Twentieth Annual Joint Conference of the IEEE Computer and Communications Society (Cat. No.01CH37213).

[15]  Rob Sherwood,et al.  FlowVisor: A Network Virtualization Layer , 2009 .

[16]  Marcos Rogério Salvador,et al.  QuagFlow: partnering Quagga with OpenFlow , 2010, SIGCOMM '10.

[17]  Jennifer Rexford,et al.  Toward internet-wide multipath routing , 2008, IEEE Network.

[18]  Albert G. Greenberg,et al.  Towards a next generation data center architecture: scalability and commoditization , 2008, PRESTO '08.

[19]  T. V. Lakshman,et al.  RATES: a server for MPLS traffic engineering , 2000, IEEE Netw..

[20]  Vijay Srinivasan,et al.  RSVP-TE: Extensions to RSVP for LSP Tunnels , 2001, RFC.

[21]  Mikkel Thorup,et al.  Increasing Internet Capacity Using Local Search , 2004, Comput. Optim. Appl..

[22]  Amin Vahdat,et al.  Data Center Switch Architecture in the Age of Merchant Silicon , 2009, 2009 17th IEEE Symposium on High Performance Interconnects.

[23]  Taesang Choi,et al.  Design and Implementation of Traffic Engineering Server for a Large-Scale MPLS-Based IP Network , 2002, ICOIN.

[24]  Haim Kaplan,et al.  Upward Max Min Fairness , 2012, INFOCOM.

[25]  Jia Wang,et al.  Scalable flow-based networking with DIFANE , 2010, SIGCOMM '10.

[26]  Hong Yan,et al.  A clean slate 4D approach to network control and management , 2005, CCRV.

[27]  Albert G. Greenberg,et al.  VL2: a scalable and flexible data center network , 2009, SIGCOMM '09.

[28]  Martín Casado,et al.  Ethane: taking control of the enterprise , 2007, SIGCOMM '07.

[29]  Srikanth Kandula,et al.  Walking the tightrope: responsive yet stable traffic engineering , 2005, SIGCOMM '05.

[30]  David Thaler,et al.  Multipath Issues in Unicast and Multicast Next-Hop Selection , 2000, RFC.

[31]  Robert Griesemer,et al.  Paxos made live: an engineering perspective , 2007, PODC '07.

[32]  Mung Chiang,et al.  Link-State Routing With Hop-by-Hop Forwarding Can Achieve Optimal Traffic Engineering , 2011, IEEE/ACM Trans. Netw..

[33]  Arjun Singh,et al.  A practical algorithm for balancing the max-min fairness and throughput objectives in traffic engineering , 2012, 2012 Proceedings IEEE INFOCOM.

[34]  Tal Garfinkel,et al.  SANE: A Protection Architecture for Enterprise Networks , 2006, USENIX Security Symposium.

[35]  Srikanth Kandula,et al.  Achieving high utilization with software-driven WAN , 2013, SIGCOMM.

[36]  Marcos Rogério Salvador,et al.  Revisiting routing control platforms with the eyes and muscles of software-defined networking , 2012, HotSDN '12.

[37]  Daniel O. Awduche,et al.  Applicability Statement for Extensions to RSVP for LSP-Tunnels , 2001, RFC.

[38]  Mikkel Thorup,et al.  Traffic engineering with estimated traffic matrices , 2003, IMC '03.

[39]  Yin Zhang,et al.  COPE: traffic engineering in dynamic networks , 2006, SIGCOMM 2006.

[40]  Sujata Banerjee,et al.  DevoFlow: scaling flow management for high-performance networks , 2011, SIGCOMM.

[41]  J. Rexford,et al.  Network architecture for joint failure recovery and traffic engineering , 2011, PERV.