Optimizing Bulk Transfers with Software-Defined Optical WAN

Bulk transfer on the wide-area network (WAN) is a fundamental service to many globally-distributed applications. It is challenging to efficiently utilize expensive WAN bandwidth to achieve short transfer completion time and meet mission-critical deadlines. Advancements in software-defined networking (SDN) and optical hardware make it feasible and beneficial to quickly reconfigure optical devices in the optical layer, which brings a new opportunity for traffic management on the WAN. We present Owan, a novel traffic management system that optimizes wide-area bulk transfers with centralized joint control of the optical and network layers. \sysname can dynamically change the network-layer topology by reconfiguring the optical devices. We develop efficient algorithms to jointly optimize optical circuit setup, routing and rate allocation, and dynamically adapt them to traffic demand changes. We have built a prototype of Owan with commodity optical and electrical hardware. Testbed experiments and large-scale simulations on two ISP topologies and one inter-DC topology show that \sysname completes transfers up to 4.45x faster on average, and up to 1.36x more transfers meet their deadlines, as compared to prior methods that only control the network layer.

[1]  Qunfeng Dong,et al.  WaveCube: A scalable, fault-tolerant, high-performance optical data center architecture , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[2]  Ming Zhang,et al.  Guaranteeing deadlines for inter-datacenter transfers , 2015, EuroSys.

[3]  Amin Vahdat,et al.  Dahu: Commodity switches for direct connect data center networks , 2013, Architectures for Networking and Communications Systems.

[4]  George Varghese,et al.  CONGA: distributed congestion-aware load balancing for datacenters , 2015, SIGCOMM.

[5]  Amin Vahdat,et al.  Integrating microsecond circuit switching into the data center , 2013, SIGCOMM.

[6]  Keren Bergman,et al.  Accelerating cast traffic delivery in data centers leveraging physical layer optics and SDN , 2014, 2014 International Conference on Optical Network Design and Modeling.

[7]  Amin Vahdat,et al.  Helios: a hybrid electrical/optical switch architecture for modular data centers , 2010, SIGCOMM '10.

[8]  Eytan Modiano,et al.  Dynamic reconfiguration and routing algorithms for IP-over-WDM networks with stochastic traffic , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[9]  ZVI GALIL,et al.  Efficient algorithms for finding maximum matching in graphs , 1986, CSUR.

[10]  Ming Zhang,et al.  Efficiently Delivering Online Services over Integrated Infrastructure , 2016, NSDI.

[11]  Srikanth Kandula,et al.  Traffic engineering with forward fault correction , 2014 .

[12]  Arjun Singh,et al.  A practical algorithm for balancing the max-min fairness and throughput objectives in traffic engineering , 2012, 2012 Proceedings IEEE INFOCOM.

[13]  Amin Vahdat,et al.  BwE: Flexible, Hierarchical Bandwidth Allocation for WAN Distributed Computing , 2015, Comput. Commun. Rev..

[14]  Nick McKeown,et al.  pFabric: minimal near-optimal datacenter transport , 2013, SIGCOMM.

[15]  Mark D. Feuer,et al.  Constraint routing and regenerator site concentration in ROADM networks , 2013, IEEE/OSA Journal of Optical Communications and Networking.

[16]  Edith Cohen,et al.  Making intra-domain routing robust to changing and uncertain traffic demands: understanding fundamental tradeoffs , 2003, SIGCOMM '03.

[17]  Ion Stoica,et al.  Efficient coflow scheduling with Varys , 2014 .

[18]  K. Bergman,et al.  An optical programmable network architecture supporting iterative multicast for data-intensive applications , 2014, 2014 Optical Interconnects Conference.

[19]  Xin Jin,et al.  Dynamic scheduling of network updates , 2014 .

[20]  Haim Kaplan,et al.  Upward Max Min Fairness , 2012, 2012 Proceedings IEEE INFOCOM.

[21]  Olivier Bonaventure,et al.  A Declarative and Expressive Approach to Control Forwarding Paths in Carrier-Grade Networks , 2015, SIGCOMM.

[22]  Srinivasan Keshav,et al.  Quartz: a new design element for low-latency DCNs , 2015, SIGCOMM.

[23]  Konstantina Papagiannaki,et al.  c-Through: part-time optics in data centers , 2010, SIGCOMM '10.

[24]  Albert G. Greenberg,et al.  Data center TCP (DCTCP) , 2010, SIGCOMM '10.

[25]  T. N. Vijaykumar,et al.  Deadline-aware datacenter tcp (D2TCP) , 2012, SIGCOMM '12.

[26]  Sujata Banerjee,et al.  DevoFlow: scaling flow management for high-performance networks , 2011, SIGCOMM.

[27]  Devavrat Shah,et al.  Fastpass: a centralized "zero-queue" datacenter network , 2015, SIGCOMM.

[28]  Byrav Ramamurthy,et al.  Virtual topology reconfiguration of wavelength-routed optical WDM networks , 2000, Globecom '00 - IEEE. Global Telecommunications Conference. Conference Record (Cat. No.00CH37137).

[29]  Mikkel Thorup,et al.  Internet traffic engineering by optimizing OSPF weights , 2000, Proceedings IEEE INFOCOM 2000. Conference on Computer Communications. Nineteenth Annual Joint Conference of the IEEE Computer and Communications Societies (Cat. No.00CH37064).

[30]  He Liu,et al.  Circuit Switching Under the Radar with REACToR , 2014, NSDI.

[31]  Xin Wu,et al.  DARD: Distributed Adaptive Routing for Datacenter Networks , 2012, 2012 IEEE 32nd International Conference on Distributed Computing Systems.

[32]  Ming Zhang,et al.  MicroTE: fine grained traffic engineering for data centers , 2011, CoNEXT '11.

[33]  Himanshu Shah,et al.  FireFly: a reconfigurable wireless data center fabric using free-space optics , 2015, SIGCOMM.

[34]  Wei Bai,et al.  PIAS: Practical Information-Agnostic Flow Scheduling for Commodity Data Centers , 2015, IEEE/ACM Transactions on Networking.

[35]  Sujata Banerjee,et al.  ElasticSwitch: practical work-conserving bandwidth guarantees for cloud computing , 2013, SIGCOMM.

[36]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[37]  Ion Stoica,et al.  Efficient Coflow Scheduling Without Prior Knowledge , 2015, SIGCOMM.

[38]  Pascale Vicat-Blanc Primet,et al.  Scheduling deadline-constrained bulk data transfers to minimize network congestion , 2007, Seventh IEEE International Symposium on Cluster Computing and the Grid (CCGrid '07).

[39]  Prasant Mohapatra,et al.  QRON: QoS-aware routing in overlay networks , 2004, IEEE Journal on Selected Areas in Communications.

[40]  Qixiang Cheng,et al.  A low latency optical top of rack switch for data centre networks with minimized processor energy load , 2014, OFC 2014.

[41]  Byrav Ramamurthy,et al.  On provisioning diverse circuits in heterogeneous multi-layer optical networks , 2013, Comput. Commun..

[42]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[43]  Hari Balakrishnan,et al.  Best-path vs. multi-path overlay routing , 2003, IMC '03.

[44]  Antony I. T. Rowstron,et al.  Better never than late: meeting deadlines in datacenter networks , 2011, SIGCOMM.

[45]  Brighten Godfrey,et al.  Finishing flows quickly with preemptive scheduling , 2012, CCRV.

[46]  D. Zats,et al.  DeTail: reducing the flow completion time tail in datacenter networks , 2012, CCRV.

[47]  Srikanth Kandula,et al.  Achieving high utilization with software-driven WAN , 2013, SIGCOMM.

[48]  Amin Vahdat,et al.  Hedera: Dynamic Flow Scheduling for Data Center Networks , 2010, NSDI.

[49]  Sanjay Ranka,et al.  Advance Reservations and Scheduling for Bulk Transfers in Research Networks , 2009, IEEE Transactions on Parallel and Distributed Systems.

[50]  Minghua Chen,et al.  Intra-data-center traffic engineering with ensemble routing , 2013, 2013 Proceedings IEEE INFOCOM.

[51]  Donald F. Towsley,et al.  On the interaction between overlay routing and underlay routing , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[52]  Maurizio A. Bonuccelli,et al.  Scheduling of real-time messages in optical broadcast-and-select networks , 2001, TNET.

[53]  Dahai Xu,et al.  Architectures and Protocols for Capacity Efficient, Highly Dynamic and Highly Resilient Core Networks [Invited] , 2012, IEEE/OSA Journal of Optical Communications and Networking.

[54]  A. Rowstron,et al.  Towards predictable datacenter networks , 2011, SIGCOMM.

[55]  Sujata Banerjee,et al.  Application-driven bandwidth guarantees in datacenters , 2014 .

[56]  Sambit Sahu,et al.  Rethinking the physical layer of data center networks of the next decade: using optics to enable efficient *-cast connectivity , 2013, CCRV.

[57]  Srikanth Kandula,et al.  Calendaring for wide area networks , 2014 .

[58]  Michael Sirivianos,et al.  Inter-datacenter bulk transfers with netstitcher , 2011, SIGCOMM.

[59]  Kyle Guan.  Joint optimization of topology, switching, routing and wavelength assignment , 2007 .

[60]  T. S. Eugene Ng,et al.  A cross-layer SDN control plane for optical multicast-featured datacenters , 2014, HotSDN.

[61]  Srikanth Kandula,et al.  Walking the tightrope: responsive yet stable traffic engineering , 2005, SIGCOMM '05.

[62]  Albert G. Greenberg,et al.  EyeQ: Practical Network Performance Isolation at the Edge , 2013, NSDI.

[63]  Ankit Singla,et al.  OSA: An Optical Switching Architecture for Data Center Networks With Unprecedented Flexibility , 2012, IEEE/ACM Transactions on Networking.