EAalo: Enhanced coflow scheduling without prior knowledge in a datacenter network

Coflow scheduling without prior knowledge has been proposed recently. However, the previous solution, Aalo, has two problems: it does not explicitly control the flow rate; it assumes the network is ideally non-blocking (with perfect traffic balancing and no oversubscription). In this paper, we show the performance loss caused by the above two problems. We propose EAalo to cope with the problems. It has three enhancements to Aalo: per-flow bandwidth enforcement, centralized traffic balancing and oversubscription adaption. We evaluate EAalo with simulations. The simulation results show that EAalo has much better performance than Aalo. It can speed up coflow completion by up to 11%, 17% and 19% in non-blocking networks, networks without oversubscription and networks with 2∶1 oversubscription respectively.

[1]  Amin Vahdat,et al.  Hedera: Dynamic Flow Scheduling for Data Center Networks , 2010, NSDI.

[2]  Ion Stoica,et al.  Efficient Coflow Scheduling Without Prior Knowledge , 2015, SIGCOMM.

[3]  Keqiang He,et al.  AC/DC TCP: Virtual Congestion Control Enforcement for Datacenter Networks , 2016, SIGCOMM.

[4]  Ming Zhang,et al.  MicroTE: fine grained traffic engineering for data centers , 2011, CoNEXT '11.

[5]  Abdul Kabbani,et al.  FlowBender: Flow-level Adaptive Routing for Improved Latency and Throughput in Datacenter Networks , 2014, CoNEXT.

[6]  Brighten Godfrey,et al.  Micro Load Balancing in Data Centers with DRILL , 2015, HotNets.

[7]  Marco Chiesa,et al.  Traffic engineering with Equal-Cost-Multipath: An algorithmic perspective , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[8]  George Varghese,et al.  CONGA: distributed congestion-aware load balancing for datacenters , 2015, SIGCOMM.

[9]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[10]  Haitao Wu,et al.  Per-packet load-balanced, low-latency routing for clos-based data center networks , 2013, CoNEXT.

[11]  StoicaIon,et al.  Efficient Coflow Scheduling Without Prior Knowledge , 2015 .

[12]  Randy H. Katz,et al.  DeTail: reducing the flow completion time tail in datacenter networks , 2012, SIGCOMM '12.

[13]  Amin Vahdat,et al.  Jupiter Rising: A Decade of Clos Topologies and Centralized Control in Google's Datacenter Network , 2015, Comput. Commun. Rev..

[14]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[15]  Antony I. T. Rowstron,et al.  Decentralized task-aware scheduling for data center networks , 2014, SIGCOMM.

[16]  Keqiang He,et al.  Presto: Edge-based Load Balancing for Fast Datacenter Networks , 2015, Conference on Applications, Technologies, Architectures, and Protocols for Computer Communication.

[17]  Yanhui Geng,et al.  CODA: Toward Automatically Identifying and Scheduling Coflows in the Dark , 2016, SIGCOMM.

[18]  Sheng Wang,et al.  Rapier: Integrating routing and scheduling for coflow-aware data center networks , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[19]  Michael I. Jordan,et al.  Managing data transfers in computer clusters with orchestra , 2011, SIGCOMM.

[20]  Hua Chen,et al.  Pingmesh: A Large-Scale System for Data Center Network Latency Measurement and Analysis , 2015, SIGCOMM.

[21]  Jennifer Rexford,et al.  HULA: Scalable Load Balancing Using Programmable Data Planes , 2016, SOSR.

[22]  Ion Stoica,et al.  Coflow: a networking abstraction for cluster applications , 2012, HotNets-XI.

[23]  Mark Handley,et al.  Improving datacenter performance and robustness with multipath TCP , 2011, SIGCOMM 2011.

[24]  Ion Stoica,et al.  Efficient coflow scheduling with Varys , 2015, SIGCOMM.