Coflow Scheduling With Unknown Prior Information in Data Center Networks

In order to solve the problem of flow scheduling in cluster computing framework, the scheduling strategy based on coflow has become a research hot spot. A coflow is a collection of data flows between two different stages of the same parallel computing task. Coflow scheduling in the case of unknown prior information depends on the data flow information of the sent part to infer the data size of coflow and allocate the scheduling sequence for coflow, which is easy to cause congestion. In this paper, we design an effective coflow scheduling mechanism namely, Classification According to Ports Number (CAPN). In the mechanism, firstly, coflows are quickly classified according to the Few Ports Number Scheduling First (FPSF) algorithm, and then coflows with different priorities are scheduled and adjusted, which greatly reduce the average coflow completion time (CCT). Simulation results show that compared with the classical Aalo and MCS mechanisms, our CAPN mechanism can reduce the completion time of coflow by by 31.32% and 25.72%, respectively.

[1]  Efficient coflow scheduling with Varys , 2014, SIGCOMM.

[2]  Wei Wang,et al.  Fair Coflow Scheduling without Prior Knowledge , 2018, 2018 IEEE 38th International Conference on Distributed Computing Systems (ICDCS).

[3]  Minyi Guo,et al.  Fast Coflow Scheduling via Traffic Compression and Stage Pipelining in Datacenter Networks , 2019, IEEE Transactions on Computers.

[4]  Theophilus A. Benson,et al.  Providing In-network Support to Coflow Scheduling , 2020, ArXiv.

[5]  Fengyuan Ren,et al.  Distributed Bottleneck-Aware Coflow Scheduling in Data Centers , 2019, IEEE Transactions on Parallel and Distributed Systems.

[6]  Yiming Zhang,et al.  OPTAS: Decentralized flow monitoring and scheduling for tiny tasks , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[7]  Joseph M. Hellerstein,et al.  MapReduce Online , 2010, NSDI.

[8]  Wenyu Qu,et al.  OSTB: Optimizing Fairness and Efficiency for Coflow Scheduling without Prior Knowledge , 2019, 2019 IEEE 21st International Conference on High Performance Computing and Communications; IEEE 17th International Conference on Smart City; IEEE 5th International Conference on Data Science and Systems (HPCC/SmartCity/DSS).

[9]  Rahul Vaze,et al.  Non-clairvoyant Scheduling of Coflows , 2020, 2020 18th International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOPT).

[10]  StoicaIon,et al.  Efficient Coflow Scheduling Without Prior Knowledge , 2015 .

[11]  Sheng Wang,et al.  Rapier: Integrating routing and scheduling for coflow-aware data center networks , 2015, 2015 IEEE Conference on Computer Communications (INFOCOM).

[12]  Li Dongsheng,et al.  Recent Advances in Datacenter Flow Scheduling , 2018 .

[13]  Xiaobo Zhou,et al.  Leveraging Endpoint Flexibility when Scheduling Coflows across Geo-distributed Datacenters , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[14]  Siegfried Schaible,et al.  Fractional programming: The sum-of-ratios case , 2003, Optim. Methods Softw..

[15]  Hong Xu,et al.  Bottleneck-Aware Coflow Scheduling Without Prior Knowledge , 2020, IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS).

[16]  Kai Chen,et al.  Stream: Decentralized opportunistic inter-coflow scheduling for datacenter networks , 2016, 2016 IEEE 24th International Conference on Network Protocols (ICNP).

[17]  Yiming Zhang,et al.  Rate-aware flow scheduling for commodity data center networks , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.