Efficient scheduling for synchronized demands in stochastic networks

There is a rich theory and plethora of algorithms in the literature aiming at the efficient scheduling of stochastic networks. These solutions are predominantly designed under the assumption of traffic demands that are independently generated at network nodes, without any requirement for synchronization among their received services. In this work, we note that many applications, including cloud computing, virtual reality, gaming, autonomous vehicular networks and collaborative design, generate traffic simultaneously at multiple nodes when they arrive, with possibly non-uniform file sizes, whose performance relies on the synchronous completion of the traffic across the network. This calls for the design of new scheduling algorithms that aims to coordinate the service of packets of the same traffic across the network. Towards this end, we propose a novel scheduling algorithm that not only accounts for the heterogeneity of the file size distributions, but also works towards synchronizing the completion time of the same traffic stream across the network. This is achieved by employing two insights that emanate from key motivating examples we develop: (1) the normalization of traffic load with respect to the non-uniform file sizes; and (2) the incorporation of deviation of normalized loads across network nodes that serve synchronized traffic. After establishing the throughput-optimality of our algorithm in general stochastic networks, we perform extensive simulations under various (spanning both wired and wireless) settings to reveal the potential completion time gains that it yields over other throughput-optimal strategies designed under the assumption of independent traffic generation.

[1]  Michael J. Franklin,et al.  Resilient Distributed Datasets: A Fault-Tolerant Abstraction for In-Memory Cluster Computing , 2012, NSDI.

[2]  R. Srikant,et al.  Fair resource allocation in wireless networks using queue-length-based scheduling and congestion control , 2007, TNET.

[3]  Leandros Tassiulas,et al.  Stability properties of constrained queueing systems and scheduling policies for maximum throughput in multihop radio networks , 1990, 29th IEEE Conference on Decision and Control.

[4]  Lei Ying,et al.  Communication Networks - An Optimization, Control, and Stochastic Networks Perspective , 2014 .

[5]  Bo Li,et al.  Adia: Achieving High Link Utilization with Coflow-Aware Scheduling in Data Center Networks , 2019, IEEE Transactions on Cloud Computing.

[6]  Qingkai Liang,et al.  Coflow scheduling in input-queued switches: Optimal delay scaling and algorithms , 2017, IEEE INFOCOM 2017 - IEEE Conference on Computer Communications.

[7]  Yanhui Geng,et al.  CODA: Toward Automatically Identifying and Scheduling Coflows in the Dark , 2016, SIGCOMM.

[8]  Xiaojun Lin,et al.  The Impact of Imperfect Scheduling on Cross-Layer Rate Control in Multihop Wireless Networks , 2022 .

[9]  Ness B. Shroff,et al.  The impact of imperfect scheduling on cross-layer rate control in wireless networks , 2005, Proceedings IEEE 24th Annual Joint Conference of the IEEE Computer and Communications Societies..

[10]  Ion Stoica,et al.  Efficient Coflow Scheduling Without Prior Knowledge , 2015, SIGCOMM.

[11]  Bo Li,et al.  Optimizing coflow completion times with utility max-min fairness , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[12]  Mung Chiang Balancing transport and physical Layers in wireless multihop networks: jointly optimal congestion control and power control , 2005 .

[13]  Iordanis Koutsopoulos,et al.  Streaming big data meets backpressure in distributed network computation , 2016, IEEE INFOCOM 2016 - The 35th Annual IEEE International Conference on Computer Communications.

[14]  Michael J. Neely Energy Optimal Control for Time-Varying Wireless Networks , 2006, IEEE Trans. Inf. Theory.

[15]  Mor Harchol-Balter Performance Modeling and Design of Computer Systems: The M/G/1 Queue and the Inspection Paradox , 2013 .

[16]  Hairong Kuang,et al.  The Hadoop Distributed File System , 2010, 2010 IEEE 26th Symposium on Mass Storage Systems and Technologies (MSST).

[17]  Jipeng Zhou,et al.  Efficient online coflow routing and scheduling , 2016, MobiHoc.

[18]  Ion Stoica,et al.  Efficient coflow scheduling with Varys , 2015, SIGCOMM.

[19]  Sheng Wang,et al.  Minimizing average coflow completion time with decentralized scheduling , 2015, 2015 IEEE International Conference on Communications (ICC).

[20]  Yuan Zhong,et al.  Minimizing the Total Weighted Completion Time of Coflows in Datacenter Networks , 2015, SPAA.

[21]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[22]  Sheng Wang,et al.  Towards Practical and Near-Optimal Coflow Scheduling for Data Center Networks , 2016, IEEE Transactions on Parallel and Distributed Systems.

[23]  Alexander L. Stolyar,et al.  Maximizing Queueing Network Utility Subject to Stability: Greedy Primal-Dual Algorithm , 2005, Queueing Syst. Theory Appl..

[24]  A. Stolyar MaxWeight scheduling in a generalized switch: State space collapse and workload minimization in heavy traffic , 2004 .

[25]  Leandros Tassiulas,et al.  Dynamic scheduling for minimum delay in tandem and parallel constrained queueing models , 1994, Ann. Oper. Res..

[26]  R. Srikant,et al.  Asymptotically tight steady-state queue length bounds implied by drift conditions , 2011, Queueing Syst. Theory Appl..

[27]  Leandros Tassiulas Scheduling and performance limits of networks with constantly changing topology , 1997 .

[28]  Antony I. T. Rowstron,et al.  Decentralized task-aware scheduling for data center networks , 2014, SIGCOMM.