A State-Merging Scheduling Method for Bulk Transfers with Store-and-Forward over Inter-DC Optical Networks

The time- and space-varying nature of residual bandwidth in inter-datacenter networks (inter-DCNs) makes conventional end-to-end connections difficult to fully utilize the residual bandwidth. A promising solution is to introduce datacenter storage into data-plane paths. Delay-tolerant bulk data can be temporarily stored at intermediate sites and forwarded (SnF) at a later time when inter-DCN is less congested. However, the conventional methods attempt to involve multi-dimensional state information of the entire network in scheduling, which results in high computational complexity. In this paper, our studies reveal that there exist redundant states in scheduling, which cannot provide any performance benefit while imposing extra computational burden. Inspired by this finding, we propose a state-merging scheduling (SMS) method. By merging the state information of the pre-selected links, the SMS method naturally reduces the redundant states involved, which greatly improves the efficiency of SnF scheduling. Simulations demonstrate that the SMS method can outperform the conventional scheduling method, given a limit of the computational cost.

[1]  Alex X. Liu,et al.  Multiple bulk data transfers scheduling among datacenters , 2014, Comput. Networks.

[2]  Eli Dart,et al.  The Science DMZ: A network design pattern for data-intensive science , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[3]  Xiao Lin,et al.  Time-shifted multilayer graph: A routing framework for bulk data transfer in optical circuit-switched networks with assistive storage , 2016, IEEE/OSA Journal of Optical Communications and Networking.

[4]  Zongpeng Li,et al.  Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters , 2017, IEEE Transactions on Cloud Computing.

[5]  Xin Wang,et al.  Cost-Efficient Scheduling of Bulk Transfers in Inter-Datacenter WANs , 2019, IEEE/ACM Transactions on Networking.

[6]  Zuqing Zhu,et al.  Data-Oriented Task Scheduling in Fixed- and Flexible-Grid Multilayer Inter-DC Optical Networks: A Comparison Study , 2017, Journal of Lightwave Technology.

[7]  Weiqiang Sun,et al.  Joint provisioning of lightpaths and storage in store-and-transfer wavelength-division multiplexing networks , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[8]  Weiqiang Sun,et al.  Comparison of Two Sharing Modes for a Proposed Optical Enterprise-Access SDN Architecture , 2018, 2018 28th International Telecommunication Networks and Applications Conference (ITNAC).

[9]  Weiqiang Sun,et al.  Bulk Data Transfer with Store-and-Forward in Wide Area Networks , 2019, 2019 21st International Conference on Transparent Optical Networks (ICTON).

[10]  Eiji Oki,et al.  Optical Switch in the Middle (OSM) architecture for DCNs with Hadoop adaptations , 2017, 2017 IEEE International Conference on Communications (ICC).

[11]  Pablo Rodriguez,et al.  Algorithms for Constrained Bulk-Transfer of Delay-Tolerant Data , 2010, 2010 IEEE International Conference on Communications.

[12]  Haiying Shen,et al.  Co-scheduler: Accelerating Data-Parallel Jobs in Datacenter Networks with Optical Circuit Switching , 2019, 2019 IEEE 39th International Conference on Distributed Computing Systems (ICDCS).

[13]  Xiao Lin,et al.  Design of an SNF Scheduling Method for Bulk Data Transfers over Inter-Datacenter WANs , 2019, 2019 IEEE 20th International Conference on High Performance Switching and Routing (HPSR).

[14]  George Iosifidis,et al.  Distributed Storage Control Algorithms for Dynamic Networks , 2017, IEEE/ACM Transactions on Networking.

[15]  R. Nejabati,et al.  Software-defined optical networks technology and infrastructure: Enabling software-defined optical network operations [invited] , 2013, IEEE/OSA Journal of Optical Communications and Networking.

[16]  Weiqiang Sun,et al.  Dimensioning of store-and-transfer WDM networks with stratified ROADM node , 2019, Opt. Switch. Netw..

[17]  Reza Nejabati,et al.  Software defined optical networks technology and infrastructure: Enabling software-defined optical network operations , 2013 .

[18]  José Luis García-Dorado,et al.  Cost-aware Multi Data-Center Bulk Transfers in the Cloud from a Customer-Side Perspective , 2019, IEEE Transactions on Cloud Computing.

[19]  Yojiro Mori,et al.  Design and Evaluation of Optical Circuit Switches for Intra-Datacenter Networking , 2019, Journal of Lightwave Technology.

[20]  Chankyun Lee,et al.  Efficient design and scalable control for store-and-forward capable optical transport networks , 2017, IEEE/OSA Journal of Optical Communications and Networking.