Design of an SNF Scheduling Method for Bulk Data Transfers over Inter-Datacenter WANs

Increasingly bulk data transfers impose an unprecedented challenge on inter-datacenter (DC) wide area networks (WANs). To overcome this challenge, DC storage is exploited and introduced into the data-plane path so that delay-tolerant bulk data can be temporarily stored and forwarded (SnF) at a later time when network is less congested. However, the use of storage introduces additional complexity into the transfer process. In this paper, we evaluate the computational complexity of the SNF scheduling problem. Studies show that desirable performance can be attained by considering only a few alternate routes rather than dynamically routing over the entire network topology. Inspired by this finding, we propose a time-space decoupled (TSD) SNF scheduling method. By decoupling the problem into its spatial and temporal components and solving these components separately, the TSD method obtains good performance while maintaining low complexity. Simulations show that the TSD method can outperform the conventional joint method when the traffic load is medium or higher.

[1]  George N. Rouskas,et al.  On the Design of Online Scheduling Algorithms for Advance Reservations and QoS in Grids , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[2]  Xiaoying Zheng,et al.  Delay Tolerant Bulk Transfers on Inter-Datacenter Networks , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[3]  Zuqing Zhu,et al.  Data-Oriented Task Scheduling in Fixed- and Flexible-Grid Multilayer Inter-DC Optical Networks: A Comparison Study , 2017, Journal of Lightwave Technology.

[4]  Yi Zhu,et al.  Routing and horizon scheduling for time-shift advance reservation , 2009, 2009 Conference on Optical Fiber Communication - incudes post deadline papers.

[5]  George Iosifidis,et al.  Distributed Storage Control Algorithms for Dynamic Networks , 2017, IEEE/ACM Transactions on Networking.

[6]  Zongpeng Li,et al.  Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters , 2017, IEEE Transactions on Cloud Computing.

[7]  Cauligi S. Raghavendra,et al.  Datacenter Traffic Control: Understanding Techniques and Tradeoffs , 2017, IEEE Communications Surveys & Tutorials.

[8]  Wei Guo,et al.  Performance analysis of storage-based routing for circuit-switched networks , 2016, IEEE/OSA Journal of Optical Communications and Networking.

[9]  Peng Zhang,et al.  D4D: Inter-datacenter Bulk Transfers with ISP Friendliness , 2012, 2012 IEEE International Conference on Cluster Computing.

[10]  Ilias Giechaskiel,et al.  Delay Tolerant Bulk Data Transfers on the Internet , 2014 .

[11]  Xiao Lin,et al.  Slotted store-and-forward optical circuit-switched networks: A performance study , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[12]  Alessandro Birolini Reliability Engineering: Theory and Practice , 1999 .

[13]  Xiao Lin,et al.  Time-shifted multilayer graph: A routing framework for bulk data transfer in optical circuit-switched networks with assistive storage , 2016, IEEE/OSA Journal of Optical Communications and Networking.

[14]  Jiang Liu,et al.  A Completion Time-Based Flow Scheduling for Inter-Data Center Traffic Optimization , 2018, IEEE Access.

[15]  Marco Tacca,et al.  Time-Shift Circuit Switching , 2008, OFC/NFOEC 2008 - 2008 Conference on Optical Fiber Communication/National Fiber Optic Engineers Conference.

[16]  Xiaobo Zhou,et al.  TrafficShaper: Shaping Inter-Datacenter Traffic to Reduce the Transmission Cost , 2018, IEEE/ACM Transactions on Networking.

[17]  Yi Zhu,et al.  Routing and Scheduling for Time-Shift Advance Reservation , 2009, 2009 Proceedings of 18th International Conference on Computer Communications and Networks.

[18]  Weiqiang Sun,et al.  Joint provisioning of lightpaths and storage in store-and-transfer wavelength-division multiplexing networks , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[19]  Srikanth Kandula,et al.  QuickCast: Fast and Efficient Inter-Datacenter Transfers Using Forwarding Tree Cohorts , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[20]  Tram Truong-Huu,et al.  Deadline-Aware Scheduling and Flexible Bandwidth Allocation for Big-Data Transfers , 2018, IEEE Access.

[21]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[22]  Chankyun Lee,et al.  Efficient design and scalable control for store-and-forward capable optical transport networks , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[23]  Michael Sirivianos,et al.  Inter-datacenter bulk transfers with netstitcher , 2011, SIGCOMM.

[24]  Alex X. Liu,et al.  Multiple bulk data transfers scheduling among datacenters , 2014, Comput. Networks.

[25]  Eiji Oki,et al.  Optical Switch in the Middle (OSM) architecture for DCNs with Hadoop adaptations , 2017, 2017 IEEE International Conference on Communications (ICC).

[26]  Baochun Li,et al.  Deadline-Aware Scheduling and Routing for Inter-Datacenter Multicast Transfers , 2018, 2018 IEEE International Conference on Cloud Engineering (IC2E).

[27]  Bo Li,et al.  Postcard: Minimizing Costs on Inter-Datacenter Traffic with Store-and-Forward , 2012, 2012 32nd International Conference on Distributed Computing Systems Workshops.