Performance study of an SnF scheduling method for bulk data transfers over inter-datacenter WANs

Abstract The rise of emerging services has brought a rapid growth in bulk data transfers across inter-datacenter (DC) wide area networks (WANs). To accommodate the demands of such services, DC storage is introduced into the data-plane path so that delay-tolerant bulk data can be temporarily stored and forwarded (SnF) when network is less busy. However, the use of storage transforms a conventional routing problem into a scheduling problem, where spatial routing and temporal scheduling must be jointly performed. Moreover, the complexity of the problem increases exponentially with the scale of the network as well as the number of storage sites. While prior SnF scheduling methods are effective for small networks or for static traffic, they are either too complex or insufficient for large networks with dynamic traffic. In this paper, we study the performance-complexity tradeoff in SnF scheduling. Studies show that desirable performance can be attained by considering a few alternate routes rather than dynamically routing over the entire network topology. Thus, our proposed SnF scheduling method aims to decouple the problem into its spatial and temporal components, and then solve these components separately. The proposed method hence has low complexity while still achieving high performance. Simulations demonstrate that when the traffic load is medium or higher, the conventional joint method may suffer from the detour issue. In this case, the proposed method has the potential to outperform the conventional joint method even with only three alternate routes considered in SnF scheduling.

[1]  George N. Rouskas,et al.  On the Design of Online Scheduling Algorithms for Advance Reservations and QoS in Grids , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[2]  Weiqiang Sun,et al.  Joint provisioning of lightpaths and storage in store-and-transfer wavelength-division multiplexing networks , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[3]  D. R. Fulkerson,et al.  Flows in Networks. , 1964 .

[4]  Srikanth Kandula,et al.  QuickCast: Fast and Efficient Inter-Datacenter Transfers Using Forwarding Tree Cohorts , 2018, IEEE INFOCOM 2018 - IEEE Conference on Computer Communications.

[5]  Xiao Lin,et al.  Time-shifted multilayer graph: A routing framework for bulk data transfer in optical circuit-switched networks with assistive storage , 2016, IEEE/OSA Journal of Optical Communications and Networking.

[6]  Jiang Liu,et al.  A Completion Time-Based Flow Scheduling for Inter-Data Center Traffic Optimization , 2018, IEEE Access.

[7]  David A. Maltz,et al.  Network traffic characteristics of data centers in the wild , 2010, IMC '10.

[8]  Chankyun Lee,et al.  Efficient design and scalable control for store-and-forward capable optical transport networks , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[9]  Wei Guo,et al.  Performance analysis of storage-based routing for circuit-switched networks , 2016, IEEE/OSA Journal of Optical Communications and Networking.

[10]  Tram Truong-Huu,et al.  Deadline-Aware Scheduling and Flexible Bandwidth Allocation for Big-Data Transfers , 2018, IEEE Access.

[11]  Nikolaos Laoutaris Bulk data transfers on the Internet or how to book some terabytes on red-eye bandwidth , 2010, 2010 IEEE Information Theory Workshop on Information Theory (ITW 2010, Cairo).

[12]  Xiaoyuan Yang,et al.  Inter-datacenter bulk transfers with netstitcher , 2011 .

[13]  George Iosifidis,et al.  The impact of storage capacity on end-to-end delay in time varying networks , 2011, 2011 Proceedings IEEE INFOCOM.

[14]  Baochun Li,et al.  Deadline-Aware Scheduling and Routing for Inter-Datacenter Multicast Transfers , 2018, 2018 IEEE International Conference on Cloud Engineering (IC2E).

[15]  Zongpeng Li,et al.  Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters , 2017, IEEE Transactions on Cloud Computing.

[16]  Bo Li,et al.  Postcard: Minimizing Costs on Inter-Datacenter Traffic with Store-and-Forward , 2012, 2012 32nd International Conference on Distributed Computing Systems Workshops.

[17]  Xiao Lin,et al.  Slotted store-and-forward optical circuit-switched networks: A performance study , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[18]  Cauligi S. Raghavendra,et al.  Datacenter Traffic Control: Understanding Techniques and Tradeoffs , 2017, IEEE Communications Surveys & Tutorials.

[19]  Yi Zhu,et al.  Routing and Scheduling for Time-Shift Advance Reservation , 2009, 2009 Proceedings of 18th International Conference on Computer Communications and Networks.

[20]  Hong-Fang Yu,et al.  Deadline-Guaranteed Point-to-Multipoint Bulk Transfers in Inter-Datacenter Networks , 2018, 2018 IEEE International Conference on Communications (ICC).

[21]  Xiaobo Zhou,et al.  TrafficShaper: Shaping Inter-Datacenter Traffic to Reduce the Transmission Cost , 2018, IEEE/ACM Transactions on Networking.

[22]  Peng Zhang,et al.  D4D: Inter-datacenter Bulk Transfers with ISP Friendliness , 2012, 2012 IEEE International Conference on Cluster Computing.

[23]  Xiao Lin,et al.  Design of an SNF Scheduling Method for Bulk Data Transfers over Inter-Datacenter WANs , 2019, 2019 IEEE 20th International Conference on High Performance Switching and Routing (HPSR).

[24]  Weiqiang Sun,et al.  Dimensioning of the store-and-transfer WDM network with limited node storage under the sliding scheduled traffic model , 2017, IEEE/OSA Journal of Optical Communications and Networking.

[25]  Xiaoying Zheng,et al.  Delay Tolerant Bulk Transfers on Inter-Datacenter Networks , 2016, 2016 IEEE Globecom Workshops (GC Wkshps).

[26]  George Iosifidis,et al.  Distributed Storage Control Algorithms for Dynamic Networks , 2017, IEEE/ACM Transactions on Networking.

[27]  Alex X. Liu,et al.  Multiple bulk data transfers scheduling among datacenters , 2014, Comput. Networks.

[28]  Imrich Chlamtac,et al.  Lightpath communications: an approach to high bandwidth optical WAN's , 1992, IEEE Trans. Commun..

[29]  Eli Dart,et al.  The Science DMZ: A network design pattern for data-intensive science , 2013, 2013 SC - International Conference for High Performance Computing, Networking, Storage and Analysis (SC).

[30]  Pablo Rodriguez,et al.  Delay-tolerant bulk data transfers on the internet , 2013, TNET.

[31]  Naoaki Yamanaka,et al.  A Dynamic Network Design for High-Speed Enterprise Access Links , 2014, 2015 IEEE Global Communications Conference (GLOBECOM).

[32]  Eiji Oki,et al.  Optical Switch in the Middle (OSM) architecture for DCNs with Hadoop adaptations , 2017, 2017 IEEE International Conference on Communications (ICC).

[33]  Pablo Rodriguez,et al.  Algorithms for Constrained Bulk-Transfer of Delay-Tolerant Data , 2010, 2010 IEEE International Conference on Communications.

[34]  Neal Charbonneau,et al.  A Survey of Advance Reservation Routing and Wavelength Assignment in Wavelength-Routed WDM Networks , 2012, IEEE Communications Surveys & Tutorials.

[35]  Zuqing Zhu,et al.  Data-Oriented Task Scheduling in Fixed- and Flexible-Grid Multilayer Inter-DC Optical Networks: A Comparison Study , 2017, Journal of Lightwave Technology.

[36]  Yi Zhu,et al.  Routing and horizon scheduling for time-shift advance reservation , 2009, 2009 Conference on Optical Fiber Communication - incudes post deadline papers.