Profit-Aware Distributed Online Scheduling for Data-Oriented Tasks in Cloud Datacenters

As there is an increasing trend to deploy geographically distributed (geo-distributed) cloud datacenters (DCs), the scheduling of data-oriented tasks in such cloud DC systems becomes an appealing research topic. Specifically, it is challenging to achieve the distributed online scheduling that can handle the tasks’ acceptance, data-transfers, and processing jointly and efficiently. In this paper, by considering the store-and-forward and anycast schemes, we formulate an optimization problem to maximize the time-average profit from serving data-oriented tasks in a cloud DC system and then leverage the Lyapunov optimization techniques to propose an efficient scheduling algorithm, i.e., GlobalAny. We also extend the proposed algorithm by designing a data-transfer acceleration scheme to reduce the data-transfer latency. Extensive simulations verify that our algorithms can maximize the time-average profit in a distributed online manner. The results also indicate that GlobalAny and GlobalAny_Ext (i.e., GlobalAny with data-transfer acceleration) outperform several existing algorithms in terms of both time-average profit and computation time.

[1]  Raouf Boutaba,et al.  ViNEYard: Virtual Network Embedding Algorithms With Coordinated Node and Link Mapping , 2012, IEEE/ACM Transactions on Networking.

[2]  Wei Lu,et al.  Joint Spectrum and IT Resource Allocation for Efficient VNF Service Chaining in Inter-Datacenter Elastic Optical Networks , 2016, IEEE Communications Letters.

[3]  Zuqing Zhu,et al.  Distributed Online Hybrid Cloud Management for Profit-Driven Multimedia Cloud Computing , 2015, IEEE Transactions on Multimedia.

[4]  P. Sadayappan,et al.  Modeling and Optimizing Large-Scale Wide-Area Data Transfers , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[5]  Zuqing Zhu,et al.  Cost-Efficient Virtual Network Function Graph (vNFG) Provisioning in Multidomain Elastic Optical Networks , 2017, Journal of Lightwave Technology.

[6]  Wei Lu,et al.  Dynamic Service Provisioning of Advance Reservation Requests in Elastic Optical Networks , 2013, Journal of Lightwave Technology.

[7]  Bo Li,et al.  On arbitrating the power-performance tradeoff in SaaS clouds , 2013, 2013 Proceedings IEEE INFOCOM.

[8]  Zuqing Zhu,et al.  Virtual Optical Network Embedding (VONE) Over Elastic Optical Networks , 2014, Journal of Lightwave Technology.

[9]  Wei Lu,et al.  Fragmentation-aware service provisioning for advance reservation multicast in SD-EONs. , 2015, Optics express.

[10]  Anand Sivasubramaniam,et al.  Optimal power cost management using stored energy in data centers , 2011, PERV.

[11]  S. J. B. Yoo,et al.  Experimental demonstration of heterogeneous cross stratum broker for scientific applications , 2016, 2016 Optical Fiber Communications Conference and Exhibition (OFC).

[12]  Zuqing Zhu,et al.  Availability-aware survivable virtual network embedding in optical datacenter networks , 2015, IEEE/OSA Journal of Optical Communications and Networking.

[13]  S. Martello,et al.  Algorithms for Knapsack Problems , 1987 .

[14]  Zongpeng Li,et al.  Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters , 2017, IEEE Transactions on Cloud Computing.

[15]  Srikanth Kandula,et al.  Achieving high utilization with software-driven WAN , 2013, SIGCOMM.

[16]  Biswanath Mukherjee,et al.  On Hybrid IR and AR Service Provisioning in Elastic Optical Networks , 2015, Journal of Lightwave Technology.

[17]  Jingjing Yao,et al.  Highly efficient data migration and backup for big data applications in elastic optical inter-data-center networks , 2015, IEEE Network.

[18]  Xin Wang,et al.  Towards Operational Cost Minimization in Hybrid Clouds for Dynamic Resource Provisioning with Delay-Aware Optimization , 2015, IEEE Transactions on Services Computing.

[19]  Yonggang Wen,et al.  Toward profit-seeking virtual network embedding algorithm via global resource capacity , 2014, IEEE INFOCOM 2014 - IEEE Conference on Computer Communications.

[20]  Qing Ling,et al.  Maximizing Utility of Time-Constrained Emergency Backup in Inter-Datacenter Networks , 2016, IEEE Communications Letters.

[21]  Biswanath Mukherjee,et al.  Optimizing deadline-driven bulk-data transfer to revitalize spectrum fragments in EONs [Invited] , 2015, IEEE/OSA Journal of Optical Communications and Networking.

[22]  Zuqing Zhu,et al.  Distributed Online Scheduling and Routing of Multicast-Oriented Tasks for Profit-Driven Cloud Computing , 2016, IEEE Communications Letters.

[23]  Hai Jin,et al.  Building a network highway for big data: architecture and challenges , 2014, IEEE Network.

[24]  Leandros Tassiulas,et al.  Resource Allocation and Cross-Layer Control in Wireless Networks , 2006, Found. Trends Netw..

[25]  Ulas C. Kozat,et al.  Dynamic resource allocation and power management in virtualized data centers , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[26]  Min Zhu,et al.  B4: experience with a globally-deployed software defined wan , 2013, SIGCOMM.

[27]  Junjie Liu,et al.  On Dynamic Service Function Chain Deployment and Readjustment , 2017, IEEE Transactions on Network and Service Management.

[28]  Lei Ying,et al.  On Combining Shortest-Path and Back-Pressure Routing Over Multihop Wireless Networks , 2009, IEEE INFOCOM 2009.

[29]  Michael Sirivianos,et al.  Inter-datacenter bulk transfers with netstitcher , 2011, SIGCOMM.

[30]  Zuqing Zhu,et al.  Data-Oriented Task Scheduling in Fixed- and Flexible-Grid Multilayer Inter-DC Optical Networks: A Comparison Study , 2017, Journal of Lightwave Technology.

[31]  Zuqing Zhu,et al.  On Fast and Coordinated Data Backup in Geo-Distributed Optical Inter-Datacenter Networks , 2015, Journal of Lightwave Technology.

[32]  Alex X. Liu,et al.  Multiple bulk data transfers scheduling among datacenters , 2014, Comput. Networks.

[33]  Massoud Pedram,et al.  Force-directed geographical load balancing and scheduling for batch jobs in distributed datacenters , 2013, 2013 IEEE International Conference on Cluster Computing (CLUSTER).

[34]  Wenjian Fang,et al.  Orchestrating Tree-Type VNF Forwarding Graphs in Inter-DC Elastic Optical Networks , 2016, Journal of Lightwave Technology.

[35]  Qing Ling,et al.  Evacuate Before Too Late: Distributed Backup in Inter-DC Networks with Progressive Disasters , 2018, IEEE Transactions on Parallel and Distributed Systems.

[36]  Wei Lu,et al.  Malleable Reservation Based Bulk-Data Transfer to Recycle Spectrum Fragments in Elastic Optical Networks , 2015, Journal of Lightwave Technology.

[37]  David Pisinger,et al.  Algorithms for Knapsack Problems , 1995 .

[38]  Zhong Pan,et al.  Jitter and Amplitude Noise Accumulations in Cascaded All-Optical Regenerators , 2008, Journal of Lightwave Technology.

[39]  Alexander L. Stolyar,et al.  Novel Architectures and Algorithms for Delay Reduction in Back-Pressure Scheduling and Routing , 2009, IEEE INFOCOM 2009.