Energy-efficient adaptive networked datacenters for the QoS support of real-time applications

In this paper, we develop the optimal minimum-energy scheduler for the adaptive joint allocation of the task sizes, computing rates, communication rates and communication powers in virtualized networked data centers (VNetDCs) that operate under hard per-job delay-constraints. The considered VNetDC platform works at the Middleware layer of the underlying protocol stack. It aims at supporting real-time stream service (such as, for example, the emerging big data stream computing (BDSC) services) by adopting the software-as-a-service (SaaS) computing model. Our objective is the minimization of the overall computing-plus-communication energy consumption. The main new contributions of the paper are the following ones: (i) the computing-plus-communication resources are jointly allotted in an adaptive fashion by accounting in real-time for both the (possibly, unpredictable) time fluctuations of the offered workload and the reconfiguration costs of the considered VNetDC platform; (ii) hard per-job delay-constraints on the overall allowed computing-plus-communication latencies are enforced; and, (iii) to deal with the inherently nonconvex nature of the resulting resource optimization problem, a novel solving approach is developed, that leads to the lossless decomposition of the afforded problem into the cascade of two simpler sub-problems. The sensitivity of the energy consumption of the proposed scheduler on the allowed processing latency, as well as the peak-to-mean ratio (PMR) and the correlation coefficient (i.e., the smoothness) of the offered workload is numerically tested under both synthetically generated and real-world workload traces. Finally, as an index of the attained energy efficiency, we compare the energy consumption of the proposed scheduler with the corresponding ones of some benchmark static, hybrid and sequential schedulers and numerically evaluate the resulting percent energy gaps.

[1]  Mor Harchol-Balter Performance Modeling and Design of Computer Systems: Preface , 2013 .

[2]  Karsten Schwan,et al.  VirtualPower: coordinated power management in virtualized enterprise systems , 2007, SOSP.

[3]  Athanasios V. Vasilakos,et al.  A General Framework for Achieving Energy Efficiency in Data Center Networks , 2013, ArXiv.

[4]  Jim Kurose,et al.  Computer Networking: A Top-Down Approach (6th Edition) , 2007 .

[5]  Leonardo Neumeyer,et al.  S4: Distributed Stream Computing Platform , 2010, 2010 IEEE International Conference on Data Mining Workshops.

[6]  Asser N. Tantawi,et al.  Analytic modeling of multitier Internet applications , 2007, TWEB.

[7]  Helen J. Wang,et al.  SecondNet: a data center network virtualization architecture with bandwidth guarantees , 2010, CoNEXT.

[8]  Sparsh Mittal,et al.  Power Management Techniques for Data Centers: A Survey , 2014, ArXiv.

[9]  Rodney S. Tucker,et al.  Green Cloud Computing: Balancing Energy in Processing, Storage, and Transport , 2011, Proceedings of the IEEE.

[10]  Yogesh L. Simmhan,et al.  PLAStiCC: Predictive Look-Ahead Scheduling for Continuous Dataflows on Clouds , 2014, 2014 14th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[11]  Athanasios V. Vasilakos,et al.  GreenDCN: A General Framework for Achieving Energy Efficiency in Data Center Networks , 2013, IEEE Journal on Selected Areas in Communications.

[12]  Minghua Chen,et al.  Simple and effective dynamic provisioning for power-proportional data centers , 2012, CISS.

[13]  Enzo Baccarelli,et al.  Energy-saving self-configuring networked data centers , 2013, Comput. Networks.

[14]  Amin Vahdat,et al.  A scalable, commodity data center network architecture , 2008, SIGCOMM '08.

[15]  Eyke Hüllermeier,et al.  Open challenges for data stream mining research , 2014, SKDD.

[16]  Kang G. Shin,et al.  Automated control of multiple virtualized resources , 2009, EuroSys '09.

[17]  Prashant J. Shenoy,et al.  Energy-aware load balancing in content delivery networks , 2011, 2012 Proceedings IEEE INFOCOM.

[18]  M. S. Bazaraa,et al.  Nonlinear Programming , 1979 .

[19]  Lin Wang,et al.  Incorporating Rate Adaptation Into Green Networking for Future Data Centers , 2013, 2013 IEEE 12th International Symposium on Network Computing and Applications.

[20]  Enzo Baccarelli,et al.  Optimized Power Allocation for Multiantenna Systems Impaired by Multiple Access Interference and Imperfect Channel Estimation , 2007, IEEE Transactions on Vehicular Technology.

[21]  Enzo Baccarelli,et al.  Recursive Kalman-type optimal estimation and detection of hidden Markov chains , 1996, Signal Process..

[22]  Enzo Baccarelli,et al.  Optimal Self-Adaptive QoS Resource Management in Interference-Affected Multicast Wireless Networks , 2013, IEEE/ACM Transactions on Networking.

[23]  Nagarajan Kandasamy,et al.  Power and performance management of virtualized computing environments via lookahead control , 2008, 2008 International Conference on Autonomic Computing.

[24]  Lachlan L. H. Andrew,et al.  Dynamic Right-Sizing for Power-Proportional Data Centers , 2011, IEEE/ACM Transactions on Networking.

[25]  Rami G. Melhem,et al.  Scheduling with Dynamic Voltage/Speed Adjustment Using Slack Reclamation in Multiprocessor Real-Time Systems , 2003, IEEE Trans. Parallel Distributed Syst..

[26]  Rayadurgam Srikant,et al.  The Mathematics of Internet Congestion Control (Systems and Control: Foundations and Applications) , 2004 .

[27]  Amar Phanishayee,et al.  Safe and effective fine-grained TCP retransmissions for datacenter communication , 2009, SIGCOMM '09.

[28]  Albert G. Greenberg,et al.  Data center TCP (DCTCP) , 2010, SIGCOMM '10.

[29]  Xue Liu,et al.  Challenges Towards Elastic Power Management in Internet Data Centers , 2009, 2009 29th IEEE International Conference on Distributed Computing Systems Workshops.

[30]  Ramin Yahyapour,et al.  Cloud computing networking: challenges and opportunities for innovations , 2013, IEEE Communications Magazine.

[31]  Akshat Verma,et al.  WattApp: an application aware power meter for shared data centers , 2010, ICAC '10.

[32]  Martin Hirzel,et al.  Tutorial: stream processing optimizations , 2013, DEBS.

[33]  Shahaboddin Shamshirband,et al.  Incremental proxy re-encryption scheme for mobile cloud computing environment , 2013, The Journal of Supercomputing.

[34]  Nagarajan Kandasamy,et al.  Power and Performance Management of Virtualized Computing Environments Via Lookahead Control , 2008, ICAC.

[35]  Liang Guo,et al.  A spectrum of TCP-friendly window-based congestion control algorithms , 2003, TNET.

[36]  Sharma Chakravarthy,et al.  Stream Data Processing: A Quality of Service Perspective - Modeling, Scheduling, Load Shedding, and Complex Event Processing , 2009, Advances in Database Systems.

[37]  Zhengping Qian,et al.  TimeStream: reliable stream computation in the cloud , 2013, EuroSys '13.

[38]  A. Robert Calderbank,et al.  Layering as Optimization Decomposition: A Mathematical Theory of Network Architectures , 2007, Proceedings of the IEEE.

[39]  Athanasios V. Vasilakos,et al.  GreenDCN: a General Framework for Achieving Network Energy Efficiency in Data Centers , 2014 .

[40]  Enzo Baccarelli,et al.  Stochastic traffic engineering for real-time applications over wireless networks , 2012, J. Netw. Comput. Appl..

[41]  Oliver Tamm,et al.  Eco-sustainable system and network architectures for future transport networks , 2010 .

[42]  Enzo Baccarelli,et al.  Optimal MIMO UWB-IR Transceiver for Nakagami-fading and Poisson-Arrivals , 2008, J. Commun..

[43]  Odej Kao,et al.  Exploiting Dynamic Resource Allocation for Efficient Parallel Data Processing in the Cloud , 2011, IEEE Transactions on Parallel and Distributed Systems.

[44]  Krishna M. Sivalingam,et al.  TCP improvements for data center networks , 2013, 2013 Fifth International Conference on Communication Systems and Networks (COMSNETS).

[45]  Peter J. Varman,et al.  mClock: Handling Throughput Variability for Hypervisor IO Scheduling , 2010, OSDI.

[46]  Shahaboddin Shamshirband,et al.  BSS: block-based sharing scheme for secure data storage services in mobile cloud environment , 2014, The Journal of Supercomputing.

[47]  Anand Sivasubramaniam,et al.  Statistical profiling-based techniques for effective power provisioning in data centers , 2009, EuroSys '09.

[48]  Scott Shenker,et al.  Discretized Streams: An Efficient and Fault-Tolerant Model for Stream Processing on Large Clusters , 2012, HotCloud.

[49]  Arjan Durresi,et al.  Cloud computing: networking and communication challenges , 2012, IEEE Commun. Mag..

[50]  H. Kushner,et al.  Analysis of adaptive step-size SA algorithms for parameter tracking , 1995, IEEE Trans. Autom. Control..

[51]  Peter A. Dinda,et al.  VNET/P: bridging the cloud and high performance computing through fast overlay networking , 2012, HPDC '12.

[52]  Rajkumar Buyya,et al.  Power-aware provisioning of Cloud resources for real-time services , 2009, MGC '09.

[53]  Tim Kraska,et al.  Stormy: an elastic and highly available streaming service in the cloud , 2012, EDBT-ICDT '12.

[54]  Keith W. Ross,et al.  Computer networking - a top-down approach featuring the internet , 2000 .

[55]  Alessandro Margara,et al.  Processing flows of information: From data stream to complex event processing , 2012, CSUR.

[56]  Eytan Modiano,et al.  Power allocation and routing in multibeam satellites with time-varying channels , 2003, TNET.

[57]  Din J. Wasem,et al.  Mining of Massive Datasets , 2014 .

[58]  Hai Jin,et al.  Carbon-Aware Load Balancing for Geo-distributed Cloud Services , 2013, 2013 IEEE 21st International Symposium on Modelling, Analysis and Simulation of Computer and Telecommunication Systems.

[59]  Derong Liu The Mathematics of Internet Congestion Control , 2005, IEEE Transactions on Automatic Control.