Energy-efficient algorithms for distributed storage system based on block storage structure reconfiguration

As the underlying core infrastructure for cloud computing, distributed storage systems like the Hadoop Distributed File System (HDFS) are the foundation of all kinds of cloud services. However, designers of the ever-expanding systems have ignored the problem of high energy consumption, causing serious environmental and economic problems. The data availability and performance Quality of Service (QoS) requirements make it hard to use existing energy-saving technologies to solve the problem. After researching the data block?s storage structure and mechanism, and the relationship between the server?s status and the data block?s availability, the method to solve the problem of ensuring data availability and performance QoS requirements is proposed. The energy-saving model for the distributed storage system is defined. The algorithm divides the RACK into two distinct storage areas, Active-Zone and Sleep-Zone, reconfiguring the data storage structure using the block storage structure reconfiguration algorithm. To save energy, we turn the servers in Sleep-Zone to sleep mode while the workload is low. Numerical analysis and experimental results demonstrate that the energy-efficient algorithms improved the energy efficiency for the distributed storage system.

[1]  Richard E. Brown,et al.  Report to Congress on Server and Data Center Energy Efficiency: Public Law 109-431 , 2008 .

[2]  Archana Ganapathi,et al.  Statistical Workloads for Energy Efficient MapReduce , 2010 .

[3]  Henri Casanova,et al.  Resource Allocation Using Virtual Clusters , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[4]  Feng Zhao,et al.  Energy aware consolidation for cloud computing , 2008, CLUSTER 2008.

[5]  Rajkumar Buyya,et al.  Energy-Efficient Management of Data Center Resources for Cloud Computing: A Vision, Architectural Elements, and Open Challenges , 2010, PDPTA.

[6]  Wolf-Dietrich Weber,et al.  Power provisioning for a warehouse-sized computer , 2007, ISCA '07.

[7]  San Luis Obispo.  REDUCING CLUSTER POWER CONSUMPTION BY DYNAMICALLY SUSPENDING IDLE NODES , 2010 .

[8]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[9]  Mahadev Satyanarayanan,et al.  Managing battery lifetime with energy-aware adaptation , 2004, TOCS.

[10]  A. Wierman,et al.  Optimality, fairness, and robustness in speed scaling designs , 2010, SIGMETRICS '10.

[11]  Jerome A. Rolia,et al.  Resource pool management: Reactive versus proactive or let's be friends , 2009, Comput. Networks.

[12]  Yanpei Chen,et al.  Towards Energy Efficient MapReduce , 2009 .

[13]  Darrell D. E. Long,et al.  A Spin-Up Saved Is Energy Earned: Achieving Power-Efficient, Erasure-Coded Storage , 2008, HotDep.

[14]  Hui Wang,et al.  Multi-Tiered On-Demand Resource Scheduling for VM-Based Data Center , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[15]  Akshat Verma,et al.  pMapper: Power and Migration Cost Aware Application Placement in Virtualized Systems , 2008, Middleware.

[16]  Mor Harchol-Balter,et al.  Optimal power allocation in server farms , 2009, SIGMETRICS '09.

[17]  Erik D. Demaine,et al.  Energy-Efficient Algorithms , 2016, ITCS.

[18]  Vanish Talwar,et al.  No "power" struggles: coordinated multi-level power management for the data center , 2008, ASPLOS.

[19]  Hyeonsang Eom,et al.  Towards Energy Proportional Cloud for Data Processing Frameworks , 2010, SustainIT.

[20]  Lachlan L. H. Andrew,et al.  Power-Aware Speed Scaling in Processor Sharing Systems , 2009, IEEE INFOCOM 2009.

[21]  Rajkumar Buyya,et al.  Environment-conscious scheduling of HPC applications on distributed Cloud-oriented data centers , 2011, J. Parallel Distributed Comput..

[22]  Yiming Yang,et al.  Improving Energy Efficiency and Security for Disk Systems , 2010, 2010 IEEE 12th International Conference on High Performance Computing and Communications (HPCC).

[23]  Ethan L. Miller,et al.  Pergamum: Replacing Tape with Energy Efficient, Reliable, Disk-Based Archival Storage , 2008, FAST.

[24]  Rong Ge,et al.  Improving MapReduce energy efficiency for computation intensive workloads , 2011, 2011 International Green Computing Conference and Workshops.

[25]  Gargi Dasgupta,et al.  Workload management for power efficiency in virtualized data centers , 2011, CACM.

[26]  Allen C.-H. Wu,et al.  A predictive system shutdown method for energy saving of event-driven computation , 1997, 1997 Proceedings of IEEE International Conference on Computer Aided Design (ICCAD).

[27]  Vincent Salzgeber,et al.  Making cluster applications energy-aware , 2009, ACDC '09.

[28]  Thomas F. Wenisch,et al.  PowerNap: eliminating server idle power , 2009, ASPLOS.

[29]  Hai Jin,et al.  Towards a green cluster through dynamic remapping of virtual machines , 2012, Future Gener. Comput. Syst..

[30]  Mani B. Srivastava,et al.  Predictive system shutdown and other architectural techniques for energy efficient programmable computation , 1996, IEEE Trans. Very Large Scale Integr. Syst..

[31]  Douglas L. Jones,et al.  GRACE-2: integrating fine-grained application adaptation with global adaptation for saving energy , 2009, Int. J. Embed. Syst..

[32]  GhemawatSanjay,et al.  The Google file system , 2003 .

[33]  Alan Jay Smith,et al.  Improving dynamic voltage scaling algorithms with PACE , 2001, SIGMETRICS '01.

[34]  V. Kavitha,et al.  A survey on security issues in service delivery models of cloud computing , 2011, J. Netw. Comput. Appl..

[35]  Xiaoyun Zhu,et al.  PARTIC: Power-Aware Response Time Control for Virtualized Web Servers , 2011, IEEE Transactions on Parallel and Distributed Systems.

[36]  Austin Donnelly,et al.  Sierra: a power-proportional, distributed storage system , 2009 .

[37]  Amar Phanishayee,et al.  FAWNdamentally Power-efficient Clusters , 2009, HotOS.

[38]  Antony I. T. Rowstron,et al.  Write off-loading: Practical power management for enterprise storage , 2008, TOS.

[39]  P. Mell,et al.  The NIST Definition of Cloud Computing , 2011 .

[40]  Luiz André Barroso,et al.  The Case for Energy-Proportional Computing , 2007, Computer.

[41]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[42]  Derek McAuley,et al.  Energy is just another resource: energy accounting and energy pricing in the Nemesis OS , 2001, Proceedings Eighth Workshop on Hot Topics in Operating Systems.

[43]  Christoforos E. Kozyrakis,et al.  On the energy (in)efficiency of Hadoop clusters , 2010, OPSR.

[44]  Jin-Soo Kim,et al.  Energy Reduction in Consolidated Servers through Memory-Aware Virtual Machine Scheduling , 2011, IEEE Transactions on Computers.

[45]  Aameek Singh,et al.  Shares and utilities based power consolidation in virtualized server environments , 2009, 2009 IFIP/IEEE International Symposium on Integrated Network Management.

[46]  Vasudeva Varma,et al.  Dynamic energy efficient data placement and cluster reconfiguration algorithm for MapReduce framework , 2012, Future Gener. Comput. Syst..

[47]  Haruo Yokota,et al.  An Evaluation of Power-Proportional Data Placement for Hadoop Distributed File Systems , 2011, 2011 IEEE Ninth International Conference on Dependable, Autonomic and Secure Computing.

[48]  Rajkumar Buyya,et al.  Power-aware provisioning of Cloud resources for real-time services , 2009, MGC '09.

[49]  Carla Schlatter Ellis,et al.  Experiences in managing energy with ECOSystem , 2005, IEEE Pervasive Computing.

[50]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[51]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[52]  Bruno Schulze,et al.  Proceedings of the 7th International Workshop on Middleware for Grids, Clouds and e-Science , 2009, Middleware 2009.

[53]  Rini T. Kaushik,et al.  GreenHDFS: towards an energy-conserving, storage-efficient, hybrid Hadoop compute cluster , 2010 .

[54]  Trevor N. Mudge,et al.  Understanding and Designing New Server Architectures for Emerging Warehouse-Computing Environments , 2008, 2008 International Symposium on Computer Architecture.

[55]  Nagarajan Kandasamy,et al.  Power and performance management of virtualized computing environments via lookahead control , 2008, 2008 International Conference on Autonomic Computing.

[56]  Luca Benini,et al.  A survey of design techniques for system-level dynamic power management , 2000, IEEE Trans. Very Large Scale Integr. Syst..

[57]  Dalit Naor,et al.  Low power mode in cloud storage systems , 2009, 2009 IEEE International Symposium on Parallel & Distributed Processing.

[58]  Lachlan L. H. Andrew,et al.  Power-aware speed scaling in processor sharing systems: Optimality and robustness , 2012, Perform. Evaluation.

[59]  Alexander S. Szalay,et al.  Low-power amdahl-balanced blades for data intensive computing , 2010, OPSR.