Energy efficiency in big data complex systems: a comprehensive survey of modern energy saving techniques

The growing need of computation and processing has led to the generation of data centers. These data centers are usually comprised of hundreds of thousands of servers and other components. This complicated arrangement of the systems lead to the adoption of complex systems. Complex systems prevail in our society as combination of lots of entities, e.g., immune system, human brain and ecosystems. The adoption and interaction of the entities is possible through nonlinear interactions. The interaction between the components of the complex system is carried out in distributed fashion. Big data which is comprised of thousands of machines is also considered to be a form of complex adaptive systems which makes use of large entities, components and nonlinear interactions with each other. The development of such a complex systems raises certain challenges. Apart from management, energy is the most concerned one which is the core discussion of this research. This paper, surveys the state of the art on modern tools, techniques, architectures and algorithms which has been proposed and deployed to achieve energy efficiency in big data over the period of 2007–2015. We group existing approaches aimed at achieving energy efficiency in the complex paradigm of big data. In this categorization, we aim to provide an easy and concise view of the underlined model adapted by each approach in the context of big data.

[1]  Fred Douglis,et al.  Adaptive Disk Spin-Down Policies for Mobile Computers , 1995, Comput. Syst..

[2]  Mani B. Srivastava,et al.  Predictive system shutdown and other architectural techniques for energy efficient programmable computation , 1996, IEEE Trans. Very Large Scale Integr. Syst..

[3]  Allen C.-H. Wu,et al.  A predictive system shutdown method for energy saving of event-driven computation , 1997, 1997 Proceedings of IEEE International Conference on Computer Aided Design (ICCAD).

[4]  Seongsoo Lee,et al.  Run-time voltage hopping for low-power real-time systems , 2000, DAC.

[5]  Luca Benini,et al.  A survey of design techniques for system-level dynamic power management , 2000, IEEE Trans. Very Large Scale Integr. Syst..

[6]  Enrique V. Carrera,et al.  Load balancing and unbalancing for power and performance in cluster-based systems , 2001 .

[7]  Frank Bellosa,et al.  Process cruise control: event-driven clock scaling for dynamic power management , 2002, CASES '02.

[8]  Giorgio C. Buttazzo,et al.  Scalable Applications for Energy-Aware Processors , 2002, EMSOFT.

[9]  Dirk Grunwald,et al.  Massive Arrays of Idle Disks For Storage Archives , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[10]  Suresh Singh,et al.  Greening of the internet , 2003, SIGCOMM '03.

[11]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[12]  W. Krauth Statistical Mechanics: Algorithms and Computations , 2006 .

[13]  Karsten Schwan,et al.  VirtualPower: coordinated power management in virtualized enterprise systems , 2007, SOSP.

[14]  Luiz André Barroso,et al.  The Case for Energy-Proportional Computing , 2007, Computer.

[15]  Geoffrey C. Fox,et al.  MapReduce for Data Intensive Scientific Analyses , 2008, 2008 IEEE Fourth International Conference on eScience.

[16]  Thomas F. Wenisch,et al.  PowerNap: eliminating server idle power , 2009, ASPLOS.

[17]  Hai Jin,et al.  Evaluating MapReduce on Virtual Machines: The Hadoop Case , 2009, CloudCom.

[18]  John Cieslewicz,et al.  SQL/MapReduce: A practical approach to self-describing, polymorphic, and parallelizable user-defined functions , 2009, Proc. VLDB Endow..

[19]  Beng Chin Ooi,et al.  The performance of MapReduce , 2010, Proc. VLDB Endow..

[20]  Susanne Albers,et al.  Energy-efficient algorithms , 2010, Commun. ACM.

[21]  A. Wierman,et al.  Optimality, fairness, and robustness in speed scaling designs , 2010, SIGMETRICS '10.

[22]  Rini T. Kaushik,et al.  GreenHDFS: towards an energy-conserving, storage-efficient, hybrid Hadoop compute cluster , 2010 .

[23]  Archana Ganapathi,et al.  To compress or not to compress - compute vs. IO tradeoffs for mapreduce energy efficiency , 2010, Green Networking '10.

[24]  O. VanGeet Nrel,et al.  Best Practices Guide for Energy-Efficient Data Center Design , 2010 .

[25]  Erol Gelenbe,et al.  Energy-Efficient Cloud Computing , 2010, Comput. J..

[26]  Juan Li,et al.  An overview of energy efficiency techniques in cluster computing systems , 2013, Cluster Computing.

[27]  Thomas Hérault,et al.  DAGuE: A Generic Distributed DAG Engine for High Performance Computing , 2011, 2011 IEEE International Symposium on Parallel and Distributed Processing Workshops and Phd Forum.

[28]  Alexandra Carpen-Amarie,et al.  MapReduce Applications in the Cloud: A Cost Evaluation of Computation and Storage , 2012, Globe.

[29]  M. A. Niazi,et al.  An Intelligent Self-Organizing Power-Saving Architecture: An Agent-Based Approach , 2012, 2012 Fourth International Conference on Computational Intelligence, Modelling and Simulation.

[30]  Jinoh Kim,et al.  FREP: Energy proportionality for disk storage using replication , 2012, J. Parallel Distributed Comput..

[31]  Rajkumar Buyya,et al.  Optimal online deterministic algorithms and adaptive heuristics for energy and performance efficient dynamic consolidation of virtual machines in Cloud data centers , 2012, Concurr. Comput. Pract. Exp..

[32]  Jordi Torres,et al.  GreenHadoop: leveraging green energy in data-processing frameworks , 2012, EuroSys '12.

[33]  Yanpei Chen,et al.  Energy efficiency for large-scale MapReduce workloads with significant interactive analysis , 2012, EuroSys '12.

[34]  Aravind Menon,et al.  Big data @ facebook , 2012 .

[35]  Junaid Shuja,et al.  Energy-efficient data centers , 2012, Computing.

[36]  Jing Li,et al.  Energy Efficient Cloud Storage Service: Key Issues and Challenges , 2013, 2013 Fourth International Conference on Emerging Intelligent Data and Web Technologies.

[37]  Muaz A. Niazi,et al.  Cloud identity management security issues & solutions: a taxonomy , 2014, Complex Adapt. Syst. Model..

[38]  Rakesh Kumar,et al.  Open Source Solution for Cloud Computing Platform Using OpenStack , 2014 .

[39]  Yunhao Liu,et al.  Big Data: A Survey , 2014, Mob. Networks Appl..

[40]  Muaz A. Niazi,et al.  Self-organized power consumption approximation in the Internet of Things , 2015, 2015 IEEE International Conference on Consumer Electronics (ICCE).