Dynamically hierarchical resource-allocation algorithm in cloud computing environment

The development of big data challenges the computing power and communication capability of cloud architecture, but traditional resource-allocation algorithms perform poorly due to the large-scale communication among cloud nodes. In this paper, a dynamically hierarchical, resource-allocation algorithm is proposed for multiple cloud nodes collaborating in big data environment. Using fuzzy pattern recognition, the algorithm dynamically divides tasks and nodes into different levels based on computing power and storage factors. Thus a dynamically adjusted mapping is generated between tasks and nodes. When a new task arrives, only the nodes corresponding to the task level join in the bid. The algorithm takes advantages of dynamical hierarchy to reduce the communication traffic during resource allocation. Both theoretical and experimental results illustrate that the proposed algorithm outperforms the MinMin algorithm in terms of communication traffic and makespan.

[1]  Rajkumar Buyya,et al.  Energy-aware resource allocation heuristics for efficient management of data centers for Cloud computing , 2012, Future Gener. Comput. Syst..

[2]  Salim Hariri,et al.  Performance-Effective and Low-Complexity Task Scheduling for Heterogeneous Computing , 2002, IEEE Trans. Parallel Distributed Syst..

[3]  M. Shamim Hossain,et al.  QoS-aware Resource Provisioning for Big Data Processing in Cloud Computing Environment , 2014, 2014 International Conference on Computational Science and Computational Intelligence.

[4]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .

[5]  Bing-yuan Cao,et al.  Using Two-Level Fuzzy Pattern Recognition in the Classification of Convex Quadrilateral , 2010 .

[6]  Chia-Ming Wu,et al.  A green energy-efficient scheduling algorithm using the DVFS technique for cloud datacenters , 2014, Future Gener. Comput. Syst..

[7]  Rajkumar Buyya,et al.  Deadline-driven provisioning of resources for scientific applications in hybrid clouds with Aneka , 2012, Future Gener. Comput. Syst..

[8]  Seref Sagiroglu,et al.  Big data: A review , 2013, 2013 International Conference on Collaboration Technologies and Systems (CTS).

[9]  Hamid Beigy,et al.  A new fuzzy negotiation protocol for grid resource allocation , 2014, J. Netw. Comput. Appl..

[10]  Nathan R. Sturtevant,et al.  Optimized algorithms for multi-agent routing , 2008, AAMAS.

[11]  Jiong Yu,et al.  A workflow task scheduling algorithm based on the resources' fuzzy clustering in cloud computing environment , 2015, Int. J. Commun. Syst..

[12]  Naixue Xiong,et al.  A game-theoretic method of fair resource allocation for cloud computing services , 2010, The Journal of Supercomputing.

[13]  Ting Fang,et al.  Task Scheduling Model Based on Multi-Agent and Multi-Objective Dynamical Scheduling Algorithm , 2014, J. Networks.

[14]  Kenli Li,et al.  Proactive workload management in dynamic virtualized environments , 2014, J. Comput. Syst. Sci..

[15]  Sanjay Chaudhary,et al.  Policy based resource allocation in IaaS cloud , 2012, Future Gener. Comput. Syst..

[16]  Sahil R. Kalra,et al.  Big Challenges? Big Data … , 2015 .

[17]  V. Marx.  Biology: The big challenges of big data , 2013, Nature.

[18]  Zhen Xiao,et al.  Dynamic Resource Allocation Using Virtual Machines for Cloud Computing Environment , 2013, IEEE Transactions on Parallel and Distributed Systems.

[19]  Randy H. Katz,et al.  A view of cloud computing , 2010, CACM.

[20]  Young-Sik Jeong,et al.  Two‐phase grouping‐based resource management for big data processing in mobile cloud computing , 2014, Int. J. Commun. Syst..

[21]  Samuel Madden,et al.  From Databases to Big Data , 2012, IEEE Internet Comput..

[22]  Odej Kao,et al.  Exploiting Dynamic Resource Allocation for Efficient Parallel Data Processing in the Cloud , 2011, IEEE Transactions on Parallel and Distributed Systems.

[23]  R. Grekovs.  METHODS OF FUZZY PATTERN RECOGNITION , 2002 .

[24]  Rajkumar Buyya,et al.  CloudSim: a toolkit for modeling and simulation of cloud computing environments and evaluation of resource provisioning algorithms , 2011, Softw. Pract. Exp..

[25]  Witold Pedrycz,et al.  Fuzzy sets in pattern recognition: Methodology and methods , 1990, Pattern Recognit..

[26]  Min Wang,et al.  Safety Evaluation of Urban Flood Control System Based on Variable Fuzzy Pattern Recognition , 2010 .

[27]  Jelena Mirkovic,et al.  Optimal application allocation on multiple public clouds , 2014, Comput. Networks.

[28]  James C. Bezdek,et al.  Pattern Recognition with Fuzzy Objective Function Algorithms , 1981, Advanced Applications in Pattern Recognition.

[29]  Sung Chan Jun,et al.  An SLA-based cloud computing that facilitates resource allocation in the distributed data centers of a cloud provider , 2012, The Journal of Supercomputing.

[30]  Aman Kansal,et al.  Q-clouds: managing performance interference effects for QoS-aware clouds , 2010, EuroSys '10.

[31]  Fatma A. Omara,et al.  Dynamic task scheduling using fuzzy logic in distributed memory systems , 2010, 2010 The 7th International Conference on Informatics and Systems (INFOS).

[32]  Jing Zhang,et al.  The placement method of resources and applications based on request prediction in cloud data center , 2014, Inf. Sci..

[33]  Albert Y. Zomaya,et al.  Energy efficient utilization of resources in cloud computing systems , 2010, The Journal of Supercomputing.

[34]  Liu Xian-hui,et al.  Common key technology system of cloud manufacturing service platform for small and medium enterprises , 2011 .