Task Allocation and Re-allocation for Big Data Applications in Cloud Computing Environments

Resource allocation for Big data streams in cloud systems involves selecting the appropriate cloud resources. Since incorrect resource allocation results in either under provisioning or over provisioning, accurate resource allocation becomes challenging in Big data applications. Hence, the objective of this work is to design an optimal solution for resource allocation for minimizing the network bandwidth and response delay. In this paper, a task allocation and re-allocation mechanism for Big data applications is designed. It consists of two important agents: RE-allocation Agent (REA) and Resource Agent (RA). The RA is responsible for mapping the user requirements to the available VMs. The REA monitors the resources and chooses the VMs for resource reconfiguration. Then, it dispatches an allocation or de-allocation request to RA, running in the physical system, based on the varying requirements of virtual machines. Experimental results show that the proposed TARA has less execution time and achieves better utilization of resources, when compared to existing tool.

[1]  Shahin Vakilinia Energy efficient temporal load aware resource allocation in cloud computing datacenters , 2017, Journal of Cloud Computing.

[2]  Meikang Qiu,et al.  Cloud Infrastructure Resource Allocation for Big Data Applications , 2018, IEEE Transactions on Big Data.

[3]  Shiyong Lu,et al.  Scheduling Big Data Workflows in the Cloud under Deadline Constraints , 2018, 2018 IEEE Fourth International Conference on Big Data Computing Service and Applications (BigDataService).

[4]  Sa'ed Abed,et al.  Enhancement of Task Scheduling Technique of Big Data Cloud Computing , 2018, 2018 International Conference on Advances in Big Data, Computing and Data Communication Systems (icABCD).

[5]  Tram Truong-Huu,et al.  Deadline-Aware Scheduling and Flexible Bandwidth Allocation for Big-Data Transfers , 2018, IEEE Access.

[6]  Sandeep K. Sood,et al.  Dynamic resource allocation for big data streams based on data characteristics (5Vs) , 2017, Int. J. Netw. Manag..

[7]  Min Chen,et al.  Job schedulers for Big data processing in Hadoop environment: testing real-life schedulers using benchmark programs , 2017, Digit. Commun. Networks.