A cluster-based decentralized job dispatching for the large-scale cloud

The remarkable development of cloud computing in the past few years, and its proven ability to handle web hosting workloads, is prompting researchers to investigate whether clouds are suitable to run large-scale computations. Cloud load balancing is one of the solution to provide reliable and scalable cloud services. Especially, load balancing for the multimedia streaming requires dynamic and real-time load balancing strategies. With this context, this paper aims to propose an Inter Cloud Manager (ICM) job dispatching algorithm for the large-scale cloud environment. ICM mainly performs two tasks: clustering (neighboring) and decision-making. For clustering, ICM uses Hello packets that observe and collect data from its neighbor nodes, and decision-making is based on both the measured execution time and network delay in forwarding the jobs and receiving the result of the execution. We then run experiments on a large-scale laboratory test-bed to evaluate the performance of ICM, and compare it with well-known decentralized algorithms such as Ant Colony, Workload and Client Aware Policy (WCAP), and the Honey-Bee Foraging Algorithm (HFA). Measurements focus in particular on the observed total average response time including network delay in congested environments. The experimental results show that for most cases, ICM is better at avoiding system saturation under the heavy load.

[1]  Mohsen Guizani,et al.  Cognitive radio based hierarchical communications infrastructure for smart grid , 2011, IEEE Network.

[2]  Kuo-Qin Yan,et al.  Towards a Load Balancing in a three-level cloud computing network , 2010, 2010 3rd International Conference on Computer Science and Information Technology.

[3]  Edmundo Roberto Mauro Madeira,et al.  Load Balancing for Internet Distributed Services Using Limited Redirection Rates , 2011, 2011 5th Latin-American Symposium on Dependable Computing.

[4]  Athanasios V. Vasilakos,et al.  Resource and Revenue Sharing with Coalition Formation of Cloud Providers: Game Theoretic Approach , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[5]  Athanasios V. Vasilakos,et al.  Managing Performance Overhead of Virtual Machines in Cloud Computing: A Survey, State of the Art, and Future Directions , 2014, Proceedings of the IEEE.

[6]  Andreas Thor,et al.  Load Balancing for MapReduce-based Entity Resolution , 2011, 2012 IEEE 28th International Conference on Data Engineering.

[7]  Utpal Biswas,et al.  A smart job scheduling system for cloud computing service providers and users: Modeling and simulation , 2012, 2012 1st International Conference on Recent Advances in Information Technology (RAIT).

[8]  J. Wenny Rahayu,et al.  Mobile cloud computing: A survey , 2013, Future Gener. Comput. Syst..

[9]  Xiaolei Dong,et al.  Security and privacy for storage and computation in cloud computing , 2014, Inf. Sci..

[10]  George Gross,et al.  Optimized FTR Portfolio Construction Based on the Identification of Congested Network Elements , 2013, IEEE Transactions on Power Systems.

[11]  Athanasios V. Vasilakos,et al.  Survey on routing in data centers: insights and future directions , 2011, IEEE Network.

[12]  Atul Mishra,et al.  A Comparative Study of Load Balancing Algorithms in Cloud Computing Environment , 2014, ArXiv.

[13]  Bruce R. Dawson,et al.  A cloud monitoring system for remote sites , 1998 .

[14]  L. D. Dhinesh Babu,et al.  Honey bee behavior inspired load balancing of tasks in cloud computing environments , 2013, Appl. Soft Comput..

[15]  Jin Shao,et al.  A Runtime Model Based Monitoring Approach for Cloud , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[16]  Erol Gelenbe,et al.  Energy-QoS Trade-Offs in Mobile Service Selection , 2013, Future Internet.

[17]  Erol Gelenbe,et al.  Energy packet networks: smart electricity storage to meet surges in demand , 2012, SimuTools.

[18]  Xuejie Zhang,et al.  A load balancing mechanism based on ant colony and complex network theory in open cloud computing federation , 2010, 2010 The 2nd International Conference on Industrial Mechatronics and Automation.

[19]  Athanasios V. Vasilakos,et al.  A General Framework for Achieving Energy Efficiency in Data Center Networks , 2013, ArXiv.

[20]  Athanasios V. Vasilakos,et al.  Thermal-Aware Scheduling of Batch Jobs in Geographically Distributed Data Centers , 2014, IEEE Transactions on Cloud Computing.

[21]  Athanasios V. Vasilakos,et al.  SecCloud: Bridging Secure Storage and Computation in Cloud , 2010, 2010 IEEE 30th International Conference on Distributed Computing Systems Workshops.

[22]  Athanasios V. Vasilakos,et al.  An Online Mechanism for Resource Allocation and Pricing in Clouds , 2016, IEEE Transactions on Computers.

[23]  Athanasios V. Vasilakos,et al.  Mobile Cloud Computing: A Survey, State of Art and Future Directions , 2013, Mobile Networks and Applications.

[24]  Athanasios V. Vasilakos,et al.  Energy-Efficient Flow Scheduling and Routing with Hard Deadlines in Data Center Networks , 2014, 2014 IEEE 34th International Conference on Distributed Computing Systems.

[25]  Marija Ilic,et al.  Distributed scheduling of demand resources in a congested network , 2014, 2014 IEEE PES General Meeting | Conference & Exposition.

[26]  Muhammad Arshad,et al.  An Efficient Algorithm for Load Balancing in Cloud Computing , 2014 .

[27]  Manohar Chandwani,et al.  Decentralized content aware load balancing algorithm for distributed computing environments , 2011, ICWET.

[28]  R. Krishnaveni,et al.  Toward Transcoding As A Service In A Multimedia Cloud Energy-Efficient Job Dispatching Algorithm , 2016 .

[29]  Erol Gelenbe,et al.  Trade-offs between energy and quality of service , 2012, 2012 Sustainable Internet and ICT for Sustainability (SustainIT).

[30]  Athanasios V. Vasilakos,et al.  GreenDCN: A General Framework for Achieving Energy Efficiency in Data Center Networks , 2013, IEEE Journal on Selected Areas in Communications.

[31]  Haibing Guan,et al.  A survey on data center networking for cloud computing , 2015, Comput. Networks.

[32]  Jameela Al-Jaroodi,et al.  DDFTP: Dual-Direction FTP , 2011, 2011 11th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[33]  Erol Gelenbe,et al.  Energy-Efficient Cloud Computing , 2010, Comput. J..

[34]  Erol Gelenbe,et al.  A Framework for Energy-Aware Routing in Packet Networks , 2011, Comput. J..

[35]  Song Gao,et al.  Travelers' Day-to-Day Route Choice Behavior with Real-Time Information in a Congested Risky Network , 2014 .

[36]  Erol Gelenbe,et al.  Choosing a Local or Remote Cloud , 2012, 2012 Second Symposium on Network Cloud Computing and Applications.

[37]  Nitin,et al.  Load Balancing of Nodes in Cloud Using Ant Colony Optimization , 2012, 2012 UKSim 14th International Conference on Computer Modelling and Simulation.