Resource allocation and scheduling in cloud computing

Cloud computing is a platform that hosts applications and services for businesses and users to accesses computing as a service. In this paper, we identify two scheduling and resource allocation problems in cloud computing. We describe Hadoop MapReduce and its schedulers, and present recent research efforts in this area including alternative schedulers and enhancements to existing schedulers. The second scheduling problem is the provisioning of virtual machines to resources in the cloud. We present a survey of the different approaches to solve this resource allocation problem. We also include recent research and standards for inter-connecting clouds and discuss the suitability of running scientific applications in the cloud.

[1]  Randy H. Katz,et al.  Topology-aware resource allocation for data-intensive workloads , 2011, Comput. Commun. Rev..

[2]  Hans De Sterck,et al.  Case Study of Scientific Data Processing on a Cloud Using Hadoop , 2009, HPCS.

[3]  Jie Li,et al.  Bridging the Gap between Desktop and the Cloud for eScience Applications , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[4]  Dario Vlah,et al.  Speculative pipelining for compute cloud programming , 2010, 2010 - MILCOM 2010 MILITARY COMMUNICATIONS CONFERENCE.

[5]  Yuan Yu,et al.  Dryad: distributed data-parallel programs from sequential building blocks , 2007, EuroSys '07.

[6]  Marc Frîncu.  Distributed Scheduling Policy in Service Oriented Environments , 2009, 2009 11th International Symposium on Symbolic and Numeric Algorithms for Scientific Computing.

[7]  Eero Vainikko,et al.  SciCloud: Scientific Computing on the Cloud , 2010, 2010 10th IEEE/ACM International Conference on Cluster, Cloud and Grid Computing.

[8]  Buqing Cao,et al.  A Service-Oriented Qos-Assured and Multi-Agent Cloud Computing Architecture , 2009, CloudCom.

[9]  Jamie Shiers,et al.  Grid today, clouds on the horizon , 2009, Comput. Phys. Commun..

[10]  Sanjay Ghemawat,et al.  MapReduce: Simplified Data Processing on Large Clusters , 2004, OSDI.

[11]  Thomas Sandholm,et al.  Dynamic Proportional Share Scheduling in Hadoop , 2010, JSSPP.

[12]  Xuejie Zhang,et al.  Realization of open cloud computing federation based on mobile agent , 2009, 2009 IEEE International Conference on Intelligent Computing and Intelligent Systems.

[13]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[14]  Jiang Ningkang,et al.  Distributed Scheduling Extension on Hadoop , 2009, CloudCom.

[15]  Ganesh Venkitachalam,et al.  The design of a practical system for fault-tolerant virtual machines , 2010, OPSR.

[16]  Daniel M. Batista,et al.  A Survey of Large Scale Data Management Approaches in Cloud Environments , 2011, IEEE Communications Surveys & Tutorials.

[17]  Barbara Panicucci,et al.  Autonomic Management of Cloud Service Centers with Availability Guarantees , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[18]  Roozbeh Farahbod,et al.  Dynamic Resource Allocation in Computing Clouds Using Distributed Multiple Criteria Decision Analysis , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[19]  Xindong Wu,et al.  K-Means Clustering with Bagging and MapReduce , 2011, 2011 44th Hawaii International Conference on System Sciences.

[20]  Fangzhe Chang,et al.  Optimal Resource Allocation in Clouds , 2010, 2010 IEEE 3rd International Conference on Cloud Computing.

[21]  Quan Chen,et al.  SAMR: A Self-adaptive MapReduce Scheduling Algorithm in Heterogeneous Environment , 2010, 2010 10th IEEE International Conference on Computer and Information Technology.

[22]  Won Kim,et al.  Cloud Computing: Today and Tomorrow , 2009, J. Object Technol..

[23]  Hai Jin,et al.  Tools and Technologies for Building Clouds , 2010, Cloud Computing.

[24]  Geoffrey C. Fox,et al.  Granules: A lightweight, streaming runtime for cloud computing with support, for Map-Reduce , 2009, 2009 IEEE International Conference on Cluster Computing and Workshops.

[25]  Irfan Ahmad,et al.  Modeling workloads and devices for IO load balancing in virtualized environments , 2010, PERV.

[26]  Arkady Kanevsky,et al.  Enabling a marketplace of clouds: VMware's vCloud director , 2010, OPSR.

[27]  Joanna Berlinska,et al.  Scheduling divisible MapReduce computations , 2011, J. Parallel Distributed Comput..

[28]  Yun Tian,et al.  Improving MapReduce performance through data placement in heterogeneous Hadoop clusters , 2010, 2010 IEEE International Symposium on Parallel & Distributed Processing, Workshops and Phd Forum (IPDPSW).

[29]  Michael R. Head,et al.  Virtual Hypervisor: Enabling fair and economical resource partitioning in cloud environments , 2010, 2010 IEEE Network Operations and Management Symposium - NOMS 2010.

[30]  Matei Zaharia,et al.  Job Scheduling for Multi-User MapReduce Clusters , 2009 .

[31]  Seoyoung Kim,et al.  Adaptable scheduling schemes for scientific applications on science cloud , 2010, 2010 IEEE International Conference On Cluster Computing Workshops and Posters (CLUSTER WORKSHOPS).

[32]  Jason Lawrence,et al.  Building and Using a Database of One Trillion Natural-Image Patches , 2011, IEEE Computer Graphics and Applications.

[33]  José A. B. Fortes,et al.  CloudBLAST: Combining MapReduce and Virtualization on Distributed Resources for Bioinformatics Applications , 2008, 2008 IEEE Fourth International Conference on eScience.

[34]  Beng-Hong Lim,et al.  Fast Transparent Migration for Virtual Machines , 2005, USENIX Annual Technical Conference, General Track.

[35]  Jehn-Ruey Jiang.  Nondominated local coteries for resource allocation in grids and clouds , 2011, Inf. Process. Lett..

[36]  Scott Shenker,et al.  Delay scheduling: a simple technique for achieving locality and fairness in cluster scheduling , 2010, EuroSys '10.

[37]  Andrew V. Goldberg,et al.  Quincy: fair scheduling for distributed computing clusters , 2009, SOSP '09.

[38]  Marios D. Dikaiakos,et al.  Cloud Computing: Distributed Internet Computing for IT and Scientific Research , 2009, IEEE Internet Computing.

[39]  Salvatore Venticinque,et al.  Cloud Agency: A Mobile Agent Based Cloud System , 2010, 2010 International Conference on Complex, Intelligent and Software Intensive Systems.

[40]  Majd F. Sakr,et al.  Initial Findings for Provisioning Variation in Cloud Computing , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[41]  Rajkumar Buyya,et al.  Article in Press Future Generation Computer Systems ( ) – Future Generation Computer Systems Cloud Computing and Emerging It Platforms: Vision, Hype, and Reality for Delivering Computing as the 5th Utility , 2022 .