Nefeli: Hint-Based Execution of Workloads in Clouds

Virtualization of computer systems has made feasible the provision of entire distributed infrastructures in the form of services. Such services do not expose the internal operational and physical characteristics of the underlying machinery to either users or applications. In this way, infrastructures including computers in data-centers, clusters of workstations, and networks of machines are shrouded in "clouds". Mainly through the deployment of virtual machines, such networks of computing nodes become cloud-computing environments. In this paper, we propose Nefeli, a virtual infrastructure gateway that is capable of effectively handling diverse workloads of jobs in cloud environments. By and large, users and their workloads remain agnostic to the internal features of clouds at all times. Exploiting execution patterns as well as logistical constraints, users provide Nefeli with hints for the handling of their jobs. Hints provide no hard requirements for application deployment in terms of pairing virtual-machines to specific physical cloud elements. Nefeli helps avoid bottlenecks within the cloud through the realization of viable virtual machine deployment mappings. As the types of jobs change over time, deployment mappings must follow suit. To this end, Nefeli offers mechanisms to migrate virtual machines as needed to adapt to changing performance needs. Using our prototype system, we show significant improvements in overall time needed and energy consumed for the execution of workloads in both simulated and real cloud computing environments.

[1]  Yanif Ahmad,et al.  Networked Query Processing for Distributed Stream-Based Applications , 2004, VLDB.

[2]  RosenblumMendel,et al.  Virtual Machine Monitors , 2005 .

[3]  Borja Sotomayor,et al.  Combining batch execution and leasing using virtual machines , 2008, HPDC '08.

[4]  Norman W. Paton,et al.  Adaptive Workflow Processing and Execution in Pegasus , 2008, 2008 The 3rd International Conference on Grid and Pervasive Computing - Workshops.

[5]  Mei-Hui Su,et al.  Characterization of scientific workflows , 2008, 2008 Third Workshop on Workflows in Support of Large-Scale Science.

[6]  M. Tim Jones.  Cloud Computing with Linux , 2012 .

[7]  Giuseppe Valetto,et al.  Elicitation and utilization of application-level utility functions , 2009, ICAC '09.

[8]  Margo I. Seltzer,et al.  Network-Aware Operator Placement for Stream-Processing Systems , 2006, 22nd International Conference on Data Engineering (ICDE'06).

[9]  Norman W. Paton,et al.  Adaptive workflow processing and execution in Pegasus , 2009 .

[10]  Ying Xing,et al.  Scalable Distributed Stream Processing , 2003, CIDR.

[11]  David Josephsen,et al.  Building a Monitoring Infrastructure with Nagios , 2007 .

[12]  Wil M. P. van der Aalst,et al.  Workflow Patterns , 2003, Distributed and Parallel Databases.

[13]  Tal Garfinkel,et al.  Virtual machine monitors: current technology and future trends , 2005, Computer.

[14]  Jean-Marc Menaud,et al.  Autonomic virtual resource management for service hosting platforms , 2009, 2009 ICSE Workshop on Software Engineering Challenges of Cloud Computing.

[15]  Richard Wolski,et al.  The Eucalyptus Open-Source Cloud-Computing System , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[16]  Minglu Li,et al.  Automatic Performance Tuning for the Virtualized Cluster System , 2009, 2009 29th IEEE International Conference on Distributed Computing Systems.

[17]  Andrew Warfield,et al.  Xen and the art of virtualization , 2003, SOSP '03.

[18]  C. D. Gelatt,et al.  Optimization by Simulated Annealing , 1983, Science.

[19]  Rajarshi Das,et al.  Utility functions in autonomic systems , 2004 .

[20]  Rajarshi Das,et al.  Utility functions in autonomic systems , 2004, International Conference on Autonomic Computing, 2004. Proceedings..

[21]  Norman W. Paton,et al.  Utility Driven Adaptive Work?ow Execution , 2009, 2009 9th IEEE/ACM International Symposium on Cluster Computing and the Grid.

[22]  Jeffrey O. Kephart,et al.  The Vision of Autonomic Computing , 2003, Computer.

[23]  Rajarshi Das,et al.  Achieving Self-Management via Utility Functions , 2007, IEEE Internet Computing.

[24]  Daniel S. Katz,et al.  Web-based Tools -- Montage: An astronomical image mosaic engine , 2007 .

[25]  Gang Wang,et al.  Appliance-Based Autonomic Provisioning Framework for Virtualized Outsourcing Data Center , 2007, Fourth International Conference on Autonomic Computing (ICAC'07).