Composable cost estimation and monitoring for computational applications in cloud computing environments

Abstract With the offer from cloud computing providers, scientists have the opportunity to utilize pay-as-you-go resources together with their own and shared resources. However, scientists need to decide which parts of their applications should be executed in cloud computing systems in order to balance the trade-off between cost, time and resource requirements. In this paper, we present a service for estimating, monitoring and analyzing costs associated with scientific applications in the cloud. Cost models associated with different application execution models are proposed and these cost models can be composed to determine costs of different scenarios. We present techniques to estimate costs for service dependency and to monitor costs associated with typical scientific applications. Experiments with real-world applications are performed to illustrate the usefulness of our techniques. Our service could eventually be integrated into cloud resource management and execution services to support on-the-fly resource scheduling.

[1]  Bernd Mohr,et al.  Program analysis tools for massively parallel applications: how to achieve highest performance , 2006, SC.

[2]  Thomas Fahringer,et al.  Performance Prophet: a performance modeling and prediction tool for parallel and distributed programs , 2005, 2005 International Conference on Parallel Processing Workshops (ICPPW'05).

[3]  Rajkumar Buyya,et al.  Market-Oriented Cloud Computing: Vision, Hype, and Reality for Delivering IT Services as Computing Utilities , 2008, 2008 10th IEEE International Conference on High Performance Computing and Communications.

[4]  Randy H. Katz,et al.  Above the Clouds: A Berkeley View of Cloud Computing , 2009 .

[5]  Matei Ripeanu,et al.  Amazon S3 for science grids: a viable solution? , 2008, DADC '08.

[6]  G. Bruce Berriman,et al.  On the Use of Cloud Computing for Scientific Workflows , 2008, 2008 IEEE Fourth International Conference on eScience.

[7]  Schahram Dustdar,et al.  Performance metrics and ontologies for Grid workflows , 2007, Future Gener. Comput. Syst..

[8]  Allen D. Malony,et al.  A Generic and Configurable Source-Code Instrumentation Component , 2009, ICCS.

[9]  Yong Zhao,et al.  Cloud Computing and Grid Computing 360-Degree Compared , 2008, GCE 2008.

[10]  Hong Linh Truong,et al.  DIPAS: A distributed performance analysis service for grid service-based workflows , 2009, Future Gener. Comput. Syst..

[11]  Miron Livny,et al.  The cost of doing science on the cloud: The Montage example , 2008, 2008 SC - International Conference for High Performance Computing, Networking, Storage and Analysis.

[12]  Constantinos Evangelinos,et al.  Cloud Computing for parallel Scientific HPC Applications: Feasibility of Running Coupled Atmosphere- , 2008 .

[13]  Jesús Labarta,et al.  Prediction of behavior of MPI applications , 2008, 2008 IEEE International Conference on Cluster Computing.