HPC Application Performance and Cost Efficiency in the Cloud

Unlike traditional cluster systems, the Cloud Computing paradigm provides access to an execution environment without upfront investments in hardware and facilities. Due to the elasticity and the pay-per-use billing model, it is possible to configure experimental environments with minimal idle costs. In this paper, we perform an extensive evaluation of the major commercial public clouds. Our results show that performance degradation due to virtualization and other cloud overheads is insignificant. However, the network interconnection in the cloud still remains a large bottleneck for HPC application performance.

[1]  Brian Hayes,et al.  What Is Cloud Computing? , 2019, Cloud Technologies.

[2]  Xin Yuan,et al.  A comparative study of high-performance computing on the cloud , 2013, HPDC.

[3]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[4]  Jianlong Zhong,et al.  Network Performance Aware MPI Collective Communication Operations in the Cloud , 2015, IEEE Transactions on Parallel and Distributed Systems.

[5]  Mohamed Ben Belgacem,et al.  A High Level Framework to Develop and Run E-science Applications on Cloud Infrastructures , 2015, 2015 IEEE 17th International Conference on High Performance Computing and Communications, 2015 IEEE 7th International Symposium on Cyberspace Safety and Security, and 2015 IEEE 12th International Conference on Embedded Software and Systems.

[6]  Carreño Ed,et al.  Communication Optimization of Parallel Applications in the Cloud , 2016 .

[7]  Martin Gilje Jaatun,et al.  Cloud Computing, First International Conference, CloudCom 2009, Beijing, China, December 1-4, 2009. Proceedings , 2009, CloudCom.

[8]  Cristiano André da Costa,et al.  AutoElastic: Automatic Resource Elasticity for High Performance Applications in the Cloud , 2016, IEEE Transactions on Cloud Computing.

[9]  Rolf Riesen,et al.  Communication patterns , 2006 .

[10]  Philippe Olivier Alexandre Navaux,et al.  High Performance Computing in the cloud: Deployment, performance and cost efficiency , 2012, 4th IEEE International Conference on Cloud Computing Technology and Science Proceedings.

[11]  Shujia Zhou,et al.  Case study for running HPC applications in public clouds , 2010, HPDC '10.

[12]  Omran Malik Omer Awad,et al.  Cloud computing versus in-house clusters: a comparative study , 2014, 2014 World Congress on Computer Applications and Information Systems (WCCAIS).

[13]  Dhabaleswar K. Panda,et al.  MVAPICH2 over OpenStack with SR-IOV: An Efficient Approach to Build HPC Clouds , 2015, 2015 15th IEEE/ACM International Symposium on Cluster, Cloud and Grid Computing.

[14]  Alexandru Iosup,et al.  Performance Analysis of Cloud Computing Services for Many-Tasks Scientific Computing , 2011, IEEE Transactions on Parallel and Distributed Systems.