Evaluation of messaging middleware for high-performance cloud computing

Cloud computing is posing several challenges, such as security, fault tolerance, access interface singularity, and network constraints, both in terms of latency and bandwidth. In this scenario, the performance of communications depends both on the network fabric and its efficient support in virtualized environments, which ultimately determines the overall system performance. To solve the current network constraints in cloud services, their providers are deploying high-speed networks, such as 10 Gigabit Ethernet. This paper presents an evaluation of high-performance computing message-passing middleware on a cloud computing infrastructure, Amazon EC2 cluster compute instances, equipped with 10 Gigabit Ethernet. The analysis of the experimental results, confronted with a similar testbed, has shown the significant impact that virtualized environments still have on communication performance, which demands more efficient communication middleware support to get over the current cloud network limitations.

[1]  Sabela Ramos,et al.  Java in the High Performance Computing arena: Research, practice and experience , 2013, Sci. Comput. Program..

[2]  Nectarios Koziris,et al.  MyriXen: Message Passing in Xen Virtual Machines over Myrinet and Ethernet , 2009, Euro-Par Workshops.

[3]  T. S. Eugene Ng,et al.  The Impact of Virtualization on Network Performance of Amazon EC2 Data Center , 2010, 2010 Proceedings IEEE INFOCOM.

[4]  David H. Bailey,et al.  The Nas Parallel Benchmarks , 1991, Int. J. High Perform. Comput. Appl..

[5]  George Bosilca,et al.  Open MPI: Goals, Concept, and Design of a Next Generation MPI Implementation , 2004, PVM/MPI.

[6]  Gil Neiger,et al.  Intel ® Virtualization Technology for Directed I/O , 2006 .

[7]  Young-Sik Jeong,et al.  Efficient Server Virtualization using Grid Service Infrastructure , 2010, J. Inf. Process. Syst..

[8]  Juan Touriño,et al.  F-MPJ: scalable Java message-passing communications on parallel systems , 2012, The Journal of Supercomputing.

[9]  Michael Anthony Bauer,et al.  Hpcbench - a Linux-based network benchmark for high performance networks , 2005, 19th International Symposium on High Performance Computing Systems and Applications (HPCS'05).

[10]  Kieran Mansley,et al.  Getting 10 Gb/s from Xen: Safe and Fast Device Access from Unprivileged Domains , 2007, Euro-Par Workshops.

[11]  Edward Walker,et al.  Benchmarking Amazon EC2 for High-Performance Scientific Computing , 2008, login Usenix Mag..

[12]  Yunming Ye,et al.  A comparative study of feature weighting methods for document co-clustering , 2011, Int. J. Inf. Technol. Commun. Convergence.

[13]  Guillaume Mercier,et al.  Design and evaluation of Nemesis, a scalable, low-latency, message-passing communication subsystem , 2006, Sixth IEEE International Symposium on Cluster Computing and the Grid (CCGRID'06).

[14]  Anil Kumar Karna,et al.  Cross Comparison on C Compilers' Reliability Impact on 64-Bit Windows Vista OS , 2010, 2010 2nd International Conference on Information Technology Convergence and Services.

[15]  Nathan Regola,et al.  Recommendations for Virtualization Technologies in High Performance Computing , 2010, 2010 IEEE Second International Conference on Cloud Computing Technology and Science.

[16]  Ben Lee,et al.  Eager Data Transfer Mechanism for Reducing Communication Latency in User-Level Network Protocols , 2008, J. Inf. Process. Syst..

[17]  Juan Touriño,et al.  NPB-MPJ: NAS Parallel Benchmarks Implementation for Message-Passing in Java , 2009, 2009 17th Euromicro International Conference on Parallel, Distributed and Network-based Processing.

[18]  Sabela Ramos,et al.  Design of efficient Java message-passing collectives on multi-core clusters , 2010, The Journal of Supercomputing.

[19]  Andrew Warfield,et al.  Safe Hardware Access with the Xen Virtual Machine Monitor , 2007 .

[20]  Yi-Ming Chen,et al.  Retrospective detection of malware attacks by cloud computing , 2010, 2010 International Conference on Cyber-Enabled Distributed Computing and Knowledge Discovery.

[21]  Mark Baker,et al.  MPJ: A Proposed Java Message Passing API and Environment for High Performance Computing , 2000, IPDPS Workshops.

[22]  Calvin J. Ribbens,et al.  Hybrid Computing - Where HPC meets grid and Cloud Computing , 2011, Future Gener. Comput. Syst..

[23]  Karsten Schwan,et al.  High performance and scalable I/O virtualization via self-virtualized devices , 2007, HPDC '07.

[24]  Anil Kumar Karna,et al.  Cross Comparison on C Compilers' Reliability Impact on UNIX Based Fedora OS , 2010, 2010 10th IEEE International Conference on Computer and Information Technology.

[25]  Dhabaleswar K. Panda,et al.  Virtual machine aware communication libraries for high performance computing , 2007, Proceedings of the 2007 ACM/IEEE Conference on Supercomputing (SC '07).

[26]  Geoffrey C. Fox,et al.  mpiJava 1.2: API Specification , 1999 .

[27]  Dhabaleswar K. Panda,et al.  A case for high performance computing with virtual machines , 2006, ICS '06.

[28]  Marianne Shaw,et al.  Denali: Lightweight Virtual Machines for Distributed and Networked Applications , 2001 .

[29]  Dhabaleswar K. Panda,et al.  High Performance VMM-Bypass I/O in Virtual Machines , 2006, USENIX Annual Technical Conference, General Track.