Description and Packaging of MPI Applications for Automatic Deployment on Computational Grids

Computational grids promise to deliver a vast computer power as transparently as the electric power grid supplies electricity. Thus, applications need to be automatically and transparently deployed on grids, in particular MPI parallel applications. However, deploying such applications on a grid is complex: the user must select compute nodes manually, launch processes on distributed, heterogeneous nodes, and configure the MPI library so it can adapt to its grid environment. Our objective is to hide the complexity of application deployment on computational grids. This paper proposes a software architecture designed to automatically deploy MPI applications on a grid. It also introduces a description and packaging model of MPI applications. Finally, the paper shows how the MPI library can be automatically configured with network topology information to optimize collective operations. / L'un des objectifs des grilles de calcul est de fournir de la puissance informatique de maniere aussi transparente que le reseau de distribution de l'electricite fournit de la puissance electrique. Ainsi, les applications doivent se deployer sur les grilles de calcul automatiquement et de facon transparente, en particulier en ce qui concerne les applications paralleles MPI. Cependant, le deploiement de telles applications dans un environnement de grille est une tâche complexe : l'utilisateur doit selectionner manuellement les noeuds de calcul, lancer des processus sur des noeuds distribues et heterogenes, puis configurer la bibliotheque MPI pour qu'elle puisse s'adapter a son environnement d'execution. Notre objectif est de cacher la complexite du deploiement d'applications sur des grilles de calcul. Cet article propose une architecture logicielle concue pour deployer automatiquement des applications MPI sur des grilles. Il presente egalement un modele de description et de packaging d'applications MPI. Enfin, cet article montre comment la bibliotheque MPI peut etre automatiquement configuree, en y injectant la description de la topologie reseau, afin d'optimiser la performance des operations collectives MPI.

[1]  Ian T. Foster,et al.  MPICH-G2: A Grid-enabled implementation of the Message Passing Interface , 2002, J. Parallel Distributed Comput..

[2]  Bronis R. de Supinski,et al.  Exploiting hierarchy in parallel computer networks to optimize collective operation performance , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[3]  André Ribes,et al.  A Parallel CORBA Component Model for Numerical Code Coupling , 2002, GRID.

[4]  Rajesh Raman,et al.  Matchmaking: distributed resource management for high throughput computing , 1998, Proceedings. The Seventh International Symposium on High Performance Distributed Computing (Cat. No.98TB100244).

[5]  Jacek Blazewicz,et al.  Scheduling Malleable Tasks on Parallel Processors to Minimize the Makespan , 2004, Ann. Oper. Res..

[6]  Ian T. Foster,et al.  Grid information services for distributed resource sharing , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[7]  Christian Pérez,et al.  A Software Architecture for Automatic Deployment of CORBA Components Using Grid Technologies , 2004, ArXiv.

[8]  Kees Verstoep,et al.  Network performance-aware collective communication for clustered wide-area systems , 2001, Parallel Comput..

[9]  Christian Pérez,et al.  A network topology description model for grid application deployment , 2004, Fifth IEEE/ACM International Workshop on Grid Computing.

[10]  J. L. Traff.  Implementing the MPI Process Topology Mechanism , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[11]  GWD-C.  A Hierarchy of Network Performance Characteristics for Grid Applications and Services , 2003 .

[12]  Hee Yong Youn,et al.  Mapping strategies for switch-based cluster systems of irregular topology , 2001, Proceedings. Eighth International Conference on Parallel and Distributed Systems. ICPADS 2001.

[13]  Ian T. Foster,et al.  Supporting Efficient Execution in Heterogeneous Distributed Computing Environments with Cactus and Globus , 2001, ACM/IEEE SC 2001 Conference (SC'01).

[14]  Christian Pérez,et al.  Deploying CORBA Components on a Computational Grid: General Principles and Early Experiments Using the Globus Toolkit , 2004, Component Deployment.

[15]  Werner Augustin,et al.  On Benchmarking Collective MPI Operations , 2002, PVM/MPI.

[16]  Bronis R. de Supinski,et al.  Accurately measuring MPI broadcasts in a computational grid , 1999, Proceedings. The Eighth International Symposium on High Performance Distributed Computing (Cat. No.99TH8469).

[17]  Ladislau Bölöni,et al.  A comparison study of static mapping heuristics for a class of meta-tasks on heterogeneous computing systems , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[18]  G. Mahinthakumar,et al.  Multivariate Geographic Clustering in A Metacomputing Environment Using Globus , 1999, ACM/IEEE SC 1999 Conference (SC'99).

[19]  Steve Loughran,et al.  Configuration Description, Deployment, and Lifecycle Management (CDDLM) Foundation Document , 2005 .

[20]  Henri Casanova,et al.  A realistic network/application model for scheduling divisible loads on large-scale platforms , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.