MPICH-G2 Implementation of an Interactive Artificial Neural Network Training

Distributed Training of an Artificial Neural Network (ANN) has been implemented using MPICH-G2, and deployed on the testbed of the european CrossGrid project. Load balancing, including adaptative techniques, has been used to cope with the heterogeneous setup of computing resources. First results show the feasibility of this approach, and the opportunity for a Quality of Service framework. To give an example, a reduction in the training time from 20 minutes using a single local node downto less than 3 minutes using 10 nodes distributed across Spain, Poland, and Portugal, has been obtained.

[1]  William Gropp,et al.  MPICH-GQ: Quality-of-Service for Message Passing Programs , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[2]  Ian T. Foster,et al.  MPICH-G2: A Grid-enabled implementation of the Message Passing Interface , 2002, J. Parallel Distributed Comput..

[3]  Bronis R. de Supinski,et al.  Exploiting hierarchy in parallel computer networks to optimize collective operation performance , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[4]  Brian Tierney,et al.  File and Object Replication in Data Grids , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[5]  Brian Tierney,et al.  File and Object Replication in Data Grids , 2004, Cluster Computing.

[6]  Marios D. Dikaiakos,et al.  First Prototype of the CrossGrid Testbed , 2003, European Across Grids Conference.

[7]  Jack Dongarra,et al.  Recent Advances in Parallel Virtual Machine and Message Passing Interface, 15th European PVM/MPI Users' Group Meeting, Dublin, Ireland, September 7-10, 2008. Proceedings , 2008, PVM/MPI.

[8]  O. Ponce,et al.  Training of Neural Networks: Interactive Possibilities in a Distributed Framework , 2002, PVM/MPI.

[9]  Peter Z. Kunszt,et al.  Giggle: A Framework for Constructing Scalable Replica Location Services , 2002, ACM/IEEE SC 2002 Conference (SC'02).

[10]  Ian T. Foster,et al.  The Anatomy of the Grid: Enabling Scalable Virtual Organizations , 2001, Int. J. High Perform. Comput. Appl..