Topology-Based Hypercube Structures for Global Communication in Heterogeneous Networks

Hypercube structures are heavily used by parallel algorithms that require all-to-all communication. When communicating over a heterogeneous and irregular network, the performance obtained by the hypercube structure will depend on the matching of the hypercube structure to the topology of the underlying network. In this paper, we present strategies to build topology-based hypercubes structures. These strategies do not assume any kind of topology. They take into account the communication cost between pair of nodes to provide a performance-efficient hypercube structure. These enhanced hypercube structures help improve the performance of parallel applications that require all-to-all communication in heterogeneous networks by up to ~30%

[1]  Fikret Erçal,et al.  Time-Efficient Maze Routing Algorithms on Reconfigurable Mesh Architectures , 1997, J. Parallel Distributed Comput..

[2]  Volker Heun,et al.  Efficient Dynamic Embeddings of Binary Trees into Hypercubes , 2002, J. Algorithms.

[3]  Message P Forum,et al.  MPI: A Message-Passing Interface Standard , 1994 .

[4]  Michael Mikolajczak,et al.  Designing And Building Parallel Programs: Concepts And Tools For Parallel Software Engineering , 1997, IEEE Concurrency.

[5]  Dhabaleswar K. Panda,et al.  Communication modeling of heterogeneous networks of workstations for performance characterization of collective operations , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[6]  F. Leighton,et al.  Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes , 1991 .

[7]  Sartaj Sahni,et al.  Hypercube algorithms for image processing and pattern recognition , 1990 .

[8]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[9]  Massimo Bernaschi,et al.  Collective communication operations: experimental results vs. theory , 1998, Concurr. Pract. Exp..

[10]  Paul N. Swarztrauber,et al.  Multiprocessor FFTs , 1987, Parallel Comput..

[11]  Message Passing Interface Forum.  MPI: A message - passing interface standard , 1994 .

[12]  Ian T. Foster,et al.  A Grid-Enabled MPI: Message Passing in Heterogeneous Distributed Computing Systems , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[13]  Silvia M. Figueira,et al.  Dynamically Adaptive Binomial Trees for Broadcasting in Heterogeneous Networks of Workstations , 2004, VECPAR.

[14]  Henri E. Bal,et al.  MagPIe: MPI's collective communication operations for clustered wide area systems , 1999, PPoPP '99.

[15]  M. Livingston,et al.  Embeddings in hypercubes , 1988 .

[16]  M. J. Quinn,et al.  Parallel computing (2nd ed.): theory and practice , 1994 .

[17]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[18]  John N. Tsitsiklis,et al.  Optimal Communication Algorithms for Hypercubes , 1991, J. Parallel Distributed Comput..

[19]  George K. Thiruvathukal,et al.  Wide-Area Implementation of the Message Passing Interface , 1998, Parallel Comput..

[20]  C. Loan.  Computational Frameworks for the Fast Fourier Transform , 1992 .

[21]  Bruce Lowekamp,et al.  ECO: Efficient Collective Operations for communication on heterogeneous networks , 1996, Proceedings of International Conference on Parallel Processing.

[22]  Henri E. Bal,et al.  Bandwidth-efficient collective communication for clustered wide area systems , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[23]  David S. Johnson,et al.  Computers and In stractability: A Guide to the Theory of NP-Completeness. W. H Freeman, San Fran , 1979 .

[24]  Yu-Chee Tseng,et al.  Low-congestion Embedding of Multiple Graphs in a Hypercube , 1992 .

[25]  Jack Dongarra,et al.  Pvm: A Users' Guide and Tutorial for Network Parallel Computing , 1994 .

[26]  Rong-Jaye Chen,et al.  Compact Embedding of Binary Trees Into Hyperacubes , 1995, Inf. Process. Lett..

[27]  S. Lennart Johnsson,et al.  Communication Efficient Basic Linear Algebra Computations on Hypercube Architectures , 1987, J. Parallel Distributed Comput..

[28]  Dhabaleswar K. Panda,et al.  Efficient collective communication on heterogeneous networks of workstations , 1998, Proceedings. 1998 International Conference on Parallel Processing (Cat. No.98EX205).

[29]  Bronis R. de Supinski,et al.  Exploiting hierarchy in parallel computer networks to optimize collective operation performance , 2000, Proceedings 14th International Parallel and Distributed Processing Symposium. IPDPS 2000.

[30]  M. J. Quinn,et al.  Parallel Computing: Theory and Practice , 1994 .

[31]  E T. Leighton,et al.  Introduction to parallel algorithms and architectures , 1991 .

[32]  Sartaj Sahni,et al.  Hypercube Algorithms: with Applications to Image Processing and Pattern Recognition , 1990 .