Localized Communications of Data Parallel Programs on Multi-cluster Grid Systems

The advent of widely interconnected computing resources introduces the technologies of grid computing. A typical grid system, the cluster grid, consists of several clusters located in multiple campuses that distributed globally over the Internet. Because of the Internet infrastructure of cluster grid, the communication overhead becomes as key factor to the performance of applications on cluster grid. In this paper, we present a processor reordering technique for the communication optimizations of data parallel programs on cluster grid. The alignment of data in parallel programs is considered as example to examine the proposed techniques. Effectiveness of the processor reordering technique is to reduce the inter-cluster communication overheads and to speedup the execution of parallel applications in the underlying distributed clusters. Our preliminary analysis and experimental results of the proposed method on mapping data to logical grid nodes show improvement of communication costs and conduce to better performance of parallel programs on different hierarchical grid of cluster systems.

[1]  Dharma P. Agrawal,et al.  Scheduling of periodic time critical applications for pipelined execution on heterogeneous systems , 2001, International Conference on Parallel Processing, 2001..

[2]  Ian T. Foster,et al.  Globus: a Metacomputing Infrastructure Toolkit , 1997, Int. J. High Perform. Comput. Appl..

[3]  Bu-Sung Lee,et al.  Key Message Algorithm: a communication optimization algorithm in cluster-based parallel computing , 1999, ICWC 99. IEEE Computer Society International Workshop on Cluster Computing.

[4]  Lionel M. Ni,et al.  Processor mapping techniques toward efficient data redistribution , 1994, Proceedings of 8th International Parallel Processing Symposium.

[5]  Florin Isaila,et al.  Mapping functions and data redistribution for parallel files , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[6]  Lionel M. Ni,et al.  Processor Mapping Techniques Toward Efficient Data Redistribution , 1995, IEEE Trans. Parallel Distributed Syst..

[7]  Minyi Guo,et al.  A Framework for Efficient Data Redistribution on Distributed Memory Multicomputers , 2001, The Journal of Supercomputing.

[8]  Henri E. Bal,et al.  Sensitivity of parallel applications to large differences in bandwidth and latency in two-layer interconnects , 1999, Proceedings Fifth International Symposium on High-Performance Computer Architecture.

[9]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[10]  Xiao Qin,et al.  Dynamic, reliability-driven scheduling of parallel real-time jobs in heterogeneous systems , 2001, International Conference on Parallel Processing, 2001..

[11]  Myong-Soon Park,et al.  Processor reordering algorithms toward efficient GEN_BLOCK redistribution , 2001, SAC.

[12]  Jens Knoop,et al.  Distribution Assignment Placement: Effective Optimization of Redistribution Costs , 2002, IEEE Trans. Parallel Distributed Syst..

[13]  Yves Robert,et al.  Optimal algorithms for scheduling divisible workloads on heterogeneous systems , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[14]  Ian T. Foster.  Building an open grid , 2003, Second IEEE International Symposium on Network Computing and Applications, 2003. NCA 2003..

[15]  Viktor K. Prasanna,et al.  Efficient Algorithms for Block-Cyclic Redistribution of Arrays , 1999, Algorithmica.

[16]  Subhash Saini,et al.  Local grid scheduling techniques using performance prediction , 2003 .

[17]  Ian T. Foster,et al.  Condor-G: A Computation Management Agent for Multi-Institutional Grids , 2004, Cluster Computing.

[18]  Yolanda Gil,et al.  The Role of Planning in Grid Computing , 2003, ICAPS.

[19]  Yves Robert,et al.  Scheduling divisible workloads on heterogeneous platforms , 2003, Parallel Comput..

[20]  Henri E. Bal,et al.  Optimizing parallel applications for wide-area clusters , 1998, Proceedings of the First Merged International Parallel Processing Symposium and Symposium on Parallel and Distributed Processing.

[21]  Peter E. Strazdins,et al.  Optimizing user-level communication patterns on the Fujitsu AP3000 , 1999, ICWC 99. IEEE Computer Society International Workshop on Cluster Computing.

[22]  Ian Foster,et al.  The Grid 2 - Blueprint for a New Computing Infrastructure, Second Edition , 1998, The Grid 2, 2nd Edition.