Redistribution de données à travers un réseau à haut débit. (Data Redistribution through High Performance Networks)

Nous considerons ici le probleme ou deux programmes differents situes sur deux grappes d'ordinateurs distantes, reliees par un reseau a haut debit, forment un couplage de code et echangent regulierement des donnees. Un tel echange s'effectue par une redistribution de donnees. Nous etudions comment effectuer une telle redistribution le plus efficacement possible en minimisant temps de communication et congestion du reseau. Nous utilisons pour ce faire, une modelisation du probleme a l'aide de graphes bipartis. Le modele choisi permet une prise en compte du delai d'initialisation des communications, des differentes bandes passantes et impose une limite d'une communication simultanee par interface reseau (modele 1-port) et de k communications simultanees sur la dorsale. Nous effectuons une validation experimentale du modele puis l'utilisons pour developper deux algorithmes d'ordonnancement des communications. Nous montrons que chacun d'entre eux est un algorithme d'approximation garantissant un temps d'execution dans le pire des cas 8/3 fois plus eleve que le temps optimal. Nous concluons l'etude de ces algorithmes par une serie d'experiences demontrant de bonnes performances en pratique. Enfin, nous etendons le probleme initial au cas de grappes heterogenes : ce cas imposant de sortir du modele 1-port, nous montrons comment modifier nos algorithmes pour en tirer parti. Nous etudions egalement le cas de redistributions executees en regime permanent sur un reseau d'une topologie plus complexe autorisant les communications locales.

[1]  Susan M. Mniszewski,et al.  PAWS: collective interactions and data transfers , 2001, Proceedings 10th IEEE International Symposium on High Performance Distributed Computing.

[2]  Mitsuhisa Sato,et al.  Design and implementations of Ninf: towards a global computing infrastructure , 1999, Future Gener. Comput. Syst..

[3]  Teofilo F. Gonzalez,et al.  Open Shop Scheduling to Minimize Finish Time , 1976, JACM.

[4]  Emmanuel Jeannot,et al.  New dynamic heuristics in the client-agent-server model , 2003, Proceedings International Parallel and Distributed Processing Symposium.

[5]  Yves Robert,et al.  Optimizing the steady-state throughput of Broadcasts on heterogeneous platforms , 2003 .

[6]  Aura Ganz,et al.  A time-wavelength assignment algorithm for a WDM star network , 1992, [Proceedings] IEEE INFOCOM '92: The Conference on Computer Communications.

[7]  Wu-chun Feng,et al.  Dynamic Right-Sizing: An Automated, Lightweight, and Scalable Technique for Enhancing Grid Performance , 2002, Protocols for High-Speed Networks.

[8]  Minyi Guo,et al.  A Framework for Efficient Data Redistribution on Distributed Memory Multicomputers , 2001, The Journal of Supercomputing.

[9]  T. V. Lakshman,et al.  The performance of TCP/IP for networks with high bandwidth-delay products and random loss , 1997, TNET.

[10]  Henri Casanova,et al.  Netsolve: a Network-Enabled Server for Solving Computational Science Problems , 1997, Int. J. High Perform. Comput. Appl..

[11]  Xiaotie Deng,et al.  On Approximating a Scheduling Problem , 2001, J. Comb. Optim..

[12]  Evripidis Bampis,et al.  Scheduling in Switching Networks with Set-Up Delays , 2005, J. Comb. Optim..

[13]  David W. Walker,et al.  Redistribution of block-cyclic data distributions using MPI , 1996, Concurr. Pract. Exp..

[14]  Emmanuel Jeannot,et al.  Improving middleware performance with AdOC: an adaptive online compression library for data transfer , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[15]  Richard Wolski,et al.  The network weather service: a distributed resource performance forecasting service for metacomputing , 1999, Future Gener. Comput. Syst..

[16]  Emmanuel Jeannot,et al.  Messages Scheduling for Data Redistribution between Clusters , 2003, PPAM.

[17]  André Ribes,et al.  Parallel Programming: Parallel CORBA Objects for Programming Computational Grids , 2003, IEEE Distributed Syst. Online.

[18]  Yves Robert,et al.  Optimizing the steady-state throughput of scatter and reduce operations on heterogeneous platforms , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[19]  Robert L. Grossman,et al.  PSockets: The Case for Application-level Network Striping for Data Intensive Applications using High Speed Wide Area Networks , 2000, ACM/IEEE SC 2000 Conference (SC'00).

[20]  Yves Robert,et al.  Scheduling Block-Cyclic Array Redistribution , 1998, IEEE Trans. Parallel Distributed Syst..

[21]  Bernard Tourancheau,et al.  Efficient Block Cyclic Data Redistribution , 1996, Euro-Par, Vol. I.

[22]  Ken Kennedy,et al.  Fortran D Language Specification , 1990 .

[23]  James Arthur Kohl,et al.  Cumulvs: Providing Fault Toler. Ance, Visualization, and Steer Ing of Parallel Applications , 1996, Int. J. High Perform. Comput. Appl..

[24]  Viktor K. Prasanna,et al.  Block‐cyclic redistribution over heterogeneous networks , 2004, Cluster Computing.

[25]  Ken Kennedy,et al.  Evaluation of compiler optimizations for Fortran D on MIMD distributed memory machines , 1992, ICS '92.

[26]  G. Bongiovanni,et al.  An Optimum Time Slot Assignment Algorithm for an SS/TDMA System with Variable Number of Transponders , 1981, IEEE Trans. Commun..

[27]  Fred B. Schneider,et al.  A Theory of Graphs , 1993 .

[28]  Alexander Schrijver.  Bipartite Edge Coloring in O(Delta m) Time , 1998, SIAM J. Comput..

[29]  H. Kuhn.  The Hungarian method for the assignment problem , 1955 .

[30]  Jacek Blazewicz,et al.  Some preemptive open shop scheduling problems with a renewable or a nonrenewable resource , 1992, Discret. Appl. Math..

[31]  Guang R. Gao,et al.  Data parallelism with high performance C , 1994, CASCON.

[32]  Nancy P. Kronenberg,et al.  VAXcluster: a closely-coupled distributed system , 1986, TOCS.

[33]  Emmanuel Jeannot,et al.  Message Scheduling for Data Redistribution through High Performance Networks , 2004 .

[34]  Eugene L. Lawler,et al.  Sequencing and scheduling: algorithms and complexity , 1989 .

[35]  Michael A. Bender,et al.  Flow and stretch metrics for scheduling continuous job streams , 1998, SODA '98.

[36]  Vijay K. Naik,et al.  Harmony: a desktop grid for delivering enterprise computations , 2003, Proceedings. First Latin American Web Congress.

[37]  Wu-chun Feng,et al.  Dynamic Adjustment of TCP Window Sizes , 2000 .

[38]  Henri Casanova,et al.  A Network Model for Simulation of Grid Application , 2002 .

[39]  Douglas Thain,et al.  Building Reliable Clients and Services , 2004, The Grid 2, 2nd Edition.

[40]  Ching-Hsien Hsu,et al.  A Generalized Processor Mapping Technique for Array Redistribution , 2001, IEEE Trans. Parallel Distributed Syst..

[41]  A. J. Clewett,et al.  Introduction to sequencing and scheduling , 1974 .

[42]  Anthony Skjellum,et al.  A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard , 1996, Parallel Comput..

[43]  Viktor K. Prasanna,et al.  Efficient Algorithms for Block-Cyclic Array Redistribution between Processor Sets , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[44]  Ami Marowka,et al.  The GRID: Blueprint for a New Computing Infrastructure , 2000, Parallel Distributed Comput. Pract..

[45]  Hyeong-Ah Choi,et al.  Efficient scheduling of transmissions in optical broadcast networks , 1996, TNET.

[46]  G. Bongiovanni,et al.  An Optimal Switching Algorithm for Multibeam Satellite Systems with Variable Bandwidth Beams , 1982, IEEE Trans. Commun..

[47]  Piyush Mehrotra,et al.  Dynamic data distributions in Vienna Fortran , 1993, Supercomputing '93.

[48]  James Arthur Kohl,et al.  Data redistribution and remote method invocation in parallel component architectures , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[49]  Pravin M. Vaidya,et al.  An algorithm for linear programming which requires O(((m+n)n2+(m+n)1.5n)L) arithmetic operations , 1987, Math. Program..

[50]  Charles L. Seitz,et al.  Myrinet: A Gigabit-per-Second Local Area Network , 1995, IEEE Micro.