An evolutionary approach to collective communication scheduling

In this paper, we describe two evolutionary algorithms aimed at scheduling collective communications on interconnection networks of parallel computers. To avoid contention for links and associated delays, collective communications proceed in synchronized steps. Minimum number of steps is sought for the given network topology, wormhole (pipelined) switching, minimum routing and given sets of sender and/or receiver nodes. Used algorithms are able not only re-invent optimum schedules for known symmetric topologies like hyper-cubes, but they can find schedules even for any asymmetric or irregular topologies in case of general many-to-many collective communications. In most cases does the number of steps reach the theoretical lower bound for the given type of collective communication; if it does not, non-minimum routing can provide further improvement. Optimum schedules may serve for writing high-performance communication routines for application-specific networks on chip or for development of communication libraries in case of general-purpose interconnection networ.

[1]  Sujit Dey,et al.  An Interconnect Architecture for Networking Systems on Chips , 2002, IEEE Micro.

[2]  Sudhakar Yalamanchili,et al.  Interconnection Networks: An Engineering Approach , 2002 .

[3]  Philip K. McKinley,et al.  An Extended Dominating Node Approach to Broadcast and Global Combine in Multiport Wormhole-Routed Mesh Networks , 1997, IEEE Trans. Parallel Distributed Syst..

[4]  Roger D. Hersch,et al.  Efficient liquid schedule search strategies for collective communications , 2004, Proceedings. 2004 12th IEEE International Conference on Networks (ICON 2004) (IEEE Cat. No.04EX955).

[5]  M L Arendt.  Practical parallel processing , 1986 .

[6]  Josef Schwarz,et al.  HYBRID PARALLEL SIMULATED ANNEALING USING GENETIC OPERATIONS , 2004 .

[7]  Jirí Jaros,et al.  Evolutionary Design of Group Communication Schedules for Interconnection Networks , 2005, ISCIS.

[8]  Philip K. McKinley,et al.  A Broadcast Algorithm for All-Port Wormhole-Routed Torus Networks , 1996, IEEE Trans. Parallel Distributed Syst..

[9]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[10]  J. A. Lozano,et al.  Estimation of Distribution Algorithms: A New Tool for Evolutionary Computation , 2001 .

[11]  Axel Jantsch,et al.  Networks on chip , 2003 .

[12]  Satish K. Tripathi,et al.  Parallel and Distributed Computing Handbook , 1995 .

[13]  Jirí Jaros,et al.  Evolutionary Design of OAB and AAB Communication Schedules for Interconnection Networks , 2006, EvoWorkshops.

[14]  David E. Goldberg,et al.  Genetic Algorithms in Search Optimization and Machine Learning , 1988 .