Algorithmic Issues on Heterogeneous Computing Platforms Algorithmic Issues on Heterogeneous Computing Platforms Algorithmic Issues on Heterogeneous Computing Platforms

This paper discusses some algorithmic issues when computing with a heterogeneous network of workstations (the typical poor man's parallel computer). Dealing with processors of diierent speeds requires to use more involved strategies than block-cyclic data distributions. Dynamic data distribution is a rst possibility but may prove impractical and not scalable due to communication and control overhead. Static data distributions tuned to balance execution times constitute another possibility but may prove ineecient due to variations in the processor speeds (e.g. because of diierent work-loads during the computation). We introduce a static distribution strategy that can be reened on the y, and we show that it is well-suited to parallelizing scientiic computing applications such as nite-diierence stencils or LU decomposition. RRsumm Cet article traite de probllmes algorithmiques liis l'impllmen-tation de programmes paralìeles sur un rrseau de stations hhttro-ggne (typiquement la machine paralllle du programmeur pauvre). Si dans le cadre d'un rrseau de machines homoggnes, une distribution cyclique des donnnes est souvent optimale, elle n'est plus du tout adaptte cette nouvelle connguration. Plusieurs stratt-gies sont envisageables : en particulier, s'opposent les distributions eeectuues dynamiquement (surcoot de communications) celles ef-fectuues statiquement (mauvaise estimations due aux variations de charge des machines). Ici, nous ddcoupons l'espace d'ittration en tranches ; la n de l'exxcution d'une tranche, connaissant le temps passs par chaque processeur au calcul, nous rrrvaluons leur vitesse ; Nous redistribuons alors les donnnes de maniire optimale pour l'exxcution de la tranche suivante. Finalement, nous montrons par des experimentations que notre approche est appropriie pour de nombreux noyaux de calcul scientiique comme la ddcomposition LU ou des probllmes aux diiirences nies. Abstract This paper discusses some algorithmic issues when computing with a heterogeneous network of workstations (the typical poor man's parallel computer). Dealing with processors of diierent speeds requires to use more involved strategies than block-cyclic data distributions. Dynamic data distribution is a rst possibility but may prove impractical and not scalable due to communication and control overhead. Static data distributions tuned to balance execution times constitute another possibility but may prove ineecient due to variations in the processor speeds (e.g. because of diierent workloads during the computation). We introduce a static distribution strategy that can be reened on the y, and we show that it is well-suited to parallelizing scientiic computing applications such as nite-diierence stencils or LU decomposition.