Communication-aware scheduling on heterogeneous master-worker platforms

Les travaux presentes dans cette these portent sur diverses techniques d'ordonnancement de tâches independantes pour des plates-formes de type maitre-esclaves distribuees a grande echelle, lorsque les temps de communications des tâches sont pris en compte par des modeles realistes. Les contributions de cette these se situent a trois niveaux : 1) Algorithmique Parallele : nous avons montre la complexite d'ordonnancer des tâches independantes sur une plate-forme heterogene en modelisant les communications avec un modele un-port, en regardant plusieurs sources d'heterogeneite et plusieurs fonctions objectives; 2) Produit de matrices : nous avons calcule la borne theorique du volume de communication minimal necessaire pour effectuer un produit de matrices dont les donnees sont centralisees, et ou la memoire des esclaves est limitee, et nous avons defini un algorithme efficace de partage de la memoire, impliquant un volume de communication proche de la borne theorique. Nous avons ensuite etendu cet algorithme a des plate-formes heterogenes; 3) Ordonnancement : dans le cadre d'ordonnancement d'applications constituees d'un tres grand nombre de tâches independantes et de caracteristiques identiques, nous avons etudie en regime permanent comment minimiser le retard de chaque application lorsqu'elles sont plusieurs a entrer en competition pour les ressources de calcul, et comment minimiser la consommation de la plate-forme lorsqu'une seule application est deployee

[1]  Lynn Elliot Cannon,et al.  A cellular computer to implement the kalman filter algorithm , 1969 .

[2]  H. Ali,et al.  Task Scheduling in Multiprocessing Systems , 1995, Computer.

[3]  Jirí Sgall,et al.  On-line Scheduling , 1996, Online Algorithms.

[4]  Dhabaleswar K. Panda,et al.  Efficient collective communication on heterogeneous networks of workstations , 1998, Proceedings. 1998 International Conference on Parallel Processing (Cat. No.98EX205).

[5]  Michael J. Quinn,et al.  Block data decomposition for data-parallel programming on a heterogeneous workstation network , 1993, [1993] Proceedings The 2nd International Symposium on High Performance Distributed Computing.

[6]  Joseph Y.-T. Leung,et al.  Minimizing mean flowtime and makespan on master-slave systems , 2005, J. Parallel Distributed Comput..

[7]  D. Chen,et al.  Energy minimization of real-time tasks on variable voltage processors with transition energy overhead , 2003, Proceedings of the ASP-DAC Asia and South Pacific Design Automation Conference, 2003..

[8]  Larry Rudolph,et al.  Metrics and Benchmarking for Parallel Job Scheduling , 1998, JSSPP.

[9]  Jack J. Dongarra,et al.  Key Concepts for Parallel Out-of-Core LU Factorization , 1996, Parallel Comput..

[10]  Oliver Sinnen,et al.  Task Scheduling for Parallel Systems (Wiley Series on Parallel and Distributed Computing) , 2007 .

[11]  Yves Robert,et al.  Mapping and Load-Balancing Iterative Computations on Heterogeneous Clusters , 2003, PVM/MPI.

[12]  Wayne E. Smith.  Various optimizers for single‐stage production , 1956 .

[13]  Yves Robert,et al.  The Master-Slave Paradigm with Heterogeneous Processors , 2001, CLUSTER.

[14]  Rami Melhem,et al.  Multi-version scheduling in rechargeable energy-aware real-time systems , 2005, J. Embed. Comput..

[15]  Sanjay Ranka,et al.  Array Decompositions for Nonuniform Computational Environments , 1996, J. Parallel Distributed Comput..

[16]  Alexey L. Lastovetsky,et al.  Heterogeneous Distribution of Computations While Solving Linear Algebra Problems on Networks of Heterogeneous Computers , 1999, HPCN Europe.

[17]  Chita R. Das,et al.  A comprehensive performance and energy consumption analysis of scheduling alternatives in clusters , 2007, The Journal of Supercomputing.

[18]  Constantine D. Polychronopoulos.  Compiler Optimizations for Enhancing Parallelism and Their Impact on Architecture Design , 1988, IEEE Trans. Computers.

[19]  Samir Khuller,et al.  On broadcasting in heterogenous networks , 2004, SODA '04.

[20]  Tsan-sheng Hsu,et al.  Scheduling Problems in a Practical Allocation Model , 1997, J. Comb. Optim..

[21]  Javier Cuenca,et al.  Processes Distribution of Homogeneous Parallel Linear Algebra Routines on Heterogeneous Clusters , 2005, 2005 IEEE International Conference on Cluster Computing.

[22]  Jan Karel Lenstra,et al.  Scheduling subject to resource constraints: classification and complexity , 1983, Discret. Appl. Math..

[23]  Yves Robert,et al.  Static tiling for heterogeneous computing platforms , 1999, Parallel Comput..

[24]  Hiroto Yasuura,et al.  Real-time task scheduling for a variable voltage processor , 1999, Proceedings 12th International Symposium on System Synthesis.

[25]  Henri Casanova,et al.  Scheduling divisible loads on star and tree networks: results and open problems , 2005, IEEE Transactions on Parallel and Distributed Systems.

[26]  Francisco Vilar Brasileiro,et al.  Running Bag-of-Tasks applications on computational grids: the MyGrid approach , 2003, 2003 International Conference on Parallel Processing, 2003. Proceedings..

[27]  Larry Carter,et al.  On the interference of communication on computation in Java , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[28]  Yves Robert,et al.  Off-line and on-line scheduling on heterogeneous master-slave platforms , 2006, 14th Euromicro International Conference on Parallel, Distributed, and Network-Based Processing (PDP'06).

[29]  Viktor K. Prasanna,et al.  Bandwidth-aware resource allocation for heterogeneous computing systems to maximize throughput , 2003, 2003 International Conference on Parallel Processing, 2003. Proceedings..

[30]  Martin Skutella,et al.  The power of -points in preemptive single machine scheduling , 2002 .

[31]  Rami G. Melhem,et al.  Multiversion scheduling in rechargeable energy-aware real-time systems , 2003, 15th Euromicro Conference on Real-Time Systems, 2003. Proceedings..

[32]  Jeff T. Linderoth,et al.  An enabling framework for master-worker applications on the Computational Grid , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[33]  Cláudio T. Silva,et al.  A unified infrastructure for parallel out-of-core isosurface extraction and volume rendering of unstructured grids , 2001, Proceedings IEEE 2001 Symposium on Parallel and Large-Data Visualization and Graphics (Cat. No.01EX520).

[34]  Henri Casanova,et al.  Scheduling distributed applications: the SimGrid simulation framework , 2003, CCGrid 2003. 3rd IEEE/ACM International Symposium on Cluster Computing and the Grid, 2003. Proceedings..

[35]  Alan Watt,et al.  3D Computer Graphics , 1993 .

[36]  R. K. Shyamasundar,et al.  Introduction to algorithms , 1996 .

[37]  Alexey L. Lastovetsky,et al.  Heterogeneous Distribution of Computations Solving Linear Algebra Problems on Networks of Heterogeneous Computers , 2001, J. Parallel Distributed Comput..

[38]  Victor Y. Pan,et al.  Parallel Matrix Multiplication on a Linear Array with a Reconfigurable Pipelined Bus System , 2001, IEEE Trans. Computers.

[39]  Francine Berman,et al.  Overview of the Book: Grid Computing – Making the Global Infrastructure a Reality , 2003 .

[40]  Yves Robert,et al.  Revisiting Matrix Product on Master-Worker Platforms , 2006, 2007 IEEE International Parallel and Distributed Processing Symposium.

[41]  Yves Robert,et al.  The impact of heterogeneity on master-slave on-line scheduling , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[42]  Jon B. Weissman,et al.  Scheduling multi-component applications in heterogeneous wide-area networks , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[43]  Sanjeev Khanna,et al.  Approximation schemes for preemptive weighted flow time , 2002, STOC '02.

[44]  Larry Carter,et al.  Interference-Aware Scheduling , 2006, Int. J. High Perform. Comput. Appl..

[45]  Yves Robert,et al.  Matrix product on heterogeneous master-worker platforms , 2008, PPoPP.

[46]  J. M. Moore.  An n Job, One Machine Sequencing Algorithm for Minimizing the Number of Late Jobs , 1968 .

[47]  Viktor K. Prasanna,et al.  Efficient collective communication in distributed heterogeneous systems , 1999, Proceedings. 19th IEEE International Conference on Distributed Computing Systems (Cat. No.99CB37003).

[48]  Rami G. Melhem,et al.  Scheduling with Dynamic Voltage/Speed Adjustment Using Slack Reclamation in Multiprocessor Real-Time Systems , 2003, IEEE Trans. Parallel Distributed Syst..

[49]  H. T. Kung,et al.  I/O complexity: The red-blue pebble game , 1981, STOC '81.

[50]  Miron Livny,et al.  Adaptive Scheduling for Master-Worker Applications on the Computational Grid , 2000, GRID.

[51]  Sudipto Guha,et al.  Message Multicasting in Heterogeneous Networks , 2000, SIAM J. Comput..

[52]  Jack J. Dongarra,et al.  Automatically Tuned Linear Algebra Software , 1998, Proceedings of the IEEE/ACM SC98 Conference.

[53]  Ian T. Foster,et al.  Condor-G: A Computation Management Agent for Multi-Institutional Grids , 2004, Cluster Computing.

[54]  Susanne Albers,et al.  Online algorithms: a survey , 2003, Math. Program..

[55]  Francine Berman,et al.  Master/slave computing on the Grid , 2000, Proceedings 9th Heterogeneous Computing Workshop (HCW 2000) (Cat. No.PR00556).

[56]  Yves Robert,et al.  A realistic model and an efficient heuristic for scheduling with heterogeneous processors , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[57]  Leonel Sousa,et al.  Communication contention in task scheduling , 2005, IEEE Transactions on Parallel and Distributed Systems.

[58]  Peter Brucker,et al.  Scheduling Algorithms , 1995 .

[59]  Oliver Sinnen,et al.  Task Scheduling for Parallel Systems , 2007, Wiley series on parallel and distributed computing.

[60]  R. F. Freund,et al.  Dynamic matching and scheduling of a class of independent tasks onto heterogeneous computing systems , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[61]  Yves Robert,et al.  Steady-state scheduling on heterogeneous clusters , 2005, Int. J. Found. Comput. Sci..

[62]  X. Hu,et al.  Energy efficient fixed-priority scheduling for real-time systems on variable voltage processors , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[63]  David B. Skillicorn,et al.  Strategies for parallel data mining , 1999, IEEE Concurr..

[64]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[65]  Michael G. Norman,et al.  Models of machines and computation for mapping in multicomputers , 1993, CSUR.

[66]  Henri Casanova,et al.  Network modeling issues for grid application scheduling , 2005, Int. J. Found. Comput. Sci..

[67]  Joseph Y.-T. Leung,et al.  Minimizing sum of completion times and makespan in master-slave systems , 2006, IEEE Transactions on Computers.

[68]  David P. Bunde.  Power-aware scheduling for makespan and flow , 2006, SPAA '06.

[69]  E.L. Lawler,et al.  Optimization and Approximation in Deterministic Sequencing and Scheduling: a Survey , 1977 .

[70]  Wahid Nasri,et al.  A Poly-Algorithm for Efficient Parallel Matrix Multiplication on Metacomputing Platforms , 2005, 2005 IEEE International Conference on Cluster Computing.

[71]  Yves Robert,et al.  Scheduling and Automatic Parallelization , 2000, Birkhäuser Boston.

[72]  Yves Robert,et al.  A Proposal for a Heterogeneous Cluster ScaLAPACK (Dense Linear Solvers) , 2001, IEEE Trans. Computers.

[73]  Rajmohan Rajaraman,et al.  Approximation Algorithms for Average Stretch Scheduling , 2004, J. Sched..

[74]  Hiroto Yasuura,et al.  Voltage scheduling problem for dynamically variable voltage processors , 1998, Proceedings. 1998 International Symposium on Low Power Electronics and Design (IEEE Cat. No.98TH8379).

[75]  Yan Alexander Li,et al.  Minimizing the Application Execution Time Through Scheduling of Subtasks and Communication Traffic in a Heterogeneous Computing System , 1997, IEEE Trans. Parallel Distributed Syst..

[76]  Manish Parashar,et al.  Understanding the Behavior and Performance of Non-blocking Communications in MPI , 2004, Euro-Par.

[77]  Jack J. Dongarra,et al.  Automated empirical optimizations of software and the ATLAS project , 2001, Parallel Comput..

[78]  Denis Trystram,et al.  A polyalgorithmic approach applied for fast matrix multiplication on clusters , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[79]  James Demmel,et al.  ScaLAPACK: A Portable Linear Algebra Library for Distributed Memory Computers - Design Issues and Performance , 1995, PARA.

[80]  Dhabaleswar K. Panda,et al.  Communication modeling of heterogeneous networks of workstations for performance characterization of collective operations , 1999, Proceedings. Eighth Heterogeneous Computing Workshop (HCW'99).

[81]  Yves Robert,et al.  Matrix Multiplication on Heterogeneous Platforms , 2001, IEEE Trans. Parallel Distributed Syst..

[82]  Yves Robert,et al.  Independent and divisible tasks scheduling on heterogeneous star-shaped platforms with limited memory , 2005, 13th Euromicro Conference on Parallel, Distributed and Network-Based Processing.

[83]  Yves Robert,et al.  A Polynomial-Time Algorithm for Allocating Independent Tasks on Heterogeneous Fork-Graphs , 2002 .

[84]  Larry Carter,et al.  Centralized versus distributed schedulers for multiple bag-of-task applications , 2006, Proceedings 20th IEEE International Parallel & Distributed Processing Symposium.

[85]  Jiri Sgall,et al.  On-line scheduling --- a survey , 1997 .

[86]  Rami G. Melhem,et al.  Determining optimal processor speeds for periodic real-time tasks with different power characteristics , 2001, Proceedings 13th Euromicro Conference on Real-Time Systems.

[87]  Mario Lauria,et al.  Self-Organizing Scheduling on the Organic Grid , 2006, Int. J. High Perform. Comput. Appl..

[88]  Ali R. Hurson,et al.  Scheduling and Load Balancing in Parallel and Distributed Systems , 1995 .

[89]  Kevin Skadron,et al.  Temperature-aware microarchitecture: Modeling and implementation , 2004, TACO.

[90]  Fernando Guirado,et al.  Improving static scheduling using inter-task concurrency measures , 2001, Proceedings International Conference on Parallel Processing Workshops.

[91]  Henri Casanova,et al.  Practical divisible load scheduling on grid platforms with APST-DV , 2005, 19th IEEE International Parallel and Distributed Processing Symposium.

[92]  Debasish Ghose,et al.  Multi-installment load distribution in tree networks with delays , 1995 .

[93]  Yves Robert,et al.  Offline and Online Scheduling of Concurrent Bags-of-Tasks on Heterogeneous Platforms , 2007 .

[94]  Radu Marculescu,et al.  Communication-Aware Task Scheduling and Voltage Selection for Total Systems Energy Minimization , 2003, ICCAD 2003.

[95]  Leonel Sousa,et al.  Experimental Evaluation of Task Scheduling Accuracy: Implications for the Scheduling Model , 2003 .

[96]  David S. Johnson,et al.  Computers and In stractability: A Guide to the Theory of NP-Completeness. W. H Freeman, San Fran , 1979 .

[97]  Pierre-François Dutot,et al.  Complexity of master-slave tasking on heterogeneous trees , 2005, Eur. J. Oper. Res..

[98]  Rong Ge,et al.  Performance-constrained Distributed DVS Scheduling for Scientific Applications on Power-aware Clusters , 2005, ACM/IEEE SC 2005 Conference (SC'05).

[99]  Larry Carter,et al.  Scheduling strategies for master-slave tasking on heterogeneous processor platforms , 2004, IEEE Transactions on Parallel and Distributed Systems.

[100]  Jack Dongarra,et al.  PVM: Parallel virtual machine: a users' guide and tutorial for networked parallel computing , 1995 .

[101]  Ian T. Foster,et al.  MPICH-G2: A Grid-enabled implementation of the Message Passing Interface , 2002, J. Parallel Distributed Comput..

[102]  Prudence W. H. Wong,et al.  Energy efficient online deadline scheduling , 2007, SODA '07.

[103]  Frédéric Vivien,et al.  Minimizing the stretch when scheduling flows of biological requests , 2006, SPAA '06.

[104]  Zizhong Chen,et al.  Self-adapting software for numerical linear algebra and LAPACK for clusters , 2003, Parallel Comput..

[105]  Z Liu,et al.  Scheduling Theory and its Applications , 1997 .

[106]  Tao Yang,et al.  DSC: Scheduling Parallel Tasks on an Unbounded Number of Processors , 1994, IEEE Trans. Parallel Distributed Syst..

[107]  Yves Robert,et al.  The impact of heterogeneity on master-slave scheduling , 2008, Parallel Comput..

[108]  Kirk Pruhs,et al.  Dynamic speed scaling to manage energy and temperature , 2004, 45th Annual IEEE Symposium on Foundations of Computer Science.

[109]  Larry Carter,et al.  Dynamic autonomous scheduling on heterogeneous systems , 2003 .

[110]  Adi Shamir,et al.  A method for obtaining digital signatures and public-key cryptosystems , 1978, CACM.

[111]  Susanne Albers,et al.  On randomized online scheduling , 2002, STOC '02.

[112]  Arjen K. Lenstra,et al.  A World Wide Number Field Sieve Factoring Record: On to 512 Bits , 1996, ASIACRYPT.

[113]  A. J. Clewett,et al.  Introduction to sequencing and scheduling , 1974 .

[114]  Anthony Skjellum,et al.  A High-Performance, Portable Implementation of the MPI Message Passing Interface Standard , 1996, Parallel Comput..

[115]  Simone Santini.  We Are Sorry to Inform You , 2005, Computer.

[116]  Viktor K. Prasanna,et al.  Distributed adaptive task allocation in heterogeneous computing environments to maximize throughput , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[117]  Tsan-sheng Hsu,et al.  Task Allocation on a Network of Processors , 2000, IEEE Trans. Computers.

[118]  Alexey L. Lastovetsky,et al.  Data partitioning with a realistic performance model of networks of heterogeneous computers , 2004, 18th International Parallel and Distributed Processing Symposium, 2004. Proceedings..

[119]  Sivan Toledo,et al.  A survey of out-of-core algorithms in numerical linear algebra , 1999, External Memory Algorithms.

[120]  William Gropp,et al.  MPICH2: A New Start for MPI Implementations , 2002, PVM/MPI.

[121]  Frédéric Vivien,et al.  Minimizing the stretch when scheduling flows of divisible requests , 2008, J. Sched..

[122]  Francine Berman,et al.  Adaptive scheduling of master/worker applications on distributed computational resources , 2001 .

[123]  Michael A. Bender,et al.  Flow and stretch metrics for scheduling continuous job streams , 1998, SODA '98.

[124]  Mohammed J. Zaki,et al.  Compile-Time Scheduling Algorithms for a Heterogeneous Network of Workstations , 1997, Comput. J..

[125]  Yves Robert,et al.  Mapping and load-balancing iterative computations , 2004, IEEE Transactions on Parallel and Distributed Systems.

[126]  Miron Livny,et al.  Condor-a hunter of idle workstations , 1988, [1988] Proceedings. The 8th International Conference on Distributed.

[127]  Barbara B. Simons,et al.  Multiprocessor Scheduling of Unit-Time Jobs with Arbitrary Release Times and Deadlines , 1983, SIAM J. Comput..

[128]  Francine Berman,et al.  Adaptive Computing on the Grid Using AppLeS , 2003, IEEE Trans. Parallel Distributed Syst..

[129]  Viktor K. Prasanna,et al.  Scalable and Modular Algorithms for Floating-Point Matrix Multiplication on Reconfigurable Computing Systems , 2007, IEEE Transactions on Parallel and Distributed Systems.

[130]  Larry Carter,et al.  Bandwidth-centric allocation of independent tasks on heterogeneous platforms , 2002, Proceedings 16th International Parallel and Distributed Processing Symposium.

[131]  Pangfeng Liu.  Broadcast Scheduling Optimization for Heterogeneous Cluster Systems , 2002, J. Algorithms.

[132]  Srinivasan Parthasarathy,et al.  Customized dynamic load balancing for a network of workstations , 1996, Proceedings of 5th IEEE International Symposium on High Performance Distributed Computing.

[133]  Dror Irony,et al.  Communication lower bounds for distributed-memory matrix multiplication , 2004, J. Parallel Distributed Comput..

[134]  David Gamarnik,et al.  Asymptotically Optimal Algorithms for Job Shop Scheduling and Packet Routing , 1999, J. Algorithms.

[135]  Yves Robert,et al.  Partitioning a Square into Rectangles: NP-Completeness and Approximation Algorithms , 2002, Algorithmica.

[136]  Francine Berman,et al.  High-performance schedulers , 1998 .

[137]  Arnold L. Rosenberg,et al.  Optimal sharing of bags of tasks in heterogeneous clusters , 2003, SPAA '03.

[138]  Sandeep K. S. Gupta,et al.  Thermal-Aware Task Scheduling to Minimize Energy Usage of Blade Server Based Datacenters , 2006, 2006 2nd IEEE International Symposium on Dependable, Autonomic and Secure Computing.