Practical performance model for optimizing dynamic load balancing of adaptive applications

Optimizing the performance of dynamic load balancing toolkits and applications requires the adjustment of several runtime parameters; however, determining sufficiently good values for these parameters through repeated experimentation can be an expensive and prohibitive process. We describe an analytic modeling method which allows developers to study and optimize adaptive application performance in the presence of dynamic load balancing. To aid tractibility, we first derive a "bi-modal" step function which simplifies and approximates task execution behavior. This allows for the creation of an analytic modeling function which captures the dynamic behavior of adaptive and asynchronous applications, enabling accurate predictions of runtime performance. We validate our technique using synthetic micro-benchmarks and a parallel mesh generation application and demonstrate that this technique, when used in conjunction with the PREMA runtime toolkit, can offer users significant performance improvements over several well-known load balancing tools used in practice today.

[1]  W. Gautschi.  Numerical analysis: an introduction , 1997 .

[2]  Laxmikant V. Kalé,et al.  Run-Time Support for Adaptive Load Balancing , 2000, IPDPS Workshops.

[3]  Jörn W. Janneck,et al.  Applying an Object-Oriented Petri Net Language to Heterogeneous Systems Design , 2001 .

[4]  H. Wasserman,et al.  Modeling the Performance of Large-Scale Systems , 2003 .

[5]  Michael Mitzenmacher,et al.  On the Analysis of Randomized Load Balancing Schemes , 1997, SPAA '97.

[6]  Kevin J. Barker,et al.  An Evaluation of a Framework for the Dynamic Load Balancing of Highly Adaptive and Irregular Parallel Applications , 2003, ACM/IEEE SC 2003 Conference (SC'03).

[7]  M. V Mederos,et al.  Gautschi, Walter. Numerical analysis: an introduction, Birkhäuser, 1997 , 1999 .

[8]  Francis C. M. Lau,et al.  An analytical comparison of nearest neighbor algorithms for load balancing in parallel computers , 1995, Proceedings of 9th International Parallel Processing Symposium.

[9]  Nikos Chrisochoides,et al.  Parallel Programming Environment for Mesh Generation † , 2002 .

[10]  Laxmikant V. Kalé,et al.  Load Balancing in Parallel Molecular Dynamics , 1998, IRREGULAR.

[11]  Laura Carrington,et al.  Modeling application performance by convolving machine signatures with application profiles , 2001 .

[12]  George Cybenko,et al.  Dynamic Load Balancing for Distributed Memory Multiprocessors , 1989, J. Parallel Distributed Comput..

[13]  Kevin J. Barker,et al.  Runtime support for load balancing of parallel adaptive and irregular applications , 2004 .

[14]  Keshav Pingali,et al.  A load balancing framework for adaptive and asynchronous applications , 2004, IEEE Transactions on Parallel and Distributed Systems.

[15]  Adolfy Hoisie,et al.  Verifying large-scale system performance during installation using modelling , 2004 .

[16]  Adolfy Hoisie,et al.  Use of Predictive Performance Modeling during Large-scale System Installation , 2005, Parallel Process. Lett..

[17]  L. Paul Chew,et al.  Parallel Constrained Delaunay Meshing , 2007 .

[18]  Leslie G. Valiant,et al.  Bulk synchronous parallel computing-a paradigm for transportable software , 1995, Proceedings of the Twenty-Eighth Annual Hawaii International Conference on System Sciences.

[19]  George Karypis,et al.  Parmetis parallel graph partitioning and sparse matrix ordering library , 1997 .

[20]  Arif Ghafoor,et al.  Performance modeling of load-balancing algorithms using neural networks , 1994, Concurr. Pract. Exp..

[21]  Adolfy Hoisie,et al.  Modelling the performance of large-scale systems , 2003, IEE Proc. Softw..

[22]  Bruce M. Maggs,et al.  Tight analyses of two local load balancing algorithms , 1995, STOC '95.

[23]  Yong Luo,et al.  Poems: end-to-end performance design of large parallel adaptive computational systems , 1998, WOSP '98.

[24]  Rahul Simha,et al.  Dynamic versus Static Load Balancing in a Pipeline Computation , 1997 .

[25]  Lei Wang,et al.  Modeling Dynamic Load Balancing in Molecular Dynamics to Achieve Scalable Parallel Execution , 1998, IRREGULAR.