Graph Partitioning Algorithms with Applications to Scientific Computing

Identifying the parallelism in a problem by partitioning its data and tasks among the processors of a parallel computer is a fundamental issue in parallel computing. This problem can be modeled as a graph partitioning problem in which the vertices of a graph are divided into a specified number of subsets such that few edges join two vertices in different subsets. Several new graph partitioning algorithms have been developed in the past few years, and we survey some of this activity. We describe the terminology associated with graph partitioning, the complexity of computing good separators, and graphs that have good separators. We then discuss early algorithms for graph partitioning, followed by three new algorithms based on geometric, algebraic, and multilevel ideas. The algebraic algorithm relies on an eigenvector of a Laplacian matrix associated with the graph to compute the partition. The algebraic algorithm is justified by formulating graph partitioning as a quadratic assignment problem. We list several papers that describe applications of graph partitioning to parallel scientific computing and other applications.

[1]  L. Dagum.  Automatic partitioning of unstructured grids into connected components , 1993, Supercomputing '93.

[2]  Charbel Farhat,et al.  Implicit parallel processing in structural mechanics , 1994 .

[3]  J. G. Lewis,et al.  A fast algorithm for reordering sparse matrices for parallel factorization , 1989 .

[4]  Bruce Hendrickson,et al.  The Chaco user`s guide. Version 1.0 , 1993 .

[5]  William Gropp,et al.  Domain Decomposition: Parallel Multilevel Algorithms for Elliptic Partial Di erential Equations , 1995 .

[6]  Lie Wang,et al.  Parallel Sparse Cholesky Factorization with Spectral Nested Dissection Ordering. Chapter 1 , 1994 .

[7]  Joseph W. H. Liu.  The Minimum Degree Ordering with Constraints , 1989 .

[8]  A. K. Norr,et al.  Adaptive, Multilevel, and Hierarchical Computational Strategies , 1992 .

[9]  D. Rose,et al.  Generalized nested dissection , 1977 .

[10]  Bruce Hendrickson,et al.  Improving the Run Time and Quality of Nested Dissection Ordering , 1998, SIAM J. Sci. Comput..

[11]  Roy D. Williams,et al.  Performance of dynamic load balancing algorithms for unstructured mesh calculations , 1991, Concurr. Pract. Exp..

[12]  Bruce Hendrickson,et al.  A Multi-Level Algorithm For Partitioning Graphs , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[13]  Geoffrey C. Fox,et al.  Performance evaluation of load balancing algorithms for parallel single-phase iterative PDE solvers , 1994, Proceedings of IEEE Scalable High Performance Computing Conference.

[14]  Curt Jones.  Vertex and edge partitions of graphs , 1992 .

[15]  J. Périaux,et al.  Domain Decomposition Methods in Science and Engineering , 1994 .

[16]  Martin Berzins,et al.  Dynamic load-balancing for PDE solvers on adaptive unstructured meshes , 1995, Concurr. Pract. Exp..

[17]  Andrew B. Kahng,et al.  Recent directions in netlist partitioning: a survey , 1995, Integr..

[18]  Jochem Zowe,et al.  A Version of the Bundle Idea for Minimizing a Nonsmooth Function: Conceptual Idea, Convergence Analysis, Numerical Results , 1992, SIAM J. Optim..

[19]  Joseph W. H. Liu,et al.  Robust Ordering of Sparse Matrices using Multisection , 1998 .

[20]  B. Mohar,et al.  Eigenvalues in Combinatorial Optimization , 1993 .

[21]  Alex Pothen,et al.  Computing the block triangular form of a sparse matrix , 1990, TOMS.

[22]  Can Ozturan.  Distributed environment and load balancing for adaptive unstructured meshes , 1996 .

[23]  Philip N. Klein,et al.  Cutting down on Fill Using Nested Dissection: Provably Good Elimination Orderings , 1993 .

[24]  Noga Alon,et al.  lambda1, Isoperimetric inequalities for graphs, and superconcentrators , 1985, J. Comb. Theory, Ser. B.

[25]  R. M. Mattheyses,et al.  A Linear-Time Heuristic for Improving Network Partitions , 1982, 19th Design Automation Conference.

[26]  A. George,et al.  An Analysis of Spectral Envelope Reduction via Quadratic Assignment Problems , 1997, SIAM J. Matrix Anal. Appl..

[27]  Carlo L. Bottasso,et al.  Parallel automated adaptive procedures for unstructured meshes , 1995 .

[28]  Dirk Roose,et al.  Distributed memory parallel computers and computational fluid dynamics , 1993 .

[29]  J. Tinsley Oden,et al.  Problem decomposition for adaptive hp finite element methods , 1995 .

[30]  Joseph W. H. Liu,et al.  Modification of the minimum-degree algorithm by multiple elimination , 1985, TOMS.

[31]  Vipin Kumar,et al.  A Fast and High Quality Multilevel Scheme for Partitioning Irregular Graphs , 1998, SIAM J. Sci. Comput..

[32]  Michael T. Heath,et al.  Parallel Algorithms for Sparse Linear Systems , 1991, SIAM Rev..

[33]  Alex Pothen,et al.  A spectral algorithm for envelope reduction of sparse matrices , 1993, Supercomputing '93. Proceedings.

[34]  Byung Ro Moon,et al.  A Genetic Algorithm for a Special Class of the Quadratic Assignment Problem , 1993, Quadratic Assignment and Related Problems.

[35]  T. Chan,et al.  Domain decomposition algorithms , 1994, Acta Numerica.

[36]  Chandrajit L. Bajaj,et al.  NURBS approximation of surface/surface intersection curves , 1994, Adv. Comput. Math..

[37]  Vipin Kumar,et al.  Analysis of Multilevel Graph Partitioning , 1995, Proceedings of the IEEE/ACM SC95 Conference.

[38]  Gary L. Miller,et al.  Automatic Mesh Partitioning , 1992 .

[39]  J. Gilbert.  Graph separator theorems and sparse Gaussian elimination , 1980 .

[40]  Steven Warren Hammond,et al.  Mapping unstructured grid computations to massively parallel computers , 1992 .

[41]  G. Kedem,et al.  An algorithm for quadrisection and its application to standard cell placement , 1988 .

[42]  K. S. Arun,et al.  Constructive heuristics and lower bounds for graph partitioning based on a principal-components approximation , 1993 .

[43]  Dirk Roose,et al.  A Grapg Contraction Algorithm for the Fast Calculation of the Fiedler Vector of a Graph , 1995, PPSC.

[44]  Horst D. Simon,et al.  Fast multilevel implementation of recursive spectral bisection for partitioning unstructured problems , 1994, Concurr. Pract. Exp..

[45]  A. George,et al.  Graph theory and sparse matrix computation , 1993 .

[46]  Bruce Hendrickson,et al.  An Improved Spectral Graph Partitioning Algorithm for Mapping Parallel Computations , 1995, SIAM J. Sci. Comput..

[47]  Ravi B. Boppana,et al.  Eigenvalues and graph bisection: An average-case analysis , 1987, 28th Annual Symposium on Foundations of Computer Science (sfcs 1987).

[48]  Curt Jones,et al.  Finding Good Approximate Vertex and Edge Partitions is NP-Hard , 1992, Inf. Process. Lett..

[49]  Cecilia R. Aragon,et al.  Optimization by Simulated Annealing: An Experimental Evaluation; Part I, Graph Partitioning , 1989, Oper. Res..

[50]  David E. Keyes,et al.  Domain Decomposition: A Bridge between Nature and Parallel Computers , 1992 .

[51]  Franz Rendl,et al.  A computational study of graph partitioning , 1994, Math. Program..

[52]  S. Teng,et al.  A Cartesian Parallel Nested Dissection Algorithm , 1994 .

[53]  B. Nour-Omid,et al.  Solving finite element equations on concurrent computers , 1987 .

[54]  Brian W. Kernighan,et al.  An efficient heuristic procedure for partitioning graphs , 1970, Bell Syst. Tech. J..

[55]  E. Barnes.  An algorithm for partitioning the nodes of a graph , 1981, 1981 20th IEEE Conference on Decision and Control including the Symposium on Adaptive Processes.

[56]  M. Fiedler.  A property of eigenvectors of nonnegative symmetric matrices and its application to graph theory , 1975 .

[57]  A. George.  Nested Dissection of a Regular Finite Element Mesh , 1973 .

[58]  Bob Francis,et al.  Silicon Graphics Inc. , 1993 .

[59]  Catherine A. Schevon,et al.  Optimization by simulated annealing: An experimental evaluation , 1984 .

[60]  Curt Jones,et al.  A Heuristic for Reducing Fill-In in Sparse Matrix Factorization , 1993, PPSC.

[61]  Lie Wang,et al.  Towards a fast implementation of spectral nested dissection , 1992, Proceedings Supercomputing '92.

[62]  Barry W. Peyton,et al.  Toward a Parallel Recursive Spectral Bisection Mapping Tool , 1993, PPSC.

[63]  Peter K. Jimack,et al.  Efficient Mesh Partitioning for Parallel P.D.E. Solvers on Distributed Memory Machines , 1993, PPSC.

[64]  J. Gilbert,et al.  Graph Coloring Using Eigenvalue Decomposition , 1983 .

[65]  H. D. Simon,et al.  A spectral algorithm for envelope reduction of sparse matrices , 1993, Supercomputing '93. Proceedings.

[66]  Joseph W. H. Liu,et al.  Using domain decomposition to find graph bisectors , 1997 .

[67]  Vipin Kumar,et al.  Parallel multilevel graph partitioning , 1996, Proceedings of International Conference on Parallel Processing.

[68]  Frank Thomson Leighton,et al.  An approximate max-flow min-cut theorem for uniform multicommodity flow problems with applications to approximation algorithms , 1988, [Proceedings 1988] 29th Annual Symposium on Foundations of Computer Science.

[69]  Kenneth M. Hall.  An r-Dimensional Quadratic Placement Algorithm , 1970 .

[70]  John R. Rice,et al.  Mapping Algorithms and Software Environment for Data Parallel PDE Iterative Solvers , 1994, J. Parallel Distributed Comput..

[71]  I. Duff,et al.  Direct Methods for Sparse Matrices , 1987 .

[72]  Dirk Roose,et al.  An efficient spectral bisection algorithm for dynamic load balancing , 1994 .

[73]  Thomas J. R. Hughes,et al.  Parallel implementation of recursive spectral bisection on the Connection Machine CM-5 system , 1995 .

[74]  O. Widlund.  Domain Decomposition Algorithms , 1993 .

[75]  Andrew B. Kahng,et al.  New spectral methods for ratio cut partitioning and clustering , 1991, IEEE Trans. Comput. Aided Des. Integr. Circuits Syst..

[76]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[77]  J. Pasciak,et al.  Computer solution of large sparse positive definite systems , 1982 .

[78]  Shahid H. Bokhari,et al.  A Partitioning Strategy for Nonuniform Problems on Multiprocessors , 1987, IEEE Transactions on Computers.

[79]  B. Parlett.  The Symmetric Eigenvalue Problem , 1981 .

[80]  Horst D. Simon,et al.  Partitioning of unstructured problems for parallel processing , 1991 .

[81]  Charles M. Fiduccia,et al.  A linear-time heuristic for improving network partitions , 1988, 25 years of DAC.

[82]  Franz Rendl,et al.  A projection technique for partitioning the nodes of a graph , 1995, Ann. Oper. Res..

[83]  Michael Doob,et al.  Spectra of graphs , 1980 .

[84]  Joseph W. H. Liu.  The role of elimination trees in sparse factorization , 1990 .

[85]  Joel H. Saltz,et al.  ICASE Report No . 92-12 / iVG / / ff 3 J / ICASE THE DESIGN AND IMPLEMENTATION OF A PARALLEL UNSTRUCTURED EULER SOLVER USING SOFTWARE PRIMITIVES , 2022 .

[86]  Zdeněk Johan,et al.  Data parallel finite element techniques for large-scale computational fluid dynamics , 1992 .

[87]  Gary L. Miller,et al.  Geometric mesh partitioning: implementation and experiments , 1995, Proceedings of 9th International Parallel Processing Symposium.

[88]  A. Hoffman,et al.  Lower bounds for the partitioning of graphs , 1973 .

[89]  P. Tallec.  Domain decomposition methods in computational mechanics , 1994 .

[90]  R. Tarjan,et al.  A Separator Theorem for Planar Graphs , 1977 .

[91]  Gary L. Miller,et al.  On the performance of spectral graph partitioning methods , 1995, SODA '95.

[92]  Shang-Hua Teng,et al.  Spectral partitioning works: planar graphs and finite element meshes , 1996, Proceedings of 37th Conference on Foundations of Computer Science.

[93]  Stéphane Lanteri,et al.  TOP/DOMDEC : a software tool for mesh partitioning and parallel processing and applications to CSM a , 1995 .

[94]  Barry F. Smith,et al.  Domain Decomposition: Parallel Multilevel Methods for Elliptic Partial Differential Equations , 1996 .

[95]  M. Fiedler.  Algebraic connectivity of graphs , 1973 .

[96]  Alex Pothen,et al.  PARTITIONING SPARSE MATRICES WITH EIGENVECTORS OF GRAPHS* , 1990 .

[97]  Alan George,et al.  The Evolution of the Minimum Degree Ordering Algorithm , 1989, SIAM Rev..

[98]  B. Mohar.  THE LAPLACIAN SPECTRUM OF GRAPHS y , 1991 .

[99]  John G. Lewis,et al.  Sparse matrix test problems , 1982, SGNM.

[100]  John F. Abel,et al.  Recursive spectral algorithms for automatic domain partitioning in parallel finite element analysis , 1995 .

[101]  N. Alon,et al.  il , , lsoperimetric Inequalities for Graphs , and Superconcentrators , 1985 .