LDRD Final Report on Massively-Parallel Linear Programming: the parPCx System

This report summarizes the research and development performed from October 2002 to September 2004 at Sandia National Laboratories under the Laboratory-Directed Research and Development (LDRD) project ''Massively-Parallel Linear Programming''. We developed a linear programming (LP) solver designed to use a large number of processors. LP is the optimization of a linear objective function subject to linear constraints. Companies and universities have expended huge efforts over decades to produce fast, stable serial LP solvers. Previous parallel codes run on shared-memory systems and have little or no distribution of the constraint matrix. We have seen no reports of general LP solver runs on large numbers of processors. Our parallel LP code is based on an efficient serial implementation of Mehrotra's interior-point predictor-corrector algorithm (PCx). The computational core of this algorithm is the assembly and solution of a sparse linear system. We have substantially rewritten the PCx code and based it on Trilinos, the parallel linear algebra library developed at Sandia. Our interior-point method can use either direct or iterative solvers for the linear system. To achieve a good parallel data distribution of the constraint matrix, we use a (pre-release) version of a hypergraph partitioner from the Zoltan partitioning library. We describe the design and implementation of our new LP solver called parPCx and give preliminary computational results. We summarize a number of issues related to efficient parallel solution of LPs with interior-point methods including data distribution, numerical stability, and solving the core linear system using both direct and iterative methods. We describe a number of applications of LP specific to US Department of Energy mission areas and we summarize our efforts to integrate parPCx (and parallel LP solvers in general) into Sandia's massively-parallel integer programming solver PICO (Parallel Interger and Combinatorial Optimizer). We conclude with directions for long-term future algorithmic research and for near-term development that could improve the performance of parPCx.

[1]  Philip E. Gill,et al.  Practical optimization , 1981 .

[2]  Mauricio G. C. Resende,et al.  An Efficient Implementation of a Network Interior Point Method , 1991, Network Flows And Matching.

[3]  Bruce Hendrickson,et al.  Support Theory for Preconditioning , 2003, SIAM J. Matrix Anal. Appl..

[4]  Weichung Wang,et al.  Adaptive use of iterative methods in interior point methods for linear programming , 1995 .

[5]  Shang-Hua Teng,et al.  Solving Sparse, Symmetric, Diagonally-Dominant Linear Systems in Time O(m1.31) , 2003, ArXiv.

[6]  L. D. J. C. Loyens,et al.  A parallel interior point algorithm for linear programming on a network of transputers , 1993, Ann. Oper. Res..

[7]  Courtenay T. Vaughan,et al.  Zoltan data management services for parallel dynamic applications , 2002, Comput. Sci. Eng..

[8]  Cynthia A. Phillips,et al.  Sensor Placement in Municipal Water Networks , 2003 .

[9]  Robert R. Meyer,et al.  An Interior Point Method for Block Angular Optimization , 1991, SIAM J. Optim..

[10]  A. Schrijver.  Polyhedral combinatorics , 1996 .

[11]  Takashi Tsuchiya,et al.  Uniform Boundedness of a Preconditioned Normal Matrix Used in Interior-Point Methods , 2004, SIAM J. Optim..

[12]  MICHAEL D. GRIGORIADIS,et al.  An Interior Point Method for Bordered Block-Diagonal Linear Programs , 1996, SIAM J. Optim..

[13]  Michael A. Saunders,et al.  Primal—dual methods for linear programming , 1995, Math. Program..

[14]  Jacek Gondzio,et al.  Parallel interior-point solver for structured linear programs , 2003, Math. Program..

[15]  Cynthia A. Phillips,et al.  Pico: An Object-Oriented Framework for Parallel Branch and Bound * , 2001 .

[16]  M. Saunders,et al.  SOLVING REGULARIZED LINEAR PROGRAMS USING BARRIER METHODS AND KKT SYSTEMS , 1996 .

[17]  Cynthia A. Phillips,et al.  A General Integer-Programming-Based Framework for Sensor Placement in Municipal Water Networks , 2004 .

[18]  Alex Pothen,et al.  A Scalable Parallel Algorithm for Incomplete Factor Preconditioning , 2000, SIAM J. Sci. Comput..

[19]  Harvey J. Greenberg,et al.  A Multiple-Objective Analysis of Sensor Placement Optimization in Water Networks , 2004 .

[20]  Shang-Hua Teng,et al.  Nearly-linear time algorithms for graph partitioning, graph sparsification, and solving linear systems , 2003, STOC '04.

[21]  Stephen J. Wright,et al.  pPCx: Parallel Software for Linear Programming , 1997, PPSC.

[22]  Margaret H. Wright,et al.  Interior methods for constrained optimization , 1992, Acta Numerica.

[23]  Mark Frederick Hoemmen,et al.  An Overview of Trilinos , 2003 .

[24]  Brendan Vastenhouw,et al.  A Two-Dimensional Data Distribution Method for Parallel Sparse Matrix-Vector Multiplication , 2005, SIAM Rev..

[25]  J. Navarro-Pedreño.  Numerical Methods for Least Squares Problems , 1996 .

[26]  W. R. Pulleyblank,et al.  Polyhedral Combinatorics , 1989, ISMP.

[27]  Mark A. Turnquist,et al.  Expected losses, insurability, and benefits from reducing vulnerability to attacks. , 2004 .

[28]  Jonathan Eckstein,et al.  Parallel Branch-and-Bound Algorithms for General Mixed Integer Programming on the CM-5 , 1994, SIAM J. Optim..

[29]  Ümit V. Çatalyürek,et al.  Hypergraph-Partitioning-Based Decomposition for Parallel Sparse-Matrix Vector Multiplication , 1999, IEEE Trans. Parallel Distributed Syst..

[30]  Sivan Toledo,et al.  Maximum‐weight‐basis preconditioners , 2004, Numer. Linear Algebra Appl..

[31]  Joaquim Júdice,et al.  A Study of Preconditioners for Network Interior Point Methods , 2003, Comput. Optim. Appl..

[32]  D. Chen,et al.  VAIDYA'S PRECONDITIONERS: IMPLEMENTATION AND EXPERIMENTAL STUDY , 2003 .

[33]  M. Smelyanskiy,et al.  Construction and performance characterization of parallel interior point solver on 4-way Intel Itanium 2 multiprocessor system , 2004, IEEE International Workshop on Workload Characterization, 2004. WWC-7. 2004.

[34]  Shang-Hua Teng,et al.  Smoothed analysis of algorithms: why the simplex algorithm usually takes polynomial time , 2001, STOC '01.

[35]  Michael A. Saunders,et al.  Preconditioners for Indefinite Systems Arising in Optimization , 1992, SIAM J. Matrix Anal. Appl..

[36]  Venansius Baryamureebay,et al.  Properties and Computational Issues of aPreconditioner for Interior Point Methods , 1999 .

[37]  Shang-Hua Teng,et al.  Solving sparse, symmetric, diagonally-dominant linear systems in time O(m/sup 1.31/ , 2003, 44th Annual IEEE Symposium on Foundations of Computer Science, 2003. Proceedings..

[38]  Stephen J. Wright.  Primal-Dual Interior-Point Methods , 1997, Other Titles in Applied Mathematics.