Parallel Solvers for Mixed Integer Linear Optimization

In this chapter, we provide an overview of the current state of the art with respect to solution of mixed integer linear optimization problems (MILPs) in parallel. Sequential algorithms for solving MILPs have improved substantially in the last two decades and commercial MILP solvers are now considered effective off-the-shelf tools for optimization. Although concerted development of parallel MILP solvers has been underway since the 1990s, the impact of improvements in sequential solution algorithms has been much greater than that which came from the application of parallel computing technologies. As a result, parallelization efforts have met with only relatively modest success. In addition, improvements to the underlying sequential solution technologies have actually been somewhat detrimental with respect to the goal of creating scalable parallel algorithms. This has made efforts at parallelization an even greater challenge in recent years. With the pervasiveness of multi-core CPUs, current state-of-the-art MILP solvers have now all been parallelized and research on parallelization is once again gaining traction. We summarize the current state-of-the-art and describe how existing parallel MILP solvers can be classified according to various properties of the underlying algorithm.

[1]  Robert E. Bixby,et al.  Presolve Reductions in Mixed Integer Programming , 2020, INFORMS J. Comput..

[2]  Robin Lougee,et al.  The Common Optimization INterface for Operations Research: Promoting open-source software in the operations research community , 2003, IBM J. Res. Dev..

[3]  J. Pekny,et al.  Results from a parallel branch and bound algorithm for the asymmetric traveling salesman problem , 1989 .

[4]  Peter Sanders,et al.  A detailed analysis of random polling dynamic load balancing , 1994, Proceedings of the International Symposium on Parallel Architectures, Algorithms and Networks (ISPAN).

[5]  Salah Dowaji,et al.  Building a parallel branch and bound library , 1996, Solving Combinatorial Optimization Problems in Parallel.

[6]  Dharma P. Agrawal,et al.  A randomized parallel branch-and-bound algorithm , 1989, International Journal of Parallel Programming.

[7]  Thorsten Koch,et al.  Could we use a million cores to solve an integer program? , 2012, Mathematical Methods of Operations Research.

[8]  Jakob Witzig,et al.  Experiments with Conflict Analysis in Mixed Integer Programming , 2016, CPAIOR.

[9]  Jonathan Eckstein.  Distributed versus Centralized Storage and Control for Parallel Branch and Bound: Mixed Integer Programming on the CM-5 , 1997, Comput. Optim. Appl..

[10]  Tobias Achterberg,et al.  SCIP: solving constraint integer programs , 2009, Math. Program. Comput..

[11]  Matthew J. Saltzman,et al.  Computational Experience with a Software Framework for Parallel Integer Programming , 2009, INFORMS J. Comput..

[12]  Yan Xu,et al.  SCALABLE ALGORITHMS FOR PARALLEL TREE SEARCH , 2008 .

[13]  Arie de Bruin,et al.  Towards a taxonomy of parallel branch and bound algorithms , 1992 .

[14]  Andrea Lodi,et al.  Performance Variability in Mixed-Integer Programming , 2013 .

[15]  Tobias Achterberg,et al.  Mixed Integer Programming: Analyzing 12 Years of Progress , 2013 .

[16]  Yuji Shinano,et al.  FiberSCIP - A Shared Memory Parallelization of SCIP , 2018, INFORMS J. Comput..

[17]  Tobias Achterberg,et al.  Conflict analysis in mixed integer programming , 2007, Discret. Optim..

[18]  T. Ralphs,et al.  COIN/BCP User’s Manual , 2001 .

[19]  R. Lougee-Heimer,et al.  The Common Optimization INterface for Operations Research: Promoting open-source software in the operations research community , 2003 .

[20]  Laxmikant V. Kalé,et al.  A load balancing strategy for prioritized execution of tasks , 1993, [1993] Proceedings Seventh International Parallel Processing Symposium.

[21]  Ralph E. Gomory,et al.  Outline of an Algorithm for Integer Solutions to Linear Programs and An Algorithm for the Mixed Integer Problem , 2010, 50 Years of Integer Programming.

[22]  Stephen L. Olivier,et al.  Dynamic Load Balancing of Unbalanced Computations Using Message Passing , 2007, 2007 IEEE International Parallel and Distributed Processing Symposium.

[23]  Ted K. Ralphs,et al.  Parallel branch and cut for capacitated vehicle routing , 2003, Parallel Comput..

[24]  Qi Huangfu,et al.  Parallelizing the dual revised simplex method , 2015, Mathematical Programming Computation.

[25]  Timo Berthold.  Heuristic algorithms in global MINLP solvers , 2014 .

[26]  J. Wang,et al.  Computational Experience with Generic Decomposition using the DIP Framework , 2012 .

[27]  Martin W. P. Savelsbergh,et al.  A Computational Study of Search Strategies for Mixed Integer Programming , 1999, INFORMS J. Comput..

[28]  Yurii Nesterov,et al.  Interior-point polynomial algorithms in convex programming , 1994, Siam studies in applied mathematics.

[29]  Dominik Henrich,et al.  Initialization of Parallel Branch-and-bound Algorithms , 1993 .

[30]  Emmanuel Jeannot,et al.  Topology and Affinity Aware Hierarchical and Distributed Load-Balancing in Charm++ , 2016, 2016 First International Workshop on Communication Optimizations in HPC (COMHPC).

[31]  Lluís-Miquel Munguía,et al.  PIPS-SBB: A Parallel Distributed-Memory Branch-and-Bound Algorithm for Stochastic Mixed-Integer Programs , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW).

[32]  Ted K. Ralphs,et al.  A Branch-and-cut Algorithm for Integer Bilevel Linear Programs , 2009 .

[33]  Peter Sanders.  Tree Shaped Computations as a Model for Parallel Applications , 1998 .

[34]  Catherine Roucairol,et al.  Bob++: Framework for Solving Optimization Problems with Branch-and-Bound methods , 2006, 2006 15th IEEE International Conference on High Performance Distributed Computing.

[35]  Thorsten Koch,et al.  Solving Open MIP Instances with ParaSCIP on Supercomputers Using up to 80,000 Cores , 2016, 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS).

[36]  A. Mahajan,et al.  Presolving Mixed–Integer Linear Programs , 2011 .

[37]  Ted K. Ralphs,et al.  Parallel Branch and Cut , 2006 .

[38]  Ted K. Ralphs,et al.  Computational Experience with Hypergraph-Based Methods for Automatic Decomposition in Discrete Optimization , 2013, CPAIOR.

[39]  Bernard Gendron,et al.  Parallel Branch-and-Branch Algorithms: Survey and Synthesis , 1994, Oper. Res..

[40]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988, Wiley interscience series in discrete mathematics and optimization.

[41]  Matteo Fischetti,et al.  Self-splitting of Workload in Parallel Computation , 2014, CPAIOR.

[42]  Fernando Gustavo Tinetti,et al.  Parallel programming: techniques and applications using networked workstations and parallel computers. Barry Wilkinson, C. Michael Allen , 2000 .

[43]  Per S. Laursen.  Can Parallel Branch and Bound without Communication Be Effective? , 1994, SIAM J. Optim..

[44]  George B. Dantzig,et al.  Decomposition Principle for Linear Programs , 1960 .

[45]  Marek Olszewski,et al.  Kendo: efficient deterministic multithreading in software , 2009, ASPLOS.

[46]  Andrew J. Schaefer,et al.  Predicting the Solution Time of Branch-and-Bound Algorithms for Mixed-Integer Programs , 2011, INFORMS J. Comput..

[47]  Thorsten Koch,et al.  Progress in presolving for mixed integer programming , 2015, Math. Program. Comput..

[48]  Laurence A. Wolsey,et al.  Cutting planes in integer and mixed integer programming , 2002, Discret. Appl. Math..

[49]  Ted K. Ralphs,et al.  Integer and Combinatorial Optimization , 2013 .

[50]  Michael Jünger,et al.  Introduction to ABACUS - a branch-and-cut system , 1998, Oper. Res. Lett..

[51]  Thomas Polzer,et al.  Portable parallel branch-and-bound library(ppbb-lib): User manual version 1 , 1996 .

[52]  George L. Nemhauser,et al.  Using diversification, communication and parallelism to solve mixed-integer linear programs , 2014, Oper. Res. Lett..

[53]  Jeff T. Linderoth,et al.  An enabling framework for master-worker applications on the Computational Grid , 2000, Proceedings the Ninth International Symposium on High-Performance Distributed Computing.

[54]  Timo Berthold,et al.  Parallelization of the FICO Xpress-Optimizer , 2016, ICMS.

[55]  Vipin Kumar,et al.  Scalable Load Balancing Techniques for Parallel Computers , 1994, J. Parallel Distributed Comput..

[56]  Matteo Fischetti,et al.  Improving branch-and-cut performance by random sampling , 2016, Math. Program. Comput..

[57]  Jonathan Eckstein.  Control strategies for parallel mixed integer branch and bound , 1994, Proceedings of Supercomputing '94.

[58]  Yuji Shinano,et al.  Effectiveness of Parallelizing the ILOG-CPLEX Mixed Integer Optimizer in the PUBB2 Framework , 2003, Euro-Par.

[59]  Gerhard Wellein,et al.  Introduction to High Performance Computing for Scientists and Engineers , 2010, Chapman and Hall / CRC computational science series.

[60]  A. Lodi,et al.  Heuristics in Mixed Integer Programming , 2011 .

[61]  Enrique Alba,et al.  MALLBA: A Library of Skeletons for Combinatorial Optimisation (Research Note) , 2002, Euro-Par.

[62]  Yuji Shinano,et al.  A Dynamic Load Balancing Mechanism for New ParaLEX , 2008, 2008 14th IEEE International Conference on Parallel and Distributed Systems.

[63]  Zbigniew J. Czech,et al.  Introduction to Parallel Computing , 2017 .

[64]  A. Osman,et al.  Dynamic Load Balancing Strategies for Parallel Computers , 2002, Sci. Ann. Cuza Univ..

[65]  Qun Chen,et al.  FATCOP 2.0: Advanced Features in an Opportunistic Mixed Integer Programming Solver , 2001, Ann. Oper. Res..

[66]  M. Saltzman,et al.  ALPS : A Framework for Implementing Parallel Search Algorithms , 2004 .

[67]  Peter Sanders.  Randomized Static Load Balancing for Tree-Shaped Computations , 1994 .

[68]  Jeff Linderoth,et al.  Topics in parallel integer optimization , 1998 .

[69]  L. G. H. Cijan.  A polynomial algorithm in linear programming , 1979 .

[70]  Cynthia A. Phillips,et al.  PEBBL: an object-oriented framework for scalable parallel branch and bound , 2015, Mathematical Programming Computation.

[71]  M. Saltzman,et al.  Alps: A Framework for Implementing Parallel Tree Search Algorithms , 2005 .

[72]  Ailsa H. Land,et al.  An Automatic Method of Solving Discrete Programming Problems , 1960 .

[73]  Michael C. Ferris,et al.  Grid-Enabled Optimization with GAMS , 2009, INFORMS J. Comput..

[74]  Laxmikant V. Kalé,et al.  Periodic hierarchical load balancing for large supercomputers , 2011, Int. J. High Perform. Comput. Appl..

[75]  Jorge J. Moré,et al.  Digital Object Identifier (DOI) 10.1007/s101070100263 , 2001 .

[76]  Timo Berthold.  Primal Heuristics for Mixed Integer Programs , 2006 .

[77]  Gérard Cornuéjols,et al.  Early Estimates of the Size of Branch-and-Bound Trees , 2006, INFORMS J. Comput..

[78]  Teodor Gabriel Crainic,et al.  Parallel Branch‐and‐Bound Algorithms , 2006 .

[79]  Yuji Shinano,et al.  A generalized utility for parallel branch and bound algorithms , 1995, Proceedings.Seventh IEEE Symposium on Parallel and Distributed Processing.

[80]  Thorsten Koch,et al.  SCIP-Jack—a solver for STP and variants with parallelization extensions , 2017, Math. Program. Comput..

[81]  L. Khachiyan.  Polynomial algorithms in linear programming , 1980 .

[82]  Anthony P. Reeves,et al.  Strategies for Dynamic Load Balancing on Highly Parallel Computers , 1993, IEEE Trans. Parallel Distributed Syst..

[83]  Yuji Shinano,et al.  ParaLEX: A Parallel Extension for the CPLEX Mixed Integer Optimizer , 2007, PVM/MPI.

[84]  Jürg Nievergelt,et al.  The parallel search bench ZRAM and its applications , 1999, Ann. Oper. Res..

[85]  Yuji Shinano,et al.  Distributed Domain Propagation , 2017, SEA.

[86]  Matthew J. Saltzman,et al.  Parallel branch, cut, and price for large-scale discrete optimization , 2003, Math. Program..