Multi-language evaluation of exact solvers in graphical model discrete optimization

By representing the constraints and objective function in factorized form, graphical models can concisely define various NP-hard optimization problems. They are therefore extensively used in several areas of computer science and artificial intelligence. Graphical models can be deterministic or stochastic, optimize a sum or product of local functions, defining a joint cost or probability distribution. Simple transformations exist between these two types of models, but also with MaxSAT or linear programming. In this paper, we report on a large comparison of exact solvers which are all state-of-the-art for their own target language. These solvers are all evaluated on deterministic and probabilistic graphical models coming from the Probabilistic Inference Challenge 2011, the Computer Vision and Pattern Recognition OpenGM2 benchmark, the Weighted Partial MaxSAT Evaluation 2013, the MaxCSP 2008 Competition, the MiniZinc Challenge 2012 & 2013, and the CFLib (a library of Cost Function Networks). All 3026 instances are made publicly available in five different formats and seven formulations. To our knowledge, this is the first evaluation that encompasses such a large set of related NP-complete optimization frameworks, despite their tight connections. The results show that a small number of evaluated solvers are able to perform well on multiple areas. By exploiting the variability and complementarity of solver performances, we show that a simple portfolio approach can be very effective. This portfolio won the last UAI Evaluation 2014 (MAP task).

[1]  Tad Hogg,et al.  An Economics Approach to Hard Computational Problems , 1997, Science.

[2]  WernerTomas.  A Linear Programming Approach to Max-Sum Problem , 2007 .

[3]  Barry O'Sullivan,et al.  Dead-End Elimination for Weighted CSP , 2013, CP.

[4]  Nir Friedman,et al.  Probabilistic Graphical Models - Principles and Techniques , 2009 .

[5]  Toby Walsh,et al.  Handbook of Constraint Programming , 2006, Handbook of Constraint Programming.

[6]  S. D. Givry,et al.  Decomposing Global Cost Functions , 2011, CP 2011.

[7]  Bart Selman,et al.  Algorithm portfolios , 2001, Artif. Intell..

[8]  Simon de Givry,et al.  A logical approach to efficient Max-SAT solving , 2006, Artif. Intell..

[9]  Simon de Givry,et al.  Anytime Hybrid Best-First Search with Tree Decomposition for Weighted CSP , 2015, CP.

[10]  Christian Bessiere,et al.  Meta-constraints on violations for over constrained problems , 2000, Proceedings 12th IEEE Internationals Conference on Tools with Artificial Intelligence. ICTAI 2000.

[11]  Barry O'Sullivan,et al.  Proteus: A Hierarchical Portfolio of Solvers and Transformations , 2013, CPAIOR.

[12]  Maurizio Gabbrielli,et al.  A Multicore Tool for Constraint Solving , 2015, IJCAI.

[13]  Yuri Malitsky,et al.  Instance-specific algorithm configuration , 2014, Constraints.

[14]  Barry O'Sullivan,et al.  Constraint Programming and Combinatorial Optimisation in Numberjack , 2010, CPAIOR.

[15]  Tomás Werner,et al.  Universality of the Local Marginal Polytope , 2013, IEEE Transactions on Pattern Analysis and Machine Intelligence.

[16]  Yuri Malitsky,et al.  MaxSAT by Improved Instance-Specific Algorithm Configuration , 2014, AAAI.

[17]  Tommi S. Jaakkola,et al.  Fixing Max-Product: Convergent Message Passing Algorithms for MAP LP-Relaxations , 2007, NIPS.

[18]  Fahiem Bacchus,et al.  Exploiting the Power of mip Solvers in maxsat , 2013, SAT.

[19]  Lars Kotthoff,et al.  Algorithm Selection for Combinatorial Search Problems: A Survey , 2012, AI Mag..

[20]  Martin C. Cooper,et al.  Soft arc consistency revisited , 2010, Artif. Intell..

[21]  Simon de Givry,et al.  Mendelian Error Detection in Complex Pedigrees Using Weighted Constraint Satisfaction Techniques , 2007, Constraints.

[22]  Rina Dechter,et al.  Bucket Elimination: A Unifying Framework for Reasoning , 1999, Artif. Intell..

[23]  Eoin O'Mahony,et al.  Using Case-based Reasoning in an Algorithm Portfolio for Constraint Solving ? , 2008 .

[24]  Fahiem Bacchus,et al.  Solving MAXSAT by Solving a Sequence of Simpler SAT Instances , 2011, CP.

[25]  Simon de Givry,et al.  Computational Protein Design as a Cost Function Network Optimization Problem , 2012, CP.

[26]  Jozef Kratica,et al.  Solving the simple plant location problem by genetic algorithm , 2001, RAIRO Oper. Res..

[27]  Josep Argelich,et al.  Encoding Max-CSP into Partial Max-SAT , 2008, 38th International Symposium on Multiple Valued Logic (ismvl 2008).

[28]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[29]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Discrete Energy Minimization Problems , 2013, 2013 IEEE Conference on Computer Vision and Pattern Recognition.

[30]  M. Jünger,et al.  50 Years of Integer Programming 1958-2008 - From the Early Years to the State-of-the-Art , 2010 .

[31]  Yuri Malitsky,et al.  ISAC - Instance-Specific Algorithm Configuration , 2010, ECAI.

[32]  Arie M. C. A. Koster,et al.  Frequency assignment : models and algorithms , 1999 .

[33]  Martin C. Cooper,et al.  Optimal Soft Arc Consistency , 2007, IJCAI.

[34]  R. Dechter,et al.  Winning the PASCAL 2011 MAP Challenge with Enhanced AND / OR Branch-and-Bound , 2011 .

[35]  Simon de Givry,et al.  Radio Link Frequency Assignment , 1999, Constraints.

[36]  David Sontag,et al.  Efficiently Searching for Frustrated Cycles in MAP Inference , 2012, UAI.

[37]  Tommi S. Jaakkola,et al.  Tightening LP Relaxations for MAP using Message Passing , 2008, UAI.

[38]  Thomas Schiex,et al.  Soft Constraints , 2000, WLP.

[39]  Wei-Yin Loh,et al.  Classification and regression trees , 2011, WIREs Data Mining Knowl. Discov..

[40]  Sebastian Nowozin,et al.  A Comparative Study of Modern Inference Techniques for Structured Discrete Energy Minimization Problems , 2014, International Journal of Computer Vision.

[41]  Simon de Givry,et al.  Filtering Decomposable Global Cost Functions , 2012, AAAI.

[42]  Amir Globerson,et al.  Convergent message passing algorithms - a unifying view , 2009, UAI.

[43]  Simon de Givry,et al.  Existential arc consistency: Getting closer to full arc consistency in weighted CSPs , 2005, IJCAI.

[44]  Daniel Prusa,et al.  Universality of the Local Marginal Polytope , 2013, CVPR.

[45]  Fred W. Glover,et al.  ID Walk: A Candidate List Strategy with a Simple Diversification Device , 2004, CP.

[46]  Simon de Givry,et al.  Pairwise Decomposition for Combinatorial Optimization in Graphical Models , 2011, IJCAI.

[47]  Lars Kotthoff,et al.  LLAMA: Leveraging Learning to Automatically Manage Algorithms , 2013, ArXiv.

[48]  Fahiem Bacchus,et al.  GAC Via Unit Propagation , 2007, CP.

[49]  Peter J. Stuckey,et al.  MiniZinc: Towards a Standard CP Modelling Language , 2007, CP.

[50]  Gérard Verfaillie,et al.  Earth Observation Satellite Management , 1999, Constraints.

[51]  Thomas Schiex,et al.  A constraint satisfaction framework for decision under uncertainty , 1995, UAI.

[52]  Tomás Werner,et al.  A Linear Programming Approach to Max-Sum Problem: A Review , 2007, IEEE Transactions on Pattern Analysis and Machine Intelligence.