A self-adaptive multi-engine solver for quantified Boolean formulas

In this paper we study the problem of engineering a robust solver for quantified Boolean formulas (QBFs), i.e., a tool that can efficiently solve formulas across different problem domains without the need for domain-specific tuning. The paper presents two main empirical results along this line of research. Our first result is the development of a multi-engine solver, i.e., a tool that selects among its reasoning engines the one which is more likely to yield optimal results. In particular, we show that syntactic QBF features can be correlated to the performances of existing QBF engines across a variety of domains. We also show how a multi-engine solver can be obtained by carefully picking state-of-the-art QBF solvers as basic engines, and by harnessing inductive reasoning techniques to learn engine-selection policies. Our second result is the improvement of our multi-engine solver with the capability of updating the learned policies when they fail to give good predictions. In this way the solver becomes also self-adaptive, i.e., able to adjust its internal models when the usage scenario changes substantially. The rewarding results obtained in our experiments show that our solver AQME—Adaptive QBF Multi-Engine—can be more robust and efficient than state-of-the-art single-engine solvers, even when it is confronted with previously uncharted formulas and competitors.

[1]  Kevin Leyton-Brown,et al.  Hierarchical Hardness Models for SAT , 2007, CP.

[2]  S. Cessie,et al.  Ridge Estimators in Logistic Regression , 1992 .

[3]  Bart Selman,et al.  The Achilles' Heel of QBF , 2005, AAAI.

[4]  Kevin Leyton-Brown,et al.  : The Design and Analysis of an Algorithm Portfolio for SAT , 2007, CP.

[5]  Donald W. Loveland,et al.  A machine program for theorem-proving , 2011, CACM.

[6]  Karem A. Sakallah,et al.  Computing Vertex Eccentricity in Exponentially Large Graphs: QBF Formulation and Solution , 2003, SAT.

[7]  William W. Cohen Fast Effective Rule Induction , 1995, ICML.

[8]  Hudson Turner,et al.  Polynomial-Length Planning Spans the Polynomial Hierarchy , 2002, JELIA.

[9]  Ron Kohavi,et al.  A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection , 1995, IJCAI.

[10]  Henry Kautz,et al.  Branch and bound algorithm selection by performance prediction , 2001, Conference on Uncertainty in Artificial Intelligence.

[11]  Ian P. Gent,et al.  Encoding Connect-4 using Quantified Boolean Formulae , 2003 .

[12]  Hans Kleine Büning,et al.  Resolution for Quantified Boolean Formulas , 1995, Inf. Comput..

[13]  Stephen F. Smith,et al.  Restart Schedules for Ensembles of Problem Instances , 2007, AAAI.

[14]  Luca Pulina,et al.  Report of the Third QBF Solvers Evaluation , 2006, J. Satisf. Boolean Model. Comput..

[15]  Y. Shoham,et al.  SATzilla : An Algorithm Portfolio for SAT ∗ , 2004 .

[16]  Armando Tacchella,et al.  SAT-based planning in complex domains: Concurrency, constraints and nondeterminism , 2003, Artif. Intell..

[17]  Horst Samulowitz,et al.  Learning to Solve QBF , 2007, AAAI.

[18]  Moshe Y. Vardi,et al.  Optimizing a BDD-Based Modal Solver , 2003, CADE.

[19]  Luca Pulina,et al.  A Multi-engine Solver for Quantified Boolean Formulas , 2007, CP.

[20]  Luca Pulina,et al.  Ranking and Reputation Systems in the QBF Competition , 2007, AI*IA.

[21]  J. Ross Quinlan,et al.  C4.5: Programs for Machine Learning , 1992 .

[22]  Tad Hogg,et al.  An Economics Approach to Hard Computational Problems , 1997, Science.

[23]  Jussi Rintanen Partial Implicit Unfolding in the Davis-Putnam Procedure for Quantified Boolean Formulae , 2001, LPAR.

[24]  Armin Biere,et al.  Resolve and Expand , 2004, SAT.

[25]  Hector J. Levesque,et al.  Hard and Easy Distributions of SAT Problems , 1992, AAAI.

[26]  Ian P. Gent,et al.  Encoding Quantified CSPs as Quantified Boolean Formulae , 2004, ECAI.

[27]  Bernd Becker,et al.  Advanced SAT-Techniques for Bounded Model Checking of Blackbox Designs , 2006, Seventh International Workshop on Microprocessor Test and Verification (MTV'06).

[28]  Igor Stéphan Boolean Propagation Based on Literals for Quantified Boolean Formulae , 2006, ECAI.

[29]  Eugene C. Freuder,et al.  Using CBR to Select Solution Strategies in Constraint Programming , 2005, ICCBR.

[30]  Stefan Woltran,et al.  Solving Advanced Reasoning Tasks Using Quantified Boolean Formulas , 2000, AAAI/IAAI.

[31]  Yoav Shoham,et al.  Understanding Random SAT: Beyond the Clauses-to-Variables Ratio , 2004, CP.

[32]  Ian Witten,et al.  Data Mining , 2000 .

[33]  Nachum Dershowitz,et al.  Bounded Model Checking with QBF , 2005, SAT.

[34]  David W. Aha,et al.  Instance-Based Learning Algorithms , 1991, Machine Learning.

[35]  Alberto Maria Segre,et al.  Programs for Machine Learning , 1994 .

[36]  Marco Benedetti,et al.  sKizzo: A Suite to Evaluate and Certify QBFs , 2005, CADE.

[37]  Luca Pulina,et al.  The QBFEVAL Web Portal , 2006, JELIA.

[38]  Bart Selman,et al.  Algorithm portfolios , 2001, Artif. Intell..

[39]  Armin Biere,et al.  Compressing BMC Encodings with QBF , 2007, BMC@FLoC.

[40]  Albert R. Meyer,et al.  Word problems requiring exponential time(Preliminary Report) , 1973, STOC.