Evaluating and certifying QBFs: A comparison of state-of-the-art tools

In this paper we compare the performance of all the currently available suites to evaluate and certify QBFs. Our aim is to assess the current state of the art, and also to understand to which extent QBF encodings can be evaluated producing certificates that can be checked in a reliable and efficient way. We conclude that, while the evaluation of some QBFs is still an open challenge, producing and checking certificates for many medium-to-large scale QBFs is feasible with the current technology.

[1]  Sharad Malik,et al.  Validating SAT solvers using an independent resolution-based checker: practical implementations and other applications , 2003, 2003 Design, Automation and Test in Europe Conference and Exhibition.

[2]  Hans Kleine Büning,et al.  Resolution for Quantified Boolean Formulas , 1995, Inf. Comput..

[3]  Geoff Sutcliffe,et al.  Evaluating general purpose automated theorem proving systems , 2001, Artif. Intell..

[4]  Ian P. Gent,et al.  Encoding Connect-4 using Quantified Boolean Formulae , 2003 .

[5]  Eugene Goldberg,et al.  Verification of proofs of unsatisfiability for CNF formulas , 2003, 2003 Design, Automation and Test in Europe Conference and Exhibition.

[6]  Marco Benedetti,et al.  Quantifier Trees for QBFs , 2005, SAT.

[7]  Subhabrata Chakraborti,et al.  Nonparametric Statistical Inference , 2011, International Encyclopedia of Statistical Science.

[8]  Armin Biere,et al.  Extended Resolution Proofs for Conjoining BDDs , 2006, CSR.

[9]  Luca Pulina,et al.  Report of the Third QBF Solvers Evaluation , 2006, J. Satisf. Boolean Model. Comput..

[10]  Marco Benedetti,et al.  Evaluating QBFs via Symbolic Skolemization , 2005, LPAR.

[11]  Marco Benedetti.  Extracting Certificates from Quantified Boolean Formulas , 2005, IJCAI.

[12]  S. Malik,et al.  Validating the result of a quantified Boolean formula (QBF) solver: theory and practice , 2005, Proceedings of the ASP-DAC 2005. Asia and South Pacific Design Automation Conference, 2005..

[13]  Bart Selman,et al.  The Achilles' Heel of QBF , 2005, AAAI.

[14]  Hubie Chen,et al.  Quantified Constraint Satisfaction and Bounded Treewidth , 2004, ECAI.

[15]  Stefan Woltran,et al.  Solving Advanced Reasoning Tasks Using Quantified Boolean Formulas , 2000, AAAI/IAAI.

[16]  Jussi Rintanen.  Partial Implicit Unfolding in the Davis-Putnam Procedure for Quantified Boolean Formulae , 2001, LPAR.

[17]  Edmund M. Clarke,et al.  Symbolic Model Checking: 10^20 States and Beyond , 1990, Inf. Comput..

[18]  Sharad Malik,et al.  Analysis of Search Based Algorithms for Satisfiability of Propositional and Quantified Boolean Formulas Arising from Circuit State Space Diameter Problems , 2004, SAT (Selected Papers.

[19]  Armando Tacchella,et al.  Quantifier Structure in Search-Based Procedures for QBFs , 2006, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[20]  Roderick Bloem,et al.  Fault Localization and Correction with QBF , 2007, SAT.

[21]  Armando Tacchella,et al.  Watched Data Structures for QBF Solvers , 2003, SAT.

[22]  Armin Biere,et al.  Extended Resolution Proofs for Symbolic SAT Solving with Quantification , 2006, SAT.

[23]  Nachum Dershowitz,et al.  Bounded Model Checking with QBF , 2005, SAT.

[24]  Allen Van Gelder.  Verifying Propositional Unsatisfiability: Pitfalls to Avoid , 2007, SAT.

[25]  Albert Oliveras,et al.  Efficient Generation of Unsatisfiability Proofs and Cores in SAT , 2008, LPAR.

[26]  Karem A. Sakallah,et al.  Computing Vertex Eccentricity in Exponentially Large Graphs: QBF Formulation and Solution , 2003, SAT.

[27]  Daniel Kroening,et al.  A First Step Towards a Unified Proof Checker for QBF , 2007, SAT.

[28]  Armin Biere,et al.  Symbolic Model Checking without BDDs , 1999, TACAS.

[29]  Armando Tacchella,et al.  Clause/Term Resolution and Learning in the Evaluation of Quantified Boolean Formulas , 2006, J. Artif. Intell. Res..

[30]  Armin Biere,et al.  Compressing BMC Encodings with QBF , 2007, BMC@FLoC.

[31]  Marco Schaerf,et al.  An Algorithm to Evaluate Quantified Boolean Formulae , 1998, AAAI/IAAI.

[32]  Salil P. Vadhan,et al.  Computational Complexity , 2005, Encyclopedia of Cryptography and Security.

[33]  Armando Tacchella,et al.  QuBE++: An Efficient QBF Solver , 2004, FMCAD.

[34]  Bernd Becker,et al.  Advanced SAT-Techniques for Bounded Model Checking of Blackbox Designs , 2006, Seventh International Workshop on Microprocessor Test and Verification (MTV'06).

[35]  Allen Van Gelder.  Extracting (Easily) Checkable Proofs from a Satisfiability Solver that Employs both Preorder and Postorder Resolution , 2002, ISAIM.

[36]  Sharad Malik,et al.  Analysis of Search Based Algorithms for Satisfiability of Quantified Boolean Formulas Arising from Circuit State Space Diameter Problems , 2004, SAT.

[37]  Sharad Malik,et al.  Towards a Symmetric Treatment of Satisfaction and Conflicts in Quantified Boolean Formula Evaluation , 2002, CP.