Boosting Verification by Automatic Tuning of Decision Procedures

Parameterized heuristics abound in computer aided design and verification, and manual tuning of the respective parameters is difficult and time-consuming. Very recent results from the artificial intelligence (AI) community suggest that this tuning process can be automated, and that doing so can lead to significant performance improvements; furthermore, automated parameter optimization can provide valuable guidance during the development of heuristic algorithms. In this paper, we study how such an AI approach can improve a state-of-the-art SAT solver for large, real-world bounded model-checking and software verification instances. The resulting, automatically-derived parameter settings yielded runtimes on average 4.5 times faster on bounded model checking instances and 500 times faster on software verification problems than extensive hand-tuning of the decision procedure. Furthermore, the availability of automatic tuning influenced the design of the solver, and the automatically-derived parameter settings provided a deeper insight into the properties of problem instances.

[1]  Donald W. Loveland,et al.  A machine program for theorem-proving , 2011, CACM.

[2]  Charles Gregory Nelson,et al.  Techniques for program verification , 1979 .

[3]  Randal E. Bryant,et al.  Graph-Based Algorithms for Boolean Function Manipulation , 1986, IEEE Transactions on Computers.

[4]  Joao Marques-Silva,et al.  The Impact of Branching Heuristics in Propositional Satisfiability Algorithms , 1999, EPIA.

[5]  Daniel Jackson.  Automating first-order relational logic , 2000, SIGSOFT '00/FSE-8.

[6]  Ofer Shtrichman.  Tuning SAT Checkers for Bounded Model Checking , 2000, CAV 2000.

[7]  M. Moskewicz,et al.  Chaff: engineering an efficient SAT solver , 2001, Proceedings of the 38th Design Automation Conference (IEEE Cat. No.01CH37232).

[8]  Helena Ramalhinho Dias Lourenço,et al.  Iterated Local Search , 2001, Handbook of Metaheuristics.

[9]  Thomas Stützle,et al.  A Racing Algorithm for Configuring Metaheuristics , 2002, GECCO.

[10]  James C. Spall,et al.  Introduction to stochastic search and optimization - estimation, simulation, and control , 2003, Wiley-Interscience series in discrete mathematics and optimization.

[11]  Ohad Shacham,et al.  Tuning the VSIDS decision heuristic for bounded model checking , 2003, Proceedings. 4th International Workshop on Microprocessor Test and Verification - Common Challenges and Solutions.

[12]  Niklas Sörensson,et al.  An Extensible SAT-solver , 2003, SAT.

[13]  Inês Lynce,et al.  Heuristic backtracking algorithms for SAT , 2003, Proceedings. 4th International Workshop on Microprocessor Test and Verification - Common Challenges and Solutions.

[14]  Feng Lu,et al.  A Signal Correlation Guided ATPG solver and its applications for solving difficult industrial cases , 2003, Proceedings 2003. Design Automation Conference (IEEE Cat. No.03CH37451).

[15]  Armin Biere,et al.  A survey of recent advances in SAT-based formal verification , 2005, International Journal on Software Tools for Technology Transfer.

[16]  Thomas Stützle,et al.  Stochastic Local Search: Foundations & Applications , 2004 .

[17]  Emmanuel Zarpas,et al.  Benchmarking SAT Solvers for Bounded Model Checking , 2005, SAT.

[18]  Sanjit A. Seshia,et al.  Adaptive eager boolean encoding for arithmetic reasoning in verification , 2005 .

[19]  D. Babic,et al.  Modular Arithmetic Decision Procedure , 2005 .

[20]  Kevin Leyton-Brown,et al.  Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms , 2006, CP.

[21]  Manuel Laguna,et al.  Fine-Tuning of Algorithms Using Fractional Experimental Designs and Local Search , 2006, Oper. Res..

[22]  Kevin Leyton-Brown,et al.  Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms: An Initial Investigation ∗ , 2006 .

[23]  Alan J. Hu,et al.  Exploiting Shared Structure in Software Verification Conditions , 2007, Haifa Verification Conference.

[24]  Teofilo F. Gonzalez,et al.  Reactive Search: Machine Learning for Memory-Based Heuristics , 2007 .

[25]  Alan J. Hu,et al.  Structural Abstraction of Software Verification Conditions , 2007, CAV.

[26]  Thomas Stützle,et al.  Automatic Algorithm Configuration Based on Local Search , 2007, AAAI.