Challenging SMT solvers to verify neural networks

In recent years, Satisfiability Modulo Theory (SMT) solvers are becoming increasingly popular in the Computer Aided Verification and Reasoning community. Used natively or as back-engines, they are accumulating a record of success stories and, as witnessed by the annual SMT competition, their performances and capacity are also increasing steadily. Introduced in previous contributions of ours, a new application domain providing an outstanding challenge for SMT solvers is represented by verification of Multi-Layer Perceptrons (MLPs) a widely-adopted kind of artificial neural network. In this paper we present an extensive evaluation of the current state-of-the-art SMT solvers and assess their potential in the promising domain of MLP verification.

[1]  Sandip Ray,et al.  Connecting External Deduction Tools with ACL2 , 2010 .

[2]  Kevin Leyton-Brown,et al.  Automated Configuration of Mixed Integer Programming Solvers , 2010, CPAIOR.

[3]  David R. Cok.  The SMT-LIBv2 Language and Tools: A Tutorial , 2012 .

[4]  Giulio Sandini,et al.  Learning to Exploit Proximal Force Sensing: A Comparison Approach , 2010, From Motor Learning to Interaction Learning in Robots.

[5]  Michaël Rusinowitch,et al.  A rewriting approach to satisfiability procedures , 2003, Inf. Comput..

[6]  Cesare Tinelli,et al.  Solving SAT and SAT Modulo Theories: From an abstract Davis--Putnam--Logemann--Loveland procedure to DPLL(T) , 2006, JACM.

[7]  Piergiorgio Bertoli,et al.  A SAT Based Approach for Solving Formulas over Boolean and Linear Mathematical Propositions , 2002, CADE.

[8]  Martin Fränzle,et al.  Efficient Solving of Large Non-linear Arithmetic Constraint Systems with Complex Boolean Structure , 2007, J. Satisf. Boolean Model. Comput..

[9]  Kurt Hornik,et al.  Approximation capabilities of multilayer feedforward networks , 1991, Neural Networks.

[10]  Yan Liu,et al.  Applications of Neural Networks in High Assurance Systems , 2010, Applications of Neural Networks in High Assurance Systems.

[11]  Greg Nelson,et al.  Simplification by Cooperating Decision Procedures , 1979, TOPL.

[12]  Luca Pulina,et al.  Checking Safety of Neural Networks with SMT Solvers: A Comparative Evaluation , 2011, AI*IA.

[13]  Kurt Hornik,et al.  Multilayer feedforward networks are universal approximators , 1989, Neural Networks.

[14]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[15]  Luca Pulina,et al.  An Abstraction-Refinement Approach to Verification of Artificial Neural Networks , 2010, CAV.

[16]  Bruno Dutertre,et al.  A Fast Linear-Arithmetic Solver for DPLL(T) , 2006, CAV.

[17]  Roberto Bruttomesso,et al.  The OpenSMT Solver , 2010, TACAS.

[18]  Eran Yahav,et al.  Abstraction-guided synthesis of synchronization , 2010, POPL '10.

[19]  Alessandro Armando,et al.  Bounded model checking of software using SMT solvers instead of SAT solvers , 2006, International Journal on Software Tools for Technology Transfer.

[20]  Enrico Giunchiglia,et al.  SAT-Based Procedures for Temporal Reasoning , 1999, ECP.

[21]  Claude Marché,et al.  The Why/Krakatoa/Caduceus Platform for Deductive Program Verification , 2007, CAV.

[22]  Giulio Sandini,et al.  James: A Humanoid Robot Acting over an Unstructured World , 2006, 2006 6th IEEE-RAS International Conference on Humanoid Robots.

[23]  Nguyen Ngoc Binh,et al.  Extending CREST with Multiple SMT Solvers and Real Arithmetic , 2010, 2010 Second International Conference on Knowledge and Systems Engineering.

[24]  I. Porceddu,et al.  Cybersar: A lambda grid computing infrastructure for advanced applications , 2009, 2009 IEEE Nuclear Science Symposium Conference Record (NSS/MIC).

[25]  Alan K. Mackworth.  Consistency in Networks of Relations , 1977, Artif. Intell..

[26]  Armando Solar-Lezama,et al.  Sketching concurrent data structures , 2008, PLDI '08.

[27]  S. Hyakin,et al.  Neural Networks: A Comprehensive Foundation , 1994 .

[28]  Luca Pulina,et al.  A self-adaptive multi-engine solver for quantified Boolean formulas , 2009, Constraints.

[29]  Charles M. Bachmann,et al.  Neural Networks and Their Applications , 1994 .

[30]  Giulio Sandini,et al.  The iCub humanoid robot: An open-systems platform for research in cognitive development , 2010, Neural Networks.

[31]  Aaron Stump,et al.  SMT-COMP: Satisfiability Modulo Theories Competition , 2005, CAV.

[32]  Stephan Merz,et al.  Expressiveness + Automation + Soundness: Towards Combining SMT Solvers and Interactive Proof Assistants , 2006, TACAS.

[33]  K. Rustan M. Leino,et al.  BoogiePL: A typed procedural language for checking object-oriented programs , 2005 .

[34]  Petros Koumoutsakos,et al.  Reducing the Time Complexity of the Derandomized Evolution Strategy with Covariance Matrix Adaptation (CMA-ES) , 2003, Evolutionary Computation.