ML + FV = $\heartsuit$? A Survey on the Application of Machine Learning to Formal Verification

Formal Veri€cation (Fv) and Machine Learning (Ml) can seem incompatible due to their opposite mathematical foundations and their use in real-life problems: Fv mostly relies on discrete mathematics and aims at ensuring correctness; Ml o‰en relies on probabilistic models and consists of learning paŠerns from training data. In this paper, we postulate that they are complementary in practice, and explore how Ml helps Fv in its classical approaches: static analysis, model-checking, theorem-proving, and Sat solving. We draw a landscape of the current practice and catalog some of the most prominent uses of Ml inside Fv tools, thus o‚ering a new perspective on Fv techniques that can help researchers and practitioners to beŠer locate the possible synergies. We discuss lessons learned from our work, point to possible improvements and o‚er visions for the future of the domain in the light of the science of so‰ware and systems modeling.

[1]  David Maxwell Chickering,et al.  A Bayesian Approach to Tackling Hard Computational Problems (Preliminary Report) , 2001, Electron. Notes Discret. Math..

[2]  Ivana Kruijff-Korbayová,et al.  A Portfolio Approach to Algorithm Selection , 2003, IJCAI.

[3]  Barry O’Sullivan,et al.  Satisfiability as a Classification Problem ? , 2008 .

[4]  Ronald L. Rivest,et al.  Inference of finite automata using homing sequences , 1989, STOC '89.

[5]  Howard Barringer,et al.  Proof Rules for Automated Compositional Verification through Learning , 2003 .

[6]  Sagar Chaki,et al.  Automated Assume-Guarantee Reasoning for Simulation Conformance , 2005, CAV.

[7]  Benoît Frénay,et al.  Interpretability of machine learning models and representations: an introduction , 2016, ESANN.

[8]  Marius Thomas Lindauer,et al.  AutoFolio: An Automatically Configured Algorithm Selector , 2015, J. Artif. Intell. Res..

[9]  Majid Nili Ahmadabadi,et al.  Bounded Rational Search for On-the-Fly Model Checking of LTL Properties , 2009, FSEN.

[10]  Yuanyuan Zhou,et al.  Learning from mistakes: a comprehensive study on real world concurrency bug characteristics , 2008, ASPLOS.

[11]  Kevin Barraclough,et al.  I and i , 2001, BMJ : British Medical Journal.

[12]  Babu M. Mehtre,et al.  Static Malware Analysis Using Machine Learning Methods , 2014, SNDS.

[13]  Alexander Serebrenik,et al.  Survey of Approaches for Handling Static Analysis Alarms , 2016, 2016 IEEE 16th International Working Conference on Source Code Analysis and Manipulation (SCAM).

[14]  Cezary Kaliszyk,et al.  Efficient Semantic Features for Automated Reasoning over Large Theories , 2015, IJCAI.

[15]  Sarah Smith Heckman,et al.  A systematic literature review of actionable alert identification techniques for automated static code analysis , 2011, Inf. Softw. Technol..

[16]  Yi Zhang,et al.  Classifying Software Changes: Clean or Buggy? , 2008, IEEE Transactions on Software Engineering.

[17]  Horst Samulowitz,et al.  Learning to Solve QBF , 2007, AAAI.

[18]  Krzysztof Czarnecki,et al.  Learning Rate Based Branching Heuristic for SAT Solvers , 2016, SAT.

[19]  Vahid Rafe,et al.  EMCDM: Efficient model checking by data mining for verification of complex software systems specified through architectural styles , 2016, Appl. Soft Comput..

[20]  Howard Barringer,et al.  Learning to divide and conquer: applying the L* algorithm to automate assume-guarantee reasoning , 2008, Formal Methods Syst. Des..

[21]  Krzysztof Czarnecki,et al.  Exponential Recency Weighted Average Branching Heuristic for SAT Solvers , 2016, AAAI.

[22]  Tao Wang,et al.  Convolutional Neural Networks over Tree Structures for Programming Language Processing , 2014, AAAI.

[23]  Junfeng Yang,et al.  Correlation exploitation in error ranking , 2004, SIGSOFT '04/FSE-12.

[24]  Tim Rocktäschel,et al.  End-to-end Differentiable Proving , 2017, NIPS.

[25]  Lin Tan,et al.  Finding patterns in static analysis alerts: improving actionable alert ranking , 2014, MSR 2014.

[26]  David Zuckerman,et al.  Optimal speedup of Las Vegas algorithms , 1993, [1993] The 2nd Israel Symposium on Theory and Computing Systems.

[27]  Sarah Smith Heckman,et al.  A Model Building Process for Identifying Actionable Static Analysis Alerts , 2009, 2009 International Conference on Software Testing Verification and Validation.

[28]  Kevin Leyton-Brown,et al.  SATzilla: Portfolio-based Algorithm Selection for SAT , 2008, J. Artif. Intell. Res..

[29]  Tsuyoshi Murata,et al.  {m , 1934, ACML.

[30]  Eric Horvitz,et al.  Restart Policies with Dependence among Runs: A Dynamic Programming Approach , 2002, CP.

[31]  Lawrence C. Paulson,et al.  Machine Learning for First-Order Theorem Proving , 2014, Journal of Automated Reasoning.

[32]  Win Zaw,et al.  Permission-Based Android Malware Detection , 2013 .

[33]  Tadashi Araragi,et al.  Checking Liveness Properties of Concurrent Systems by Reinforcement Learning , 2007, MoChArt.

[34]  Eric Horvitz,et al.  Dynamic restart policies , 2002, AAAI/IAAI.

[35]  Adam Naumowicz,et al.  Mizar in a Nutshell , 2010, J. Formaliz. Reason..

[36]  Robert P. Kurshan,et al.  An Analysis of SAT-Based Model Checking Techniques in an Industrial Environment , 2005, CHARME.

[37]  Maria Luisa Bonet,et al.  Structure features for SAT instances classification , 2017, J. Appl. Log..

[38]  Patrick Cousot,et al.  A gentle introduction to formal verification of computer systems by abstract interpretation , 2010, Logics and Languages for Reliability and Security.

[39]  Dana Angluin,et al.  Learning Regular Sets from Queries and Counterexamples , 1987, Inf. Comput..

[40]  Matthias Fuchs.  A Feature-Based Learning Method for Theorem Proving , 1998, AAAI/IAAI.

[41]  Josef Urban,et al.  Overview and Evaluation of Premise Selection Techniques for Large Theory Mathematics , 2012, IJCAR.

[42]  David Aspinall,et al.  Proof General: A Generic Tool for Proof Development , 2000, TACAS.

[43]  Corina S. Pasareanu,et al.  Learning Assumptions for Compositional Verification , 2003, TACAS.

[44]  Gudmund Grov,et al.  Machine Learning in Proof General: Interfacing Interfaces , 2012, UITP.

[45]  Rajeev Alur,et al.  Learning-Based Symbolic Assume-Guarantee Reasoning with Automatic Decomposition , 2006, ATVA.

[46]  Kevin Leyton-Brown,et al.  Predicting Satisfiability at the Phase Transition , 2012, AAAI.

[47]  Geoffrey E. Hinton,et al.  Deep Learning , 2015, Nature.

[48]  Latifur Khan,et al.  A Machine Learning Approach to Android Malware Detection , 2012, 2012 European Intelligence and Security Informatics Conference.

[49]  Armin Biere,et al.  Evaluating CDCL Restart Schemes , 2018, POS@SAT.

[50]  Jürgen Schmidhuber,et al.  Learning Restart Strategies , 2007, IJCAI.

[51]  Jesse Alama,et al.  Premise Selection for Mathematics by Corpus Analysis and Kernel Methods , 2011, Journal of Automated Reasoning.

[52]  Demis Hassabis,et al.  Mastering the game of Go with deep neural networks and tree search , 2016, Nature.

[53]  Armin Biere,et al.  A survey of recent advances in SAT-based formal verification , 2005, International Journal on Software Tools for Technology Transfer.

[54]  James P. Bridge.  Machine learning and automated theorem proving , 2010 .

[55]  E. Finkelstein,et al.  Development and Validation of a Deep Learning System for Diabetic Retinopathy and Related Eye Diseases Using Retinal Images From Multiethnic Populations With Diabetes , 2017, JAMA.

[56]  Cezary Kaliszyk,et al.  Erratum to : Learning-Assisted Automated Reasoning with Flyspeck , 2014, Journal of Automated Reasoning.

[57]  Ofer Strichman,et al.  SAT Based Abstraction-Refinement Using ILP and Machine Learning Techniques , 2002, CAV.

[58]  Cezary Kaliszyk,et al.  Deep Network Guided Proof Search , 2017, LPAR.

[59]  Sakir Sezer,et al.  Analysis of Bayesian classification-based approaches for Android malware detection , 2016, IET Inf. Secur..

[60]  Stephan Schulz,et al.  E - a brainiac theorem prover , 2002, AI Commun..

[61]  Padmanabhan Krishnan,et al.  Machine learning for finding bugs: An initial report , 2017, 2017 IEEE Workshop on Machine Learning Techniques for Software Quality Evaluation (MaLTeSQuE).

[62]  Osamu Mizuno,et al.  An extension of fault-prone filtering using precise training and a dynamic threshold , 2008, MSR '08.

[63]  Sebastian G. Elbaum,et al.  Predicting accurate and actionable static analysis warnings , 2008, 2008 ACM/IEEE 30th International Conference on Software Engineering.

[64]  Lawrence D. Jackel,et al.  Explaining How a Deep Neural Network Trained with End-to-End Learning Steers a Car , 2017, ArXiv.

[65]  Claes Wohlin,et al.  Guidelines for snowballing in systematic literature studies and a replication in software engineering , 2014, EASE '14.

[66]  Demis Hassabis,et al.  Mastering Chess and Shogi by Self-Play with a General Reinforcement Learning Algorithm , 2017, ArXiv.

[67]  Edmund M. Clarke,et al.  SAT-Based Compositional Verification Using Lazy Learning , 2007, CAV.

[68]  Lawrence C. Paulson,et al.  Lightweight relevance filtering for machine-generated resolution problems , 2009, J. Appl. Log..

[69]  Josef Urban,et al.  MaLeS: A Framework for Automatic Tuning of Automated Theorem Provers , 2013, Journal of Automated Reasoning.

[70]  David H. Wolpert,et al.  No free lunch theorems for optimization , 1997, IEEE Trans. Evol. Comput..

[71]  Ofer Strichman,et al.  SAT-based counterexample-guided abstraction refinement , 2004, IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems.

[72]  Krzysztof Czarnecki,et al.  Adaptive Restart and CEGAR-Based Solver for Inverting Cryptographic Hash Functions , 2016, VSTTE.

[73]  Josef Urban,et al.  DeepMath - Deep Sequence Models for Premise Selection , 2016, NIPS.

[74]  Kevin Leyton-Brown,et al.  Performance Prediction and Automated Tuning of Randomized and Parametric Algorithms , 2006, CP.

[75]  Eric Mercer,et al.  Hardness for Explicit State Software Model Checking Benchmarks , 2007, Fifth IEEE International Conference on Software Engineering and Formal Methods (SEFM 2007).

[76]  Tobias Nipkow,et al.  Automatic Proof and Disproof in Isabelle/HOL , 2011, FroCoS.

[77]  Cezary Kaliszyk,et al.  A Learning-Based Fact Selector for Isabelle/HOL , 2016, Journal of Automated Reasoning.

[78]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[79]  Joseph Sifakis,et al.  Specification and verification of concurrent systems in CESAR , 1982, Symposium on Programming.

[80]  Kevin Leyton-Brown,et al.  Hierarchical Hardness Models for SAT , 2007, CP.

[81]  Hazel Duncan,et al.  The use of data-mining for the automatic formation of tactics , 2004 .

[82]  Konrad Rieck,et al.  Generalized vulnerability extrapolation using abstract syntax trees , 2012, ACSAC '12.

[83]  Thomas G. Dietterich.  What is machine learning? , 2020, Archives of Disease in Childhood.

[84]  Michail G. Lagoudakis,et al.  Learning to Select Branching Rules in the DPLL Procedure for Satisfiability , 2001, Electron. Notes Discret. Math..

[85]  Toby Walsh,et al.  Restart Strategy Selection Using Machine Learning Techniques , 2009, SAT.

[86]  Rajeev Alur,et al.  Symbolic Compositional Verification by Learning Assumptions , 2005, CAV.

[87]  W. Marsden.  I and J , 2012 .

[88]  Krzysztof Czarnecki,et al.  An Empirical Study of Branching Heuristics Through the Lens of Global Learning Rate , 2017, SAT.

[89]  Josef Urban,et al.  MizarMode - an integrated proof assistance tool for the Mizar way of formalizing mathematics , 2006, J. Appl. Log..

[90]  Edmund M. Clarke,et al.  Formal Methods: State of the Art and Future Directions Working Group Members , 1996 .

[91]  Richard S. Sutton,et al.  Reinforcement Learning: An Introduction , 1998, IEEE Trans. Neural Networks.

[92]  Haoze Wu,et al.  Improving SAT-solving with Machine Learning , 2017, SIGCSE.

[93]  Sunghun Kim,et al.  Reducing Features to Improve Code Change-Based Bug Prediction , 2013, IEEE Transactions on Software Engineering.