NNSMT: Deep Neural Networks for SMT Solvers Fuzzing

SMT solvers are important tools in the field of software engineering, which is often used to determine the satisfiability of formulas in formal methods, such as software verification, program synthesis, program verification, etc. However, due to their complex implementations, solvers may contain critical bugs that lead to unsound results. Fuzz testing is an effective method to detect new errors or security vulnerabilities in SMT solvers, but the existing SMT solvers fuzzing are not scalable enough and too dependent on the complex grammar rules provided by human beings. To tackle this challenge, we propose a grammar-based fuzzing tool called NNSMT. NNSMT uses the attention mechanism to capture the long-distance dependencies, and adopts an encoder-decoder architecture with the embedding of these long-distance dependencies to build a language model of regular programs. Moreover, it uses the language model to automatically and continuously generates well-formed SMT test programs. We use this set of generated program to fuzz off-the-shelf SMT solvers(e.g., Z3, CVC4). We present a detailed case study to analyze the pass rate and coverage improvement of the generated SMT program. we analyze the performance of NNSMT under five new generation strategies and two sampling methods. Extensive experiments show that the average pass rate of SMT test programs generated by NNSMT is more than 85%, and the code coverage of SMT solvers has been significantly improved. In our preliminary study, we found 3 bugs of Z3 solver, all of which are actively being addressed by developers.