Verifying and improving Halide’s term rewriting system with program synthesis

Halide is a domain-specific language for high-performance image processing and tensor computations, widely adopted in industry. Internally, the Halide compiler relies on a term rewriting system to prove properties of code required for efficient and correct compilation. This rewrite system is a collection of handwritten transformation rules that incrementally rewrite expressions into simpler forms; the system requires high performance in both time and memory usage to keep compile times low, while operating over the undecidable theory of integers. In this work, we apply formal techniques to prove the correctness of existing rewrite rules and provide a guarantee of termination. Then, we build an automatic program synthesis system in order to craft new, provably correct rules from failure cases where the compiler was unable to prove properties. We identify and fix 4 incorrect rules as well as 8 rules which could give rise to infinite rewriting loops. We demonstrate that the synthesizer can produce better rules than hand-authored ones in five bug fixes, and describe four cases in which it has served as an assistant to a human compiler engineer. We further show that it can proactively improve weaknesses in the compiler by synthesizing a large number of rules without human supervision and showing that the enhanced ruleset lowers peak memory usage of compiled code without appreciably increasing compilation times.

[1]  Chang Liu,et al.  Term rewriting and all that , 2000, SOEN.

[2]  José C. Monteiro,et al.  Weakest Precondition Synthesis for Compiler Optimizations , 2014, VMCAI.

[3]  Charles Gregory Nelson,et al.  Techniques for program verification , 1979 .

[4]  Alexander Aiken,et al.  Stochastic superoptimization , 2012, ASPLOS '13.

[5]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[6]  John Regehr,et al.  Provably correct peephole optimizations with alive , 2015, PLDI.

[7]  Sergei Gorlatch,et al.  Software Artifact for: Achieving High-Performance the Functional Way - A Functional Pearl on Expressing High-Performance Optimizations as Rewrite Strategies , 2020 .

[8]  Dejan Jovanovic,et al.  Solving Nonlinear Integer Arithmetic with MCSAT , 2017, VMCAI.

[9]  Dinakar Dhurjati,et al.  Scaling up Superoptimization , 2016, ASPLOS.

[10]  Armando Solar-Lezama,et al.  The Sketching Approach to Program Synthesis , 2009, APLAS.

[11]  Isil Dillig,et al.  Synthesizing data structure transformations from input-output examples , 2015, PLDI.

[12]  BodikRastislav,et al.  A lightweight symbolic virtual machine for solver-aided host languages , 2014 .

[13]  Tao Xie,et al.  Learning stateful preconditions modulo a test generator , 2019, PLDI.

[14]  Hakjoo Oh,et al.  Optimizing homomorphic evaluation circuits by program synthesis and term rewriting , 2020, PLDI.

[15]  Emina Torlak,et al.  A Framework for Computer-Aided Design of Educational Domain Models , 2018, VMCAI.

[16]  G. Bernstein,et al.  Formal Semantics for the Halide Language , 2022, ArXiv.

[17]  Frédo Durand,et al.  Learning to optimize halide with tree search and random programs , 2019, ACM Trans. Graph..

[18]  MassalinHenry.  Superoptimizer: a look at the smallest program , 1987 .

[19]  Emina Torlak,et al.  Synthesizing interpretable strategies for solving puzzle games , 2017, FDG.

[20]  Santosh Nagarakatte,et al.  Alive-Infer: data-driven precondition inference for peephole optimizations in LLVM , 2017, PLDI.

[21]  D. Knuth,et al.  Simple Word Problems in Universal Algebras , 1983 .

[22]  Patrick Cousot,et al.  Systematic design of program analysis frameworks , 1979, POPL.

[23]  Armando Solar-Lezama,et al.  Automatic Generation of Formula Simplifiers based on Conditional Rewrite Rules , 2016, ArXiv.

[24]  Pavel Panchekha,et al.  Automatically improving accuracy for floating point expressions , 2015, PLDI.

[25]  Emina Torlak,et al.  A lightweight symbolic virtual machine for solver-aided host languages , 2014, PLDI.

[26]  Xueying Qin,et al.  Achieving high-performance the functional way: a functional pearl on expressing high-performance optimizations as rewrite strategies , 2020, Proc. ACM Program. Lang..

[27]  John Regehr,et al.  Souper: A Synthesizing Superoptimizer , 2017, ArXiv.

[28]  Saul Gorn,et al.  Handling the growth by definition of mechanical languages , 1967, AFIPS '67 (Spring).

[29]  Alberto Griggio,et al.  The MathSAT5 SMT Solver , 2013, TACAS.

[30]  Nikolaj Bjørner,et al.  Z3: An Efficient SMT Solver , 2008, TACAS.

[31]  Raymond T. Boute,et al.  The Euclidean definition of the functions div and mod , 1992, TOPL.