Smooth interpretation

We present smooth interpretation, a method to systematically approximate numerical imperative programs by smooth mathematical functions. This approximation facilitates the use of numerical search techniques like gradient descent for program analysis and synthesis. The method extends to programs the notion of Gaussian smoothing, a popular signal-processing technique that filters out noise and discontinuities from a signal by taking its convolution with a Gaussian function. In our setting, Gaussian smoothing executes a program according to a probabilistic semantics; the execution of program P on an input x after Gaussian smoothing can be summarized as follows: (1) Apply a Gaussian perturbation to x -- the perturbed input is a random variable following a normal distribution with mean x. (2) Compute and return the expected output of P on this perturbed input. Computing the expectation explicitly would require the execution of P on all possible inputs, but smooth interpretation bypasses this requirement by using a form of symbolic execution to approximate the effect of Gaussian smoothing on P. The result is an efficient but approximate implementation of Gaussian smoothing of programs. Smooth interpretation has the effect of attenuating features of a program that impede numerical searches of its input space -- for example, discontinuities resulting from conditional branches are replaced by continuous transitions. We apply smooth interpretation to the problem of synthesizing values of numerical control parameters in embedded control applications. This problem is naturally formulated as one of numerical optimization: the goal is to find parameter values that minimize the error between the resulting program and a programmer-provided behavioral specification. Solving this problem by directly applying numerical optimization techniques is often impractical due to the discontinuities in the error function. By eliminating these discontinuities, smooth interpretation makes it possible to search the parameter space efficiently by means of simple gradient descent. Our experiments demonstrate the value of this strategy in synthesizing parameters for several challenging programs, including models of an automated gear shift and a PID controller.

[1]  John A. Nelder,et al.  A Simplex Method for Function Minimization , 1965, Comput. J..

[2]  Garret N. Vanderplaats,et al.  Numerical optimization techniques , 1987 .

[3]  Herbert Wiklicky,et al.  Probabilistic Abstract Interpretation and Statistical Testing (Extended Abstract) , 2002 .

[4]  Dexter Kozen,et al.  Semantics of probabilistic programs , 1979, 20th Annual Symposium on Foundations of Computer Science (sfcs 1979).

[5]  Richard J. Lipton,et al.  Defining Software by Continuous, Smooth Functions , 1991, IEEE Trans. Software Eng..

[6]  Sanjit A. Seshia,et al.  Combinatorial sketching for finite programs , 2006, ASPLOS XII.

[7]  Noam Nisan,et al.  On the degree of boolean functions as real polynomials , 1992, STOC '92.

[8]  Sumit Gulwani,et al.  Discovering affine equalities using random interpretation , 2003, POPL '03.

[9]  John C. Russ,et al.  The Image Processing Handbook , 2016, Microscopy and Microanalysis.

[10]  M. E. Galassi,et al.  GNU SCIENTI C LIBRARY REFERENCE MANUAL , 2005 .

[11]  Bruce H. Krogh,et al.  Parameter Synthesis for Hybrid Systems with an Application to Simulink Models , 2009, HSCC.

[12]  David Lorge Parnas.  Software aspects of strategic defense systems , 1985, SOEN.

[13]  Sumit Kumar Jha,et al.  A Counterexample-Guided Approach to Parameter Synthesis for Linear Hybrid Automata , 2008, HSCC.

[14]  Thomas A. Henzinger,et al.  Using HyTech to Synthesize Control Parameters for a Steam Boiler , 1995, Formal Methods for Industrial Applications.

[15]  David Andre,et al.  State abstraction for programmable reinforcement learning agents , 2002, AAAI/IAAI.

[16]  Armando Solar-Lezama,et al.  Programming by sketching for bit-streaming programs , 2005, PLDI '05.

[17]  Noam Nisan,et al.  On the degree of boolean functions as real polynomials , 2005, computational complexity.

[18]  Sumit Gulwani,et al.  Continuity analysis of programs , 2010, POPL '10.

[19]  Johann Schumann,et al.  Under Consideration for Publication in J. Functional Programming Autobayes: a System for Generating Data Analysis Programs from Statistical Models , 2022 .

[20]  Christopher M. Bishop,et al.  Neural networks for pattern recognition , 1995 .

[21]  Michael J. A. Smith.  Probabilistic Abstract Interpretation of Imperative Programs using Truncated Normal Distributions , 2008, Electron. Notes Theor. Comput. Sci..

[22]  David Monniaux,et al.  Abstract Interpretation of Probabilistic Semantics , 2000, SAS.

[23]  Thomas A. Henzinger,et al.  The Algorithmic Analysis of Hybrid Systems , 1995, Theor. Comput. Sci..

[24]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.