Estimating Derivatives of Noisy Simulations

We employ recent work on computational noise to obtain near-optimal difference estimates of the derivative of a noisy function. Our analysis relies on a stochastic model of the noise without assuming a specific form of distribution. We use this model to derive theoretical bounds for the errors in the difference estimates and obtain an easily computable difference parameter that is provably near-optimal. Numerical results closely resemble the theory and show that we obtain accurate derivative estimates even when the noisy function is deterministic.

[1]  W. Marsden.  I and J , 2012 .

[2]  J MoréJorge,et al.  Estimating Derivatives of Noisy Simulations , 2012 .

[3]  Martin Bücker,et al.  Automatic differentiation : applications, theory, and implementations , 2006 .

[4]  Siegfried M. Rump,et al.  INTLAB - INTerval LABoratory , 1998, SCAN.

[5]  D. Keyes,et al.  Jacobian-free Newton-Krylov methods: a survey of approaches and applications , 2004 .

[6]  Timothy A. Davis,et al.  The university of Florida sparse matrix collection , 2011, TOMS.

[7]  Carol S. Woodward,et al.  Enabling New Flexibility in the SUNDIALS Suite of Nonlinear and Differential/Algebraic Equation Solvers , 2020, ACM Trans. Math. Softw..

[8]  Uwe Naumann,et al.  Automatic Differentiation: Applications, Theory, and Implementations (Lecture Notes in Computational Science and Engineering) , 2006 .

[9]  P. Gill,et al.  Computing Forward-Difference Intervals for Numerical Optimization , 1983 .

[10]  Chih-Jen Lin,et al.  Newton's Method for Large Bound-Constrained Optimization Problems , 1999, SIAM J. Optim..

[11]  Aaas News,et al.  Book Reviews , 1893, Buffalo Medical and Surgical Journal.

[12]  Philip E. Gill,et al.  Practical optimization , 1981 .

[13]  R. Haftka,et al.  Review of options for structural design sensitivity analysis. Part 1: Linear systems , 2005 .

[14]  Herbert Hamers,et al.  Gradient Estimation Schemes for Noisy Functions , 2003 .

[15]  Stefan M. Wild,et al.  Estimating Computational Noise , 2011, SIAM J. Sci. Comput..

[16]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[17]  Mei Han An,et al.  accuracy and stability of numerical algorithms , 1991 .

[18]  Jean Utke,et al.  Advances in Automatic Differentiation , 2008 .