Numerical tools for geoscience computations: Semiautomatic differentiation—SD

This paper presents a differentiation method [referred to here as semiautomatic differentiation (SD)] based on generalization and extension of the Squire and Trapp formula for complex differentiation of real-valued functions. The performance of the generalized formulas for first-order derivatives is tested and compared with manual, automatic (AD), and finite difference (FD) techniques. My results show that, in terms of accuracy, the SD technique is competitive with AD, and in terms of implementation simplicity, it is identical to the FD method with the added advantage of being step-size insensitive and, hence, free from the step-size dilemma that plagues FD. Using central differencing in the complex plane, I extend the SD method to second-order derivatives, thus enabling approximation of the Hessians. Performance of the extension formulas is evaluated and compared with AD and FD methods. The results indicate that the differencing operation reduces the accuracy of the extension formulas by four to five orders of magnitude below that of the original Squire and Trapp formula. Nonetheless, compared to FD schemes, the SD method is six to seven orders of magnitude more accurate in all tests conducted. In addition, the extension formulas exhibit step-size (h) insensitive behavior over the entire h-range of the tests (1–10−30), indicating high numerical stability of the schemes. I show by examples that SD provides a complete differentiation system that is computationally stable, efficient, highly accurate, and easy to implement.

[1]  George Trapp,et al.  Using Complex Variables to Estimate Derivatives of Real Functions , 1998, SIAM Rev..

[2]  Thomas Bewley,et al.  On the extension of the complex-step derivative technique to pseudospectral algorithms , 2003 .

[3]  David Elizondo,et al.  Automatic versus manual model differentiation to compute sensitivities and solve non-linear inverse problems , 2002 .

[4]  Jorge J. Moré,et al.  Automatic Differentiation Tools in Optimization Software , 2000, ArXiv.

[5]  R. Blakely.  Potential theory in gravity and magnetic applications , 1996 .

[6]  Howard Mark,et al.  54 – Derivatives in Spectroscopy: Part 1 – The Behavior of the Derivative , 2003 .

[7]  D. Parasnis,et al.  Principles of Applied Geophysics , 1962 .

[8]  J. N. Lyness,et al.  Numerical Differentiation of Analytic Functions , 1967 .

[9]  Joaquim R. R. A. Martins,et al.  The complex-step derivative approximation , 2003, TOMS.

[10]  Joaquim R. R. A. Martins,et al.  AN AUTOMATED METHOD FOR SENSITIVITY ANALYSIS USING COMPLEX VARIABLES , 2000 .

[11]  J. N. Lyness.  Numerical algorithms based on the theory of complex variable , 1967, ACM National Conference.

[12]  J. Uhlig.  C. Forsythe and C. B. Moler, Computer Solution of Linear Algebraic Systems. (Series in Automatic Computation) XI + 148 S. Englewood Cliffs, N.J. 1967. Prentice-Hall, Inc. Preis geb. 54 s. net , 1972 .

[13]  Peter A. Vanrolleghem,et al.  Avoiding the Finite Difference Sensitivity Analysis Deathtrap by Using the Complex-step Derivative Approximation Technique , 2006 .

[14]  Bengt Fornberg,et al.  Numerical Differentiation of Analytic Functions , 1981, TOMS.

[15]  V. N. Vatsa,et al.  Computation of sensitivity derivatives of Navier-Stokes equations using complex variables , 2000 .

[16]  G. Forsythe,et al.  Computer solution of linear algebraic systems , 1969 .

[17]  Thomas F. Coleman,et al.  ADMIT-1: automatic differentiation and MATLAB interface toolbox , 2000, TOMS.

[18]  J. C. Newman,et al.  Computationally efficient, numerically exact design space derivatives via the complex Taylor's series expansion method , 2003 .