An efficient overloaded implementation of forward mode automatic differentiation in MATLAB

The Mad package described here facilitates the evaluation of first derivatives of multidimensional functions that are defined by computer codes written in MATLAB. The underlying algorithm is the well-known forward mode of automatic differentiation implemented via operator overloading on variables of the class fmad. The main distinguishing feature of this MATLAB implementation is the separation of the linear combination of derivative vectors into a separate derivative vector class derivvec. This allows for the straightforward performance optimization of the overall package. Additionally, by internally using a matrix (two-dimensional) representation of arbitrary dimension directional derivatives, we may utilize MATLAB's sparse matrix class to propagate sparse directional derivatives for MATLAB code which uses arbitrary dimension arrays. On several examples, the package is shown to be more efficient than Verma's ADMAT package [Verma 1998a].

[1]  Christian H. Bischof,et al.  Combining source transformation and operator overloading techniques to compute derivatives for MATLAB programs , 2002, Proceedings. Second IEEE International Workshop on Source Code Analysis and Manipulation.

[2]  Acknowledgments , 2006, Molecular and Cellular Endocrinology.

[3]  E. Hairer,et al.  Solving Ordinary Differential Equations II: Stiff and Differential-Algebraic Problems , 2010 .

[4]  Trevor J. Ringrose,et al.  Simplifying Multivariate Second-Order Response Surfaces by Fitting Constrained Models Using Automatic Differentiation , 2005, Technometrics.

[5]  T. Coleman,et al.  Structure and Efficient Jacobian Calculation , 1996 .

[6]  Thomas F. Coleman,et al.  ADMIT-1: automatic differentiation and MATLAB interface toolbox , 2000, TOMS.

[7]  Thomas F. Coleman,et al.  Structured automatic differentiation , 1998 .

[8]  A. Verma,et al.  ADMAT : Automatic differentiation in MATLAB using object oriented methods ∗ , 2007 .

[9]  A. Griewank,et al.  On the calculation of Jacobian matrices by the Markowitz rule , 1991 .

[10]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[11]  C. Bischof,et al.  Eecient Computation of Gradients and Jacobians by Dynamic Exploitation of Sparsity in Automatic Diierentiation , 1996 .

[12]  John R. Gilbert,et al.  Sparse Matrices in MATLAB: Design and Implementation , 1992, SIAM J. Matrix Anal. Appl..

[13]  Uwe Naumann,et al.  Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph , 2004, Math. Program..

[14]  Christian Bischof,et al.  Adifor 2.0: automatic differentiation of Fortran 77 programs , 1996 .

[15]  Martin Berz,et al.  Computational differentiation : techniques, applications, and tools , 1996 .

[16]  Trevor J. Ringrose,et al.  Improved Fitting of Constrained Multivariate Regression Models using Automatic Differentiation , 2002, COMPSTAT.

[17]  Robert Ketzscher,et al.  High-Level Interfaces for the MAD (Matlab Automatic Differentiation) Package. , 2004 .

[18]  E. Hairer,et al.  Stiff and differential-algebraic problems , 1991 .

[19]  Thomas F. Coleman,et al.  The Efficient Computation of Sparse Jacobian Matrices Using Automatic Differentiation , 1998, SIAM J. Sci. Comput..

[20]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[21]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[22]  Shaun A. Forth.  User guide for mad---a matlab automatic differentiation toolbox , 2001 .

[23]  Christian H. Bischof,et al.  ADIC: an extensible automatic differentiation tool for ANSI‐C , 1997, Softw. Pract. Exp..

[24]  Damien Brayshaw.  Use of numerical optimisation to determine on-limit handling behaviour of race cars , 2004 .

[25]  Stephen J. Wright,et al.  Numerical Optimization , 2018, Fundamental Statistical Inference.

[26]  Shaun A. Forth,et al.  Source Transformation for MATLAB Automatic Differentiation , 2006, International Conference on Computational Science.

[27]  Jeff Borggaard,et al.  On Efficient Solutions to the Continuous Sensitivity Equation Using Automatic Differentiation , 2000, SIAM J. Sci. Comput..

[28]  Christian Bischof,et al.  Automatic differentiation for MATLAB programs , 2003 .

[29]  Lawrence C. Rich,et al.  Automatic differentiation in MATLAB , 1992 .

[30]  Uwe Naumann,et al.  Efficient calculation of Jacobian matrices by optimized application of the chain rule to computational graphs , 1999 .

[31]  Lawrence F. Shampine,et al.  Using AD to solve BVPs in MATLAB , 2005, TOMS.

[32]  S. Forth,et al.  AD tools and prospects for optimal AD in CFD flux Jacobian calculations , 2000 .

[33]  Shaun A. Forth,et al.  User Guide for MAD - A Matlab Automatic Differentiation Package, TOMLAB/MAD,Version 1.4 The Forward Mode. , 2007 .

[34]  John D. Pryce,et al.  Jacobian code generated by source transformation and vertex elimination can be as efficient as hand-coding , 2004, TOMS.

[35]  Lawrence F. Shampine,et al.  The MATLAB ODE Suite , 1997, SIAM J. Sci. Comput..

[36]  Kenneth Holmström,et al.  The TOMLAB Optimization Environment , 2004 .