Optimal accumulation of Jacobian matrices by elimination methods on the dual computational graph

Abstract.The accumulation of the Jacobian matrix F’ of a vector function can be regarded as a transformation of its linearized computational graph into a subgraph of the directed complete bipartite graph Kn,m. This transformation can be performed by applying different elimination techniques that may lead to varying costs for computing F’. This paper introduces face elimination as the basic technique for accumulating Jacobian matrices by using a minimal number of arithmetic operations. Its superiority over both edge and vertex elimination methods is shown. The intention is to establish the conceptual basis for the ongoing development of algorithms for optimizing the computation of Jacobian matrices.

[1]  M. Powell,et al.  On the Estimation of Sparse Jacobian Matrices , 1974 .

[2]  A. Griewank,et al.  On the calculation of Jacobian matrices by the Markowitz rule , 1991 .

[3]  John R. Gilbert,et al.  A Note on the NP-Completeness of Vertex Elimination on Directed Graphs , 1980, SIAM J. Algebraic Discret. Methods.

[4]  N. Metropolis,et al.  Equation of State Calculations by Fast Computing Machines , 1953, Resonance.

[5]  Charles L. Lawson,et al.  Basic Linear Algebra Subprograms for Fortran Usage , 1979, TOMS.

[6]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[7]  Ellis Horowitz,et al.  Fundamentals of Computer Algorithms , 1978 .

[8]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[9]  Uwe Naumann,et al.  Markowitz-Type Heuristics for Computing Jacobian Matrices Efficiently , 2003, International Conference on Computational Science.

[10]  Uwe Naumann,et al.  Cheaper Jacobians by Simulated Annealing , 2002, SIAM J. Optim..

[11]  Webb Miller,et al.  Software for Roundoff Analysis of Matrix Algorithms , 1980 .

[12]  Romesh M. Jessani,et al.  Comparison of Single- and Dual-Pass Multiply-Add Fused Floating-Point Units , 1998, IEEE Trans. Computers.

[13]  Andreas Griewank,et al.  Accumulating Jacobians as chained sparse matrix products , 2003, Math. Program..

[14]  Martin Berz,et al.  Computational differentiation : techniques, applications, and tools , 1996 .

[15]  Uwe Naumann,et al.  Prospects for Simulated Annealing Algorithms in Automatic Differentiation , 2001, SAGA.

[16]  John D. Pryce,et al.  Performance Issues for Vertex Elimination Methods in Computing Jacobians Using Automatic Differentiation , 2002, International Conference on Computational Science.

[17]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[18]  U. Naumann.  Elimination techniques for cheap Jacobians , 2000 .

[19]  C. Bischof,et al.  Efficient computation of gradients and Jacobians by dynamic exploitation of sparsity in automatic differentiation , 1996 .

[20]  Mohammad R. Haghighat,et al.  Hierarchical approaches to automatic differentiation , 1996 .

[21]  Giorgio Gambosi,et al.  Complexity and Approximation , 1999, Springer Berlin Heidelberg.

[22]  Giorgio Gambosi,et al.  Complexity and approximation: combinatorial optimization problems and their approximability properties , 1999 .

[23]  D. Rose,et al.  Algorithmic aspects of vertex elimination on directed graphs. , 1975 .

[24]  R. E. Wengert,et al.  A simple automatic derivative evaluation program , 1964, Commun. ACM.

[25]  L. Perelman,et al.  Hydrostatic, quasi‐hydrostatic, and nonhydrostatic ocean modeling , 1997 .

[26]  S. Forth,et al.  AD tools and prospects for optimal AD in CFD flux Jacobian calculations , 2000 .

[27]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[28]  Uwe Naumann,et al.  Efficient calculation of Jacobian matrices by optimized application of the chain rule to computational graphs , 1999 .