Sparse Jacobian Computation in Automatic Differentiation by Static Program Analysis

A major difficulty in quickly computing Jacobians by Automatic Differentiation is to deal with the nonzero structures of sparse matrices. We propose to detect the sparsity structure of Jacobians by static program analysis. The method consists in traversing the data dependence graph extended with the control-flow of the program and computing relations between array regions. Then, we safely extract informations about the dependences from program inputs to program outputs. The generation of the derived program uses these informations to produce a better result. We eventually, introduce the Automatic Differentiation tool Odyssee and present some benchmark tests.

[1]  Sergio Pissanetzky,et al.  Sparse Matrix Technology , 1984 .

[2]  John D. Ramsdell,et al.  Estimation of Sparse Jacobian Matrices , 1983 .

[3]  François Irigoin,et al.  Exact versus Approximate Array Region Analyses , 1996, LCPC.

[4]  J. J. Moré,et al.  Estimation of sparse jacobian matrices and graph coloring problems , 1983 .

[5]  Ken Kennedy,et al.  A technique for summarizing data access and its use in parallelism enhancing transformations , 1989, PLDI '89.

[6]  Aart J. C. Bik,et al.  Automatic Data Structure Selection and Transformation for Sparse Matrix Computations , 1996, IEEE Trans. Parallel Distributed Syst..

[7]  Christian Bischof,et al.  ADIC: an extensible automatic differentiation tool for ANSI-C , 1997 .

[8]  Utpal Banerjee,et al.  Loop Transformations for Restructuring Compilers: The Foundations , 1993, Springer US.

[9]  Patrick Cousot,et al.  Abstract interpretation: a unified lattice model for static analysis of programs by construction or approximation of fixpoints , 1977, POPL.

[10]  Alfred V. Aho,et al.  Compilers: Principles, Techniques, and Tools , 1986, Addison-Wesley series in computer science / World student series edition.

[11]  Ken Kennedy,et al.  An Implementation of Interprocedural Bounded Regular Section Analysis , 1991, IEEE Trans. Parallel Distributed Syst..

[12]  A. Griewank,et al.  Automatic differentiation of algorithms : theory, implementation, and application , 1994 .

[13]  Christian H. Bischof,et al.  Computing Gradients in Large-Scale Optimization Using Automatic Differentiation , 1997, INFORMS J. Comput..

[14]  Barbara M. Chapman,et al.  Supercompilers for parallel and vector computers , 1990, ACM Press frontier series.

[15]  Andreas Griewank,et al.  Algorithm 755: ADOL-C: a package for the automatic differentiation of algorithms written in C/C++ , 1996, TOMS.

[16]  William Pugh,et al.  An Exact Method for Analysis of Value-based Array Data Dependences , 1993, LCPC.

[17]  Nicole Rostaing,et al.  Automatic differentiation in Odyssée , 1993 .