On Optimal DAG Reversal

Runs of numerical computer programs can be visualized as directed acyclic graphs (DAGs). We consider the problem of restoring the intermediate values computed by such a program (the vertices in the DAG) in reverse order for a given upper bound on the available memory. The minimization of the associated computational cost in terms of the number of performed arithmetic operations is shown to be NP-complete. The reversal of the data-flow finds application, for example, in the efficient evaluation of adjoint numerical programs. We derive special cases of numerical programs that require the intermediate values exactly in reverse order, thus establishing the NP-completeness of the optimal adjoint computation problem. Last but not least we review some state-of-theart approaches to efficient data-flow reversal taken by existing software tools for automatic differentiation.

[1]  David S. Johnson,et al.  Computers and In stractability: A Guide to the Theory of NP-Completeness. W. H Freeman, San Fran , 1979 .

[2]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[3]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[4]  U. Naumann,et al.  Intraprocedural Adjoint Code Generated by the Differentiation-Enabled NAGWare Fortran Compiler , 2006 .

[5]  Patrick Heimbach,et al.  An efficient exact adjoint of the parallel MIT General Circulation Model, generated via automatic differentiation , 2005, Future Gener. Comput. Syst..

[6]  Andrea Walther,et al.  Program reversal schedules for single and multi-processor machines , 1999 .

[7]  Robert E. Tarjan,et al.  The Pebbling Problem is Complete in Polynomial Space , 1980, SIAM J. Comput..

[8]  Martin Berz,et al.  Computational differentiation : techniques, applications, and tools , 1996 .

[9]  Andreas Griewank,et al.  New results on program reversals , 2000 .

[10]  Uwe Naumann,et al.  "To be recorded" analysis in reverse-mode automatic differentiation , 2005, Future Gener. Comput. Syst..

[11]  Mauricio Araya-Polo,et al.  The Adjoint Data-Flow Analyses: Formalization, Properties, and Applications , 2006 .

[12]  Uwe Naumann,et al.  Computing Adjoints with the NAGWare Fortran 95 Compiler , 2006 .

[13]  Jean Utke,et al.  Source Templates for the Automatic Generation of Adjoint Code Through Static Call Graph Reversal , 2005, International Conference on Computational Science.

[14]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[15]  Robert E. Tarjan,et al.  The pebbling problem is complete in polynomial space , 1979, SIAM J. Comput..

[16]  R. Stephenson.  A and V , 1962, The British journal of ophthalmology.

[17]  G. G. Stokes.  "J." , 1890, The New Yale Book of Quotations.

[18]  J. Utke,et al.  OPTIMALITY-PRESERVING ELIMINATION OF LINEARITIES IN JACOBIAN ACCUMULATION , 2005 .

[19]  Jean Utke,et al.  Efficient reversal of the intraprocedural flow of control in adjoint computations , 2006, J. Syst. Softw..

[20]  Thomas Kaminski,et al.  Recipes for adjoint code construction , 1998, TOMS.

[21]  Laurent Hascoët,et al.  TAPENADE 2.1 user's guide , 2004 .