DAG reversal is NP-complete

Runs of numerical computer programs can be visualized as directed acyclic graphs (DAGs). We consider the problem of restoring the intermediate values computed by such a program (the vertices in the DAG) in reverse order for a given upper bound on the available memory. The minimization of the associated computational cost in terms of the number of performed arithmetic operations is shown to be NP-complete. The reversal of the data-flow finds application, for example, in the efficient evaluation of adjoint numerical programs. We derive special cases of numerical programs that require the intermediate values exactly in reverse order, thus establishing the NP-completeness of the optimal adjoint computation problem. Last but not least we review some state-of-the-art approaches to efficient data-flow reversal taken by existing software tools for automatic differentiation.

[1]  Andreas Griewank,et al.  New results on program reversals , 2000 .

[2]  Uwe Naumann,et al.  "To be recorded" analysis in reverse-mode automatic differentiation , 2005, Future Gener. Comput. Syst..

[3]  Andreas Griewank,et al.  Evaluating derivatives - principles and techniques of algorithmic differentiation, Second Edition , 2000, Frontiers in applied mathematics.

[4]  Richard J. Fateman,et al.  Automatic Differentiation of Algorithms: Theory, Implementation, and Application (Andreas Griewank and George F. Corliss, eds.) , 1993, SIAM Rev..

[5]  Robert E. Tarjan,et al.  The Pebbling Problem is Complete in Polynomial Space , 1980, SIAM J. Comput..

[6]  Martin Berz,et al.  Computational differentiation : techniques, applications, and tools , 1996 .

[7]  Andreas Griewank,et al.  Automatic Differentiation of Algorithms: From Simulation to Optimization , 2000, Springer New York.

[8]  Patrick Heimbach,et al.  An efficient exact adjoint of the parallel MIT General Circulation Model, generated via automatic differentiation , 2005, Future Gener. Comput. Syst..

[9]  Jean Utke,et al.  Efficient reversal of the intraprocedural flow of control in adjoint computations , 2006, J. Syst. Softw..

[10]  Andrea Walther,et al.  Program reversal schedules for single and multi-processor machines , 1999 .

[11]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[12]  J. Utke,et al.  OPTIMALITY-PRESERVING ELIMINATION OF LINEARITIES IN JACOBIAN ACCUMULATION , 2005 .

[13]  Uwe Naumann,et al.  Computing Adjoints with the NAGWare Fortran 95 Compiler , 2006 .

[14]  Jean Utke,et al.  Source Templates for the Automatic Generation of Adjoint Code Through Static Call Graph Reversal , 2005, International Conference on Computational Science.

[15]  U. Naumann,et al.  Intraprocedural Adjoint Code Generated by the Differentiation-Enabled NAGWare Fortran Compiler , 2006 .

[16]  Laurent Hascoët,et al.  TAPENADE 2.1 user's guide , 2004 .

[17]  Thomas Kaminski,et al.  Recipes for adjoint code construction , 1998, TOMS.

[18]  Mauricio Araya-Polo,et al.  The Adjoint Data-Flow Analyses: Formalization, Properties, and Applications , 2006 .