Dynamic Programming via Static Incrementalization

Dynamic programming is an important algorithm design technique. It is used for problems whose solutions involve recursively solving subproblems that share subsubproblems. While a straightforward recursive program solves common subsubproblems repeatedly, a dynamic programming algorithm solves every subsubproblem just once, saves the result, and reuses it when the subsubproblem is encountered again. This can reduce the time complexity from exponential to polynomial. This paper describes a systematic method for transforming programs written as straightforward recursions into programs that use dynamic programming. The method extends the original program to cache all possibly computed values, incrementalizes the extended program with respect to an input increment to use and maintain all cached results, prunes out cached results that are not used in the incremental computation, and uses the resulting incremental program to form an optimized new program. Incrementalization statically exploits semantics of both control structures and data structures and maintains as invariants equalities characterizing cached results. It provides the basis of a general method for achieving drastic program speedups. Compared with previous methods that perform memoization or tabulation, the method based on incrementalization is more powerful and systematic. It has been implemented in a prototype system CACHET and applied to numerous problems and succeeded on all of them.

[1]  Yanhong A. Liu,et al.  Systematic Derivation of Incremental Programs , 1995, Sci. Comput. Program..

[2]  R. Bellman.  Dynamic programming. , 1957, Science.

[3]  Robert Paige,et al.  Symbolic Finite Differencing - Part I , 1990, ESOP.

[4]  Douglas R. Smith,et al.  KIDS: A Semiautomatic Program Development System , 1990, IEEE Trans. Software Eng..

[5]  Akihiko Takano,et al.  Tupling calculation eliminates multiple data traversals , 1997, ICFP '97.

[6]  Mads Rosendahl,et al.  Automatic complexity analysis , 1989, FPCA.

[7]  Oege de Moor.  A Generic Program for Sequential Decision Processes , 1995, PLILP.

[8]  Alberto Pettorossi,et al.  A powerful strategy for deriving efficient programs by transformation , 1984, LFP '84.

[9]  Yanhong A. Liu,et al.  Eliminating Dead Code on Recursive Data , 1999, SAS.

[10]  Robert Paige,et al.  Program Derivation by Fixed Point Computation , 1989, Sci. Comput. Program..

[11]  Richard S. Bird.  The promotion and accumulation strategies in transformational programming , 1984, TOPL.

[12]  Helmuth Partsch,et al.  Specification and Transformation of Programs - A Formal Approach to Software Development , 1990, Texts and Monographs in Computer Science.

[13]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[14]  Analysis and caching of dependencies , 1996, ICFP '96.

[15]  Richard S. Bird,et al.  From Dynamic Programming to Greedy Algorithms , 1993, Formal Program Development.

[16]  John Hughes,et al.  Lazy Memo-functions , 1985, FPCA.

[17]  Robert Paige,et al.  Programming with Invariants , 1986, IEEE Software.

[18]  Wei-Ngan Chin,et al.  A bounds inference method for vector-based memoization , 1997, ICFP '97.

[19]  Tim Teitelbaum,et al.  Incremental reduction in the lambda calculus , 1990, LISP and Functional Programming.

[20]  Yanhong A. Liu,et al.  Automating Derivation of Incremental Programs , 1998, ICFP.

[21]  Paul Walton Purdom,et al.  The Analysis of Algorithms , 1995 .

[22]  Yanhong A. Liu,et al.  CACHET: an interactive, incremental-attribution-based program transformation system for deriving incremental programs , 1995, Proceedings 1995 10th Knowledge-Based Software Engineering Conference.

[23]  Richard S. Bird,et al.  Tabulation Techniques for Recursive Programs , 1980, CSUR.

[24]  Norman H. Cohen.  Eliminating Redundant Recursive Calls. , 1983, TOPL.

[25]  Eerke A. Boiten,et al.  Improving Recursive Functions by Inverting the Order of Evaluation , 1992, Sci. Comput. Program..

[26]  Yanhong A. Liu,et al.  Dependence analysis for recursive data , 1998, Proceedings of the 1998 International Conference on Computer Languages (Cat. No.98CB36225).

[27]  Yanhong A. Liu,et al.  Automatic Accurate Live Memory Analysis for Garbage-Collected Languages , 2001, OM '01.

[28]  Thomas Reps,et al.  The Synthesizer Generator: A System for Constructing Language-Based Editors , 1988 .

[29]  Mashe Sniedovich,et al.  Dynamic Programming , 1991 .

[30]  Man-Tak Shing,et al.  Computation of Matrix Chain Products. Part II , 1984, SIAM J. Comput..

[31]  William Pugh.  An improved replacement strategy for function caching , 1988, LFP '88.

[32]  William L. Scherlis,et al.  Program improvement by internal specialization , 1981, POPL '81.

[33]  Oege de Moor,et al.  Bridging the Algorithm Gap: A Linear-Time Functional Program for Paragraph Formatting , 1999, Sci. Comput. Program..

[34]  DONALD MICHIE,et al.  “Memo” Functions and Machine Learning , 1968, Nature.

[35]  Ben Wegbreit.  Goal-Directed Program Transformation , 1976, IEEE Trans. Software Eng..

[36]  Robert Paige,et al.  Finite Differencing of Computable Expressions , 1982, TOPL.

[37]  Friedrich L. Bauer,et al.  Formal Program Construction by Transformations-Computer-Aided, Intuition-Guided Programming , 1989, IEEE Trans. Software Eng..

[38]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[39]  Ben Wegbreit,et al.  Mechanical program analysis , 1975, CACM.

[40]  Wei-Ngan Chin,et al.  Tupling Functions with Multiple Recursion Parameters , 1993, WSA.

[41]  William Pugh,et al.  A practical algorithm for exact array dependence analysis , 1992, CACM.

[42]  Yanhong A. Liu,et al.  Static caching for incremental computation , 1998, TOPL.

[43]  Alberto Pettorossi,et al.  Program derivation via list introduction , 1997, Algorithmic Languages and Calculi.

[44]  John Darlington,et al.  A Transformation System for Developing Recursive Programs , 1977, J. ACM.

[45]  Sharon Curtis.  Dynamic programming: a different perspective , 1997, Algorithmic Languages and Calculi.

[46]  Wei-Ngan Chin.  Towards an automated tupling strategy , 1993, PEPM '93.

[47]  William Pugh,et al.  Incremental computation via function caching , 1989, POPL '89.

[48]  Robert M. Keller,et al.  Applicative caching , 1986, TOPL.

[49]  William Pugh,et al.  The Omega test: A fast and practical integer programming algorithm for dependence analysis , 1991, Proceedings of the 1991 ACM/IEEE Conference on Supercomputing (Supercomputing '91).

[50]  Mitchell Wand,et al.  Recursive programming through table look-up , 1976, SYMSAC '76.

[51]  Yanhong A. Liu,et al.  Discovering auxiliary information for incremental computation , 1996, POPL '96.

[52]  Yanhong A. Liu,et al.  Solving Regular Tree Grammar Based Constraints , 2001, SAS.

[53]  Jack Mostow,et al.  Automating Program Speedup by Deciding What to Cache , 1985, IJCAI.

[54]  Yanhong A. Liu,et al.  Automatic Accurate Cost-Bound Analysis for High-Level Languages , 2001, IEEE Trans. Computers.

[55]  Yanhong A. Liu,et al.  Loop optimization for aggregate array computations , 1998, Proceedings of the 1998 International Conference on Computer Languages (Cat. No.98CB36225).

[56]  Yanhong A. Liu,et al.  Principled strength reduction , 1997, Algorithmic Languages and Calculi.

[57]  Alberto Pettorossi,et al.  Rules and strategies for transforming functional and logic programs , 1996, CSUR.