Limitations of Incremental Dynamic Programming

We consider so-called “incremental” dynamic programming algorithms, and are interested in the number of subproblems produced by them. The classical dynamic programming algorithm for the Knapsack problem is incremental, produces nK subproblems and nK2 relations (wires) between the subproblems, where n is the number of items, and K is the knapsack capacity. We show that any incremental algorithm for this problem must produce about nK subproblems, and that about nKlogK wires (relations between subproblems) are necessary. This holds even for the Subset-Sum problem. We also give upper and lower bounds on the number of subproblems needed to approximate the Knapsack problem. Finally, we show that the Maximum Bipartite Matching problem and the Traveling Salesman problem require exponential number of subproblems. The goal of this paper is to leverage ideas and results of boolean circuit complexity for proving lower bounds on dynamic programming.

[1]  Mark Jerrum,et al.  Some Exact Complexity Results for Straight-Line Computations over Semirings , 1982, JACM.

[2]  David Blokh,et al.  The Maximum Traveling Salesman Problem on Banded Matrices , 2001, Int. J. Found. Comput. Sci..

[3]  Bernhard Korte,et al.  Exponential Lower Bounds on a Class of Knapsack Algorithms , 1981, Math. Oper. Res..

[4]  Georg Schnitger,et al.  Yet harder knapsack problems , 2011, Theor. Comput. Sci..

[5]  Russell Impagliazzo,et al.  Models of Greedy Algorithms for Graph Problems , 2004, SODA '04.

[6]  Allan Borodin,et al.  The Power of Priority Algorithms for Facility Location and Set Cover , 2004, Algorithmica.

[7]  Alexander A. Razborov,et al.  Neither Reading Few Bits Twice Nor Reading Illegally Helps Much , 1998, Discret. Appl. Math..

[8]  Alfred V. Aho,et al.  The Design and Analysis of Computer Algorithms , 1974 .

[9]  M. Hung,et al.  A hard knapsack problem , 1988 .

[10]  P. Helman,et al.  A Comprehensive Model of Dynamic Programming , 1985 .

[11]  Avi Wigderson,et al.  Monotone circuits for connectivity require super-logarithmic depth , 1990, STOC '88.

[12]  Ingo Wegener,et al.  Branching Programs and Binary Decision Diagrams , 1987 .

[13]  Miklós Ajtai,et al.  Determinism versus Nondeterminism for Linear Time RAMs with Memory Restrictions , 2002, J. Comput. Syst. Sci..

[14]  Paul Helman,et al.  A common schema for dynamic programming and branch and bound algorithms , 1989, JACM.

[15]  R. Bellman.  COMBINATORIAL PROCESSES AND DYNAMIC PROGRAMMING , 1958 .

[16]  L. R. Kerr.  The Effect of Algebraic Structure on the Computational Complexity of Matrix Multiplication , 1970 .

[17]  Marcin Mucha,et al.  35/44-approximation for Asymmetric Maximum TSP with Triangle Inequality , 2009, Algorithmica.

[18]  M. Held,et al.  A dynamic programming approach to sequencing problems , 1962, ACM National Meeting.

[19]  Vasek Chvátal,et al.  Hard Knapsack Problems , 1980, Oper. Res..

[20]  Russell Impagliazzo,et al.  A Stronger Model of Dynamic Programming Algorithms , 2009, Algorithmica.

[21]  Béla Bollobás,et al.  Proving Integrality Gaps without Knowing the Linear Program , 2006, Theory Comput..

[22]  Carsten Lund,et al.  Proof verification and the hardness of approximation problems , 1998, JACM.

[23]  Allan Borodin,et al.  Toward a Model for Backtracking and Dynamic Programming , 2005, 20th Annual IEEE Conference on Computational Complexity (CCC'05).

[24]  E. Balas,et al.  New classes of efficiently solvable generalized Traveling Salesman Problems , 1999, Ann. Oper. Res..

[25]  Allan Borodin,et al.  Priority Algorithms for Graph Optimization Problems , 2004, WAOA.

[26]  Allan Borodin,et al.  On lower bounds for read-k-times branching programs , 2005, computational complexity.

[27]  Oded Regev.  Priority algorithms for makespan minimization in the subset model , 2002, Inf. Process. Lett..

[28]  Stasys Jukna,et al.  A Note on Read-k Times Branching Programs , 1994, RAIRO Theor. Informatics Appl..

[29]  David S. Johnson,et al.  `` Strong '' NP-Completeness Results: Motivation, Examples, and Implications , 1978, JACM.

[30]  Abraham P. Punnen,et al.  The traveling salesman problem and its variations , 2007 .

[31]  Agustín Bompadre,et al.  Exponential Lower Bounds on the Complexity of a Class of Dynamic Programs for Combinatorial Optimization Problems , 2012, Algorithmica.

[32]  Stasys Jukna,et al.  Boolean Function Complexity Advances and Frontiers , 2012, Bull. EATCS.

[33]  Michael E. Saks,et al.  Time-space trade-off lower bounds for randomized computation of decision problems , 2003, JACM.

[34]  Allan Borodin,et al.  (Incremental) Priority Algorithms , 2002, SODA '02.

[35]  Stasys Jukna.  The Effect of Null-Chains on the Complexity of Contact Schemes , 1989, FCT.

[36]  Stasys Jukna.  A nondeterministic space-time tradeoff for linear codes , 2009, Inf. Process. Lett..

[37]  Arnon Rosenthal.  Dynamic Programming is Optimal for Nonserial Optimization Problems , 1982, SIAM J. Comput..

[38]  J. Ian Munro,et al.  Efficient Determination of the Transitive Closure of a Directed Graph , 1971, Inf. Process. Lett..

[39]  M. Held,et al.  Finite-State Processes and Dynamic Programming , 1967 .

[40]  Gerhard J. Woeginger,et al.  When Does a Dynamic Programming Formulation Guarantee the Existence of a Fully Polynomial Time Approximation Scheme (FPTAS)? , 2000, INFORMS J. Comput..

[41]  Rajeev Motwani,et al.  On syntactic versus computational views of approximability , 1994, Proceedings 35th Annual Symposium on Foundations of Computer Science.

[42]  Matthias Poloczek.  Bounds on Greedy Algorithms for MAX SAT , 2011, ESA.