Combined analytical and empirical learning framework for branch and bound algorithms: the knapsack problem

Abstract Optimization methods are being applied to engineering problem solving with increasing frequency as computer hardware and software improves. The configuration of an optimization algorithm can make a significant difference to the efficiency of the solution process. This article examines the use of one such optimization strategy, branch and bound, for the solution of the classic knapsack problem. It is shown that the best configuration of the algorithm can be data dependent and hence that an ‘intelligent’ optimization system will need to automatically configure itself with the control knowledge appropriate to the problems the user is solving. A two-step approach is taken to configuring the algorithm. First, an analytical learning method, explanation based learning is used to derive a provably correct dominance condition for the knapsack problem. Second, the algorithm is configured with and without the condition, and subjected to a rigorous statistical test of performance, on the user's data, to decide which configuration is the best.

[1]  H. Teicher.  On the Mixture of Distributions , 1960 .

[2]  Donald L. Miller,et al.  Exact Solution of Large Asymmetric Traveling Salesman Problems , 1991, Science.

[3]  Ronald L. Rivest,et al.  Introduction to Algorithms , 1990 .

[4]  R. Gomory,et al.  A Linear Programming Approach to the Cutting-Stock Problem , 1961 .

[5]  H. Teicher.  Identifiability of Mixtures , 1961 .

[6]  New York Dover,et al.  ON THE CONVERGENCE PROPERTIES OF THE EM ALGORITHM , 1983 .

[7]  Paolo Toth,et al.  Knapsack Problems: Algorithms and Computer Implementations , 1990 .

[8]  E. Balas.  Disjunctive programming and a hierarchy of relaxations for discrete optimization problems , 1985 .

[9]  G. Nemhauser,et al.  Integer Programming , 2020 .

[10]  Toshihide Ibaraki,et al.  Branch-and-Bound Procedure and State-Space Representation of Combinatorial Optimization Problems , 1978, Inf. Control..

[11]  Ellis L. Johnson,et al.  Solving Large-Scale Zero-One Linear Programming Problems , 1983, Oper. Res..

[12]  William F. Clocksin,et al.  Programming in Prolog , 1987, Springer Berlin Heidelberg.

[13]  James R. Evans,et al.  Improving the performance of enumerative search methods - part II: Computational experiments , 1995, Comput. Oper. Res..

[14]  Adam Fadlalla,et al.  Improving the performance of enumerative search methods-I. Exploiting structure and intelligence , 1995, Comput. Oper. Res..

[15]  J. Wolfowitz,et al.  Introduction to the Theory of Statistics. , 1951 .

[16]  David S. Johnson,et al.  Computers and Intractability: A Guide to the Theory of NP-Completeness , 1978 .

[17]  William H. Press,et al.  The Art of Scientific Computing Second Edition , 1998 .

[18]  Der-Chiang Li,et al.  A strategy for evolution of alhorithms to increase the computational effectiveness of NP-hard scheduling problems , 1996 .

[19]  E. Lehmann.  Testing Statistical Hypotheses , 1960 .

[20]  Egon Balas,et al.  An Algorithm for Large Zero-One Knapsack Problems , 1980, Oper. Res..

[21]  Stuart C. Shapiro,et al.  Encyclopedia of artificial intelligence, vols. 1 and 2 (2nd ed.) , 1992 .

[22]  Benjamin W. Wah,et al.  Learning Dominance Relations in Combinatorial Search Problems , 1988, IEEE Trans. Software Eng..

[23]  R. Sargent,et al.  A general algorithm for short-term scheduling of batch operations—II. Computational issues , 1993 .

[24]  Leo Breiman,et al.  Classification and Regression Trees , 1984 .

[25]  D. Rubin,et al.  Maximum likelihood from incomplete data via the EM - algorithm plus discussions on the paper , 1977 .

[26]  Laurence A. Wolsey,et al.  Integer and Combinatorial Optimization , 1988 .

[27]  Ellis Horowitz,et al.  Computing Partitions with Applications to the Knapsack Problem , 1974, JACM.

[28]  Harvey J. Greenberg,et al.  New approaches for heuristic search: A bilateral linkage with artificial intelligence , 1989 .

[29]  George Stephanopoulos,et al.  On the Application of Explanation-Based Learning to Acquire Control Knowledge for Branch and Bound Algorithms , 1998, INFORMS J. Comput..

[30]  Paolo Toth,et al.  Upper Bounds and Algorithms for Hard 0-1 Knapsack Problems , 1997, Oper. Res..

[31]  Fred W. Glover,et al.  Future paths for integer programming and links to artificial intelligence , 1986, Comput. Oper. Res..

[32]  Umesh V. Vazirani,et al.  An Introduction to Computational Learning Theory , 1994 .

[33]  A. F. Smith,et al.  Statistical analysis of finite mixture distributions , 1986 .